NLP
1.[NLP #1] mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs (EMNLP, 2021)
1 Introduction & Summary Challenge seq2seq 모델인 mT5는 cross-lingual task에서 괄목할만한 성능 보여줌 MT5가 이미 성능이 좋은데, 어떻게 하면 tranlation data 사용해서 MT5를 뛰어넘을 수 있을까
2.[NLP #2] Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation (ACL, 2022)
Summary Background: Neural Machine Translation (NMT) task에서는 training 데이터 수에 따라서 generaliation performance가 달라진다. 그래서 training data를 늘리기 위해 Augmen
3.[NLP #3] SimCSE: Simple Contrastive Learning of Sentence Embeddings (EMNLP, 2021)
한줄 요약: unlabeled or labeled data 모두에서 setence embedding 뽑을 수 있다? Paper: https://aclanthology.org/2021.emnlp-main.552/Code: https://github.co
4.[NLP #4] BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension (ACL, 2020)
Paper: https://aclanthology.org/2020.acl-main.703/ > Code: > - Ko-BART : https://github.com/SKT-AI/KoBART
5.[NLP #5] Mutual Information Alleviates Hallucinations in Abstractive Summarization (EMNLP, 2022)
paper: https://aclanthology.org/2022.emnlp-main.399.pdfcode: https://github.com/VanderpoelLiam/CPMIBackground: hallucinate: 요약한게 source docu
6.[NLP #6] How to Adapt Your Pretrained Multilingual Model to 1600 Languages (ACL, 2021)
Goal: 1600가지 언어 데이터셋을 활용해서 기존의 PMM 모델들 performance 검증Challenge: 1) 소량 2) narrow domainResult: XLM-R good
7.[NLP #8] How Far are We from Robust Long Abstractive Summarization? (EMNLP, 2022)
goal: abstractive summarization system의 수행결과를 human annotation 결과랑 비교해서 long document에 대해 얼마나 잘했는지 평가해본다. result: 비교해보니까 ROUGE result에서는 굿이었음. 그러니까 re
8.[NLP #9] On Transferability of Prompt Tuning for Natural Language Processing (NAACL, 2022)
paper: https://aclanthology.org/2022.naacl-main.290/ > velog: https://velog.io/@zvezda/On-Transferability-of-Prompt-Tuning-for-Natural-Language-Proces