TURNA: A Turkish Encoder-Decoder Language Model for Enhanced Understanding and Generation

TURNA: A Turkish Encoder-Decoder Language Model for Enhanced Understanding and Generation

25 Jan 2024 | Gökçe Uludoğan, Zeynep Yirmibeşoğlu Balal, Furkan Akkurt, Melikşah Türker, Onur Güngör, Susan Üsküdarlı
The paper introduces TURNA, a Turkish encoder-decoder language model designed to enhance understanding and generation capabilities for the low-resource language Turkish. TURNA is pre-trained using an encoder-decoder architecture based on the UL2 framework with a diverse corpus curated specifically for this purpose. The model is evaluated on three generation tasks and five understanding tasks for Turkish, outperforming several multilingual models in both areas and competing with monolingual Turkish models in understanding tasks. The paper details the methodology, including the pretraining objectives, data collection, and fine-tuning processes. It also provides a comprehensive evaluation of TURNA's performance across various downstream tasks, demonstrating its effectiveness in natural language processing tasks such as paraphrasing, summarization, named entity recognition, part-of-speech tagging, and text classification. The authors acknowledge the limitations of the model, particularly in efficiency and the need for further pretraining, and encourage further research and benchmarking in Turkish NLP.The paper introduces TURNA, a Turkish encoder-decoder language model designed to enhance understanding and generation capabilities for the low-resource language Turkish. TURNA is pre-trained using an encoder-decoder architecture based on the UL2 framework with a diverse corpus curated specifically for this purpose. The model is evaluated on three generation tasks and five understanding tasks for Turkish, outperforming several multilingual models in both areas and competing with monolingual Turkish models in understanding tasks. The paper details the methodology, including the pretraining objectives, data collection, and fine-tuning processes. It also provides a comprehensive evaluation of TURNA's performance across various downstream tasks, demonstrating its effectiveness in natural language processing tasks such as paraphrasing, summarization, named entity recognition, part-of-speech tagging, and text classification. The authors acknowledge the limitations of the model, particularly in efficiency and the need for further pretraining, and encourage further research and benchmarking in Turkish NLP.
Reach us at info@study.space
[slides] TURNA%3A A Turkish Encoder-Decoder Language Model for Enhanced Understanding and Generation | StudySpace