Teaching Large Language Models an Unseen Language on the Fly

Teaching Large Language Models an Unseen Language on the Fly

13 Jun 2024 | Chen Zhang, Xiao Liu, Jiuuheng Lin, Yansong Feng*
The paper explores the feasibility of teaching large language models (LLMs) to learn and translate an entirely new, extremely low-resource language on the fly through prompting. The authors focus on the Zhuang language, which is not supported by any current LLMs and has minimal training data available. They introduce DiPMT++, a framework that enhances LLMs' performance in machine translation (MT) tasks for Zhuang using in-context learning (ICL). DiPMT++ leverages a dictionary and a 5K parallel corpus of Zhuang-Chinese sentences to improve lexical coverage and syntactic understanding. The framework significantly enhances GPT-4's performance, achieving 16 BLEU for Chinese-to-Zhuang translation and 32 BLEU for Zhuang-to-Chinese translation. The effectiveness of DiPMT++ is also validated on Kalamang, another low-resource language. Additionally, the authors demonstrate that DiPMT++ can assist humans in translating unseen languages, contributing to the preservation of linguistic diversity. The paper discusses the limitations and future directions, emphasizing the need for more comprehensive and precise understanding of syntax in low-resource languages.The paper explores the feasibility of teaching large language models (LLMs) to learn and translate an entirely new, extremely low-resource language on the fly through prompting. The authors focus on the Zhuang language, which is not supported by any current LLMs and has minimal training data available. They introduce DiPMT++, a framework that enhances LLMs' performance in machine translation (MT) tasks for Zhuang using in-context learning (ICL). DiPMT++ leverages a dictionary and a 5K parallel corpus of Zhuang-Chinese sentences to improve lexical coverage and syntactic understanding. The framework significantly enhances GPT-4's performance, achieving 16 BLEU for Chinese-to-Zhuang translation and 32 BLEU for Zhuang-to-Chinese translation. The effectiveness of DiPMT++ is also validated on Kalamang, another low-resource language. Additionally, the authors demonstrate that DiPMT++ can assist humans in translating unseen languages, contributing to the preservation of linguistic diversity. The paper discusses the limitations and future directions, emphasizing the need for more comprehensive and precise understanding of syntax in low-resource languages.
Reach us at info@study.space