How do Large Language Models Handle Multilingualism?

How do Large Language Models Handle Multilingualism?

24 May 2024 | Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi, Lidong Bing
This study explores how large language models (LLMs) handle multilingualism. The research proposes a multilingual workflow (MWork) that suggests LLMs first understand queries by converting multilingual inputs into English, think in English in intermediate layers while incorporating multilingual knowledge, and generate responses aligned with the original language in the final layers. To validate MWork, the authors introduce Parallel Language-specific Neuron Detection (PLND), a method that identifies activated neurons for different languages without labeled data. Using PLND, they verify MWork through extensive experiments involving the deactivation of language-specific neurons across various layers and structures. The results show that deactivating language-specific neurons significantly impacts multilingual performance, with an average improvement of 3.6% for high-resource languages and 2.3% for low-resource languages across all tasks with just 400 documents. The study also demonstrates that MWork allows for the fine-tuning of language-specific neurons with a small dataset, enhancing multilingual abilities in a specific language without compromising others. The findings reveal that LLMs rely on English for thinking while extracting multilingual knowledge to support query processing. The research further confirms that the first several layers are responsible for understanding, as deactivating neurons in these layers disables LLMs on the NLU task in non-English languages. Additionally, deactivating language-specific neurons in the task-solving layer shows that LLMs rely on English, as performance drops across all languages. The study also verifies the functionality of the self-attention and feed-forward structures in the task-solving layer, showing that disabling the self-attention structure compromises the ability to solve tasks across all languages. The results further confirm the effectiveness of MWork in interpreting structure functionality for LLM's multilingual query handling, offering precise and independent methods for multilingual enhancement. The study concludes that MWork provides a framework for enhancing multilingual abilities in LLMs, with significant improvements in performance across various tasks.This study explores how large language models (LLMs) handle multilingualism. The research proposes a multilingual workflow (MWork) that suggests LLMs first understand queries by converting multilingual inputs into English, think in English in intermediate layers while incorporating multilingual knowledge, and generate responses aligned with the original language in the final layers. To validate MWork, the authors introduce Parallel Language-specific Neuron Detection (PLND), a method that identifies activated neurons for different languages without labeled data. Using PLND, they verify MWork through extensive experiments involving the deactivation of language-specific neurons across various layers and structures. The results show that deactivating language-specific neurons significantly impacts multilingual performance, with an average improvement of 3.6% for high-resource languages and 2.3% for low-resource languages across all tasks with just 400 documents. The study also demonstrates that MWork allows for the fine-tuning of language-specific neurons with a small dataset, enhancing multilingual abilities in a specific language without compromising others. The findings reveal that LLMs rely on English for thinking while extracting multilingual knowledge to support query processing. The research further confirms that the first several layers are responsible for understanding, as deactivating neurons in these layers disables LLMs on the NLU task in non-English languages. Additionally, deactivating language-specific neurons in the task-solving layer shows that LLMs rely on English, as performance drops across all languages. The study also verifies the functionality of the self-attention and feed-forward structures in the task-solving layer, showing that disabling the self-attention structure compromises the ability to solve tasks across all languages. The results further confirm the effectiveness of MWork in interpreting structure functionality for LLM's multilingual query handling, offering precise and independent methods for multilingual enhancement. The study concludes that MWork provides a framework for enhancing multilingual abilities in LLMs, with significant improvements in performance across various tasks.
Reach us at info@study.space