TAS TE: Teaching Large Language Models to Translate through Self-Reflection

TAS TE: Teaching Large Language Models to Translate through Self-Reflection

12 Jun 2024 | Yutong Wang, Jiali Zeng, Xuebo Liu, Fandong Meng, Jie Zhou, Min Zhang
The TASTE framework enables large language models (LLMs) to improve translation performance through self-reflection. It involves two stages of inference: first, LLMs generate preliminary translations and self-assess their quality. Second, they refine these translations based on the evaluation results. This approach outperforms existing methods in translation quality, as demonstrated by results on the WMT22 benchmark. The framework uses a multitask training dataset to enhance LLMs' ability to perform the entire reflective translation process. The TASTE method includes a two-stage inference process, where LLMs first generate translations and then refine them based on self-assessment. The method also incorporates a multitask fine-tuning strategy to improve translation performance. The results show that TASTE significantly enhances translation quality, particularly in German-English and Chinese-English directions. The framework is effective in refining translations and reducing errors, as evidenced by improvements in COMET scores and reductions in unaligned translation words. The TASTE method also demonstrates the effectiveness of quality labels in guiding the refinement process. The framework is compared with other methods, including CoT and ICL, and shows superior performance in many translation directions. The TASTE framework is also shown to be effective as an Automatic Post-Editing (APE) tool, improving the quality of translations generated by other systems. The study highlights the potential of LLMs in translation tasks and the importance of self-reflection in enhancing translation quality. The framework is supported by extensive experiments and analysis, demonstrating its effectiveness in improving translation performance. The study also identifies limitations, including performance inconsistencies across different translation directions and increased computational costs due to the two-stage inference process. Overall, the TASTE framework provides a promising approach to enhance the translation capabilities of LLMs through self-reflection.The TASTE framework enables large language models (LLMs) to improve translation performance through self-reflection. It involves two stages of inference: first, LLMs generate preliminary translations and self-assess their quality. Second, they refine these translations based on the evaluation results. This approach outperforms existing methods in translation quality, as demonstrated by results on the WMT22 benchmark. The framework uses a multitask training dataset to enhance LLMs' ability to perform the entire reflective translation process. The TASTE method includes a two-stage inference process, where LLMs first generate translations and then refine them based on self-assessment. The method also incorporates a multitask fine-tuning strategy to improve translation performance. The results show that TASTE significantly enhances translation quality, particularly in German-English and Chinese-English directions. The framework is effective in refining translations and reducing errors, as evidenced by improvements in COMET scores and reductions in unaligned translation words. The TASTE method also demonstrates the effectiveness of quality labels in guiding the refinement process. The framework is compared with other methods, including CoT and ICL, and shows superior performance in many translation directions. The TASTE framework is also shown to be effective as an Automatic Post-Editing (APE) tool, improving the quality of translations generated by other systems. The study highlights the potential of LLMs in translation tasks and the importance of self-reflection in enhancing translation quality. The framework is supported by extensive experiments and analysis, demonstrating its effectiveness in improving translation performance. The study also identifies limitations, including performance inconsistencies across different translation directions and increased computational costs due to the two-stage inference process. Overall, the TASTE framework provides a promising approach to enhance the translation capabilities of LLMs through self-reflection.
Reach us at info@study.space