Medical mT5: An Open-Source Multilingual Text-to-Text LLM for The Medical Domain

Medical mT5: An Open-Source Multilingual Text-to-Text LLM for The Medical Domain

11 Apr 2024 | Iker Garcia-Ferrero, Rodrigo Agerri, Aitziber Atutxa, Elena Cabrio, Iker de la Iglesia, Alberto Lavelli, Bernardo Magnini, Benjamin Molinet, Johana Ramirez-Romero, German Rigau, Jose Maria Villa-Gonzalez, Serena Villata, Andrea Zaninello
Medical mT5 is an open-source multilingual text-to-text large language model (LLM) specifically designed for the medical domain. The paper introduces the model, which is trained on a large multilingual corpus of medical texts in English, French, Italian, and Spanish, totaling 3 billion words. This corpus was compiled to address the lack of high-quality multilingual evaluation benchmarks in the medical field. The model, built upon the mT5 framework, is trained on medical domain data and is evaluated on four languages. It outperforms existing models in Spanish, French, and Italian benchmarks and is competitive with state-of-the-art models in English. The paper also presents two new evaluation benchmarks for the medical domain, focusing on argument mining and generative question answering. Medical mT5 is a versatile model that can be used for various sequence labeling tasks and question answering. The model is also noted for its relatively low hardware requirements, making it accessible for a wide range of users. The paper also discusses the challenges of acquiring medical domain data, especially for non-English languages, and highlights the importance of multilingual research in the medical field. The results show that Medical mT5 performs well in both multi-task and zero-shot cross-lingual settings, demonstrating its effectiveness in the medical domain. The model is publicly available, allowing for further research and development in the field of medical language processing.Medical mT5 is an open-source multilingual text-to-text large language model (LLM) specifically designed for the medical domain. The paper introduces the model, which is trained on a large multilingual corpus of medical texts in English, French, Italian, and Spanish, totaling 3 billion words. This corpus was compiled to address the lack of high-quality multilingual evaluation benchmarks in the medical field. The model, built upon the mT5 framework, is trained on medical domain data and is evaluated on four languages. It outperforms existing models in Spanish, French, and Italian benchmarks and is competitive with state-of-the-art models in English. The paper also presents two new evaluation benchmarks for the medical domain, focusing on argument mining and generative question answering. Medical mT5 is a versatile model that can be used for various sequence labeling tasks and question answering. The model is also noted for its relatively low hardware requirements, making it accessible for a wide range of users. The paper also discusses the challenges of acquiring medical domain data, especially for non-English languages, and highlights the importance of multilingual research in the medical field. The results show that Medical mT5 performs well in both multi-task and zero-shot cross-lingual settings, demonstrating its effectiveness in the medical domain. The model is publicly available, allowing for further research and development in the field of medical language processing.
Reach us at info@study.space