BiMediX: Bilingual Medical Mixture of Experts LLM

BiMediX: Bilingual Medical Mixture of Experts LLM

2024 | Sara Pieri, Sahal Shaji Mullappilly, Fahad Shahbaz Khan, Rao Muhammad Anwer, Salman Khan, Timothy Baldwin, Hisham Cholakkal
BiMediX is the first bilingual medical mixture of experts large language model (LLM) designed for seamless interaction in both English and Arabic. The model supports a wide range of medical interactions, including multi-turn chats, multiple-choice question answering, and open-ended question answering. It features a semi-automated English-to-Arabic translation pipeline with human refinement to ensure high-quality translations. The model also includes a comprehensive evaluation benchmark for Arabic medical LLMs and a large bilingual instruction set called BiMed1.3M, which contains over 1.3 million diverse medical interactions, resulting in over 632 million healthcare-specialized tokens for instruction tuning. BiMed1.3M includes 250,000 synthesized multi-turn doctor-patient chats and maintains a 1:2 Arabic-to-English ratio. BiMediX outperforms state-of-the-art models like Med42 and Meditron by 2.5% and 4.1%, respectively, across multiple medical evaluation benchmarks in English, while operating 8 times faster. It also outperforms the generic Arabic-English bilingual LLM, Jais-30B, by 10% on Arabic medical benchmarks and 15% on bilingual evaluations across multiple datasets. The model achieves state-of-the-art performance on multiple medical exam question datasets in both English and Arabic. BiMediX is a chat agent with state-of-the-art performance on non-chat benchmarks in both English and Arabic. The model is trained on the BiMed1.3M dataset, which includes 1.3 million diverse medical interactions, and is designed for bilingual medical interactions in both English and Arabic. The model is capable of handling multiple-choice question answering, open-ended question answering, and realistic multi-turn question answering between the model and the user. BiMediX is designed to facilitate comprehensive medical interactions, including conversations, MCQA, and QA in both English and Arabic. The model is trained on a bilingual instruction set that includes 1.3 million diverse medical interactions, including chats, and is designed for bilingual medical interactions in both English and Arabic. The model is capable of handling multiple-choice question answering, open-ended question answering, and realistic multi-turn question answering between the model and the user. BiMediX is designed to facilitate comprehensive medical interactions, including conversations, MCQA, and QA in both English and Arabic.BiMediX is the first bilingual medical mixture of experts large language model (LLM) designed for seamless interaction in both English and Arabic. The model supports a wide range of medical interactions, including multi-turn chats, multiple-choice question answering, and open-ended question answering. It features a semi-automated English-to-Arabic translation pipeline with human refinement to ensure high-quality translations. The model also includes a comprehensive evaluation benchmark for Arabic medical LLMs and a large bilingual instruction set called BiMed1.3M, which contains over 1.3 million diverse medical interactions, resulting in over 632 million healthcare-specialized tokens for instruction tuning. BiMed1.3M includes 250,000 synthesized multi-turn doctor-patient chats and maintains a 1:2 Arabic-to-English ratio. BiMediX outperforms state-of-the-art models like Med42 and Meditron by 2.5% and 4.1%, respectively, across multiple medical evaluation benchmarks in English, while operating 8 times faster. It also outperforms the generic Arabic-English bilingual LLM, Jais-30B, by 10% on Arabic medical benchmarks and 15% on bilingual evaluations across multiple datasets. The model achieves state-of-the-art performance on multiple medical exam question datasets in both English and Arabic. BiMediX is a chat agent with state-of-the-art performance on non-chat benchmarks in both English and Arabic. The model is trained on the BiMed1.3M dataset, which includes 1.3 million diverse medical interactions, and is designed for bilingual medical interactions in both English and Arabic. The model is capable of handling multiple-choice question answering, open-ended question answering, and realistic multi-turn question answering between the model and the user. BiMediX is designed to facilitate comprehensive medical interactions, including conversations, MCQA, and QA in both English and Arabic. The model is trained on a bilingual instruction set that includes 1.3 million diverse medical interactions, including chats, and is designed for bilingual medical interactions in both English and Arabic. The model is capable of handling multiple-choice question answering, open-ended question answering, and realistic multi-turn question answering between the model and the user. BiMediX is designed to facilitate comprehensive medical interactions, including conversations, MCQA, and QA in both English and Arabic.
Reach us at info@futurestudyspace.com