This paper introduces MDAgents, an adaptive collaboration framework for large language models (LLMs) in medical decision-making. The framework dynamically assigns the appropriate collaboration structure for LLMs based on the complexity of the medical task, emulating real-world medical decision-making processes. MDAgents evaluates its performance across seven medical benchmarks: MedQA, MedMCQA, PubMedQA, DDXPlus, PMC-VQA, Path-VQA, and MedVidQA, achieving the best performance in five out of seven benchmarks that require multi-modal medical reasoning. Ablation studies show that MDAgents excels in adapting the number of collaborating agents to optimize efficiency and accuracy, demonstrating robustness in diverse scenarios. The framework also explores group consensus dynamics, offering insights into how collaborative agents could behave in complex clinical team settings. The code is available at https://github.com/mitmedialab/MDAgents. Medical decision-making (MDM) is a complex process where clinicians navigate vast and diverse information to arrive at precise conclusions. LLMs have shown potential in transforming MDM by digesting medical literature and clinical information, supporting probabilistic and causal reasoning. However, their performance in healthcare has been limited due to their generalist design, which lacks specialized medical knowledge. In contrast, human clinicians use an adaptive, collaborative, and tiered approach to MDM, considering patient history, medical literature, and clinical expertise. MDAgents unfolds in four stages: 1) Medical Complexity Check - Evaluates the intricacies of the medical query, stratifying the problem into low, moderate, or high categories. 2) Expert Recruitment - Activates appropriate diagnostic methods based on the complexity level. 3) Inference Process - Uses prompting techniques for solo queries or collaborative discussions for MDT. 4) Decision Making - Synthesizes diverse inputs to arrive at a well-informed final answer. MDAgents is the first adaptive decision-making framework for LLMs that mirrors real-world MDM processes, allowing dynamic collaboration among AI agents based on task complexity. Experiments on seven medical question-answering datasets show superior performance in accuracy over previous solo and group methods on five out of seven benchmarks. The framework provides an effective trade-off between performance and API calls by varying the number of agents. Rigorous testing under various temperatures demonstrates better robustness than solo and group methods. Ablation studies show that the framework finds the appropriate complexity level for each MDM instance. The framework outperforms solo and group settings in multiple medical benchmarks, particularly in text-only and text-image datasets. It comprehends textual information with high precision and synthesizes visual data, a crucial capability in medical diagnostics. The adaptive approach selects the optimal complexity level for each MDM problem, supported by the model's ability to match complexity levels. The framework's performance is robust across different temperatures and modalities, demonstrating its effectiveness in MDM contexts. The adaptive approach avoids the shortcomings of solo and group settingsThis paper introduces MDAgents, an adaptive collaboration framework for large language models (LLMs) in medical decision-making. The framework dynamically assigns the appropriate collaboration structure for LLMs based on the complexity of the medical task, emulating real-world medical decision-making processes. MDAgents evaluates its performance across seven medical benchmarks: MedQA, MedMCQA, PubMedQA, DDXPlus, PMC-VQA, Path-VQA, and MedVidQA, achieving the best performance in five out of seven benchmarks that require multi-modal medical reasoning. Ablation studies show that MDAgents excels in adapting the number of collaborating agents to optimize efficiency and accuracy, demonstrating robustness in diverse scenarios. The framework also explores group consensus dynamics, offering insights into how collaborative agents could behave in complex clinical team settings. The code is available at https://github.com/mitmedialab/MDAgents. Medical decision-making (MDM) is a complex process where clinicians navigate vast and diverse information to arrive at precise conclusions. LLMs have shown potential in transforming MDM by digesting medical literature and clinical information, supporting probabilistic and causal reasoning. However, their performance in healthcare has been limited due to their generalist design, which lacks specialized medical knowledge. In contrast, human clinicians use an adaptive, collaborative, and tiered approach to MDM, considering patient history, medical literature, and clinical expertise. MDAgents unfolds in four stages: 1) Medical Complexity Check - Evaluates the intricacies of the medical query, stratifying the problem into low, moderate, or high categories. 2) Expert Recruitment - Activates appropriate diagnostic methods based on the complexity level. 3) Inference Process - Uses prompting techniques for solo queries or collaborative discussions for MDT. 4) Decision Making - Synthesizes diverse inputs to arrive at a well-informed final answer. MDAgents is the first adaptive decision-making framework for LLMs that mirrors real-world MDM processes, allowing dynamic collaboration among AI agents based on task complexity. Experiments on seven medical question-answering datasets show superior performance in accuracy over previous solo and group methods on five out of seven benchmarks. The framework provides an effective trade-off between performance and API calls by varying the number of agents. Rigorous testing under various temperatures demonstrates better robustness than solo and group methods. Ablation studies show that the framework finds the appropriate complexity level for each MDM instance. The framework outperforms solo and group settings in multiple medical benchmarks, particularly in text-only and text-image datasets. It comprehends textual information with high precision and synthesizes visual data, a crucial capability in medical diagnostics. The adaptive approach selects the optimal complexity level for each MDM problem, supported by the model's ability to match complexity levels. The framework's performance is robust across different temperatures and modalities, demonstrating its effectiveness in MDM contexts. The adaptive approach avoids the shortcomings of solo and group settings