The paper introduces MDAgents, a novel framework designed to enhance the utility of Large Language Models (LLMs) in complex medical decision-making by dynamically structuring effective collaboration models. MDAgents emulates real-world medical decision-making processes, adapting the collaboration structure (solo or group) based on the complexity of the medical task. The framework is evaluated across seven challenging medical benchmarks, achieving the best performance in five out of seven benchmarks that require multi-modal medical reasoning. Ablation studies reveal that MDAgents excels in adapting the number of collaborating agents to optimize efficiency and accuracy, showcasing its robustness in diverse scenarios. The framework also explores the dynamics of group consensus, providing insights into how collaborative agents behave in complex clinical team dynamics. The code for MDAgents is available at <https://github.com/mitmedialab/MDAgents>. Medical Decision-Making (MDM) involves clinicians navigating vast and diverse sources of information to arrive at precise conclusions under complexity. Large Language Models (LLMs) have shown potential in transforming MDM by digesting medical literature and clinical information, supporting probabilistic and causal reasoning processes crucial to medical practice. However, the severe implications of inaccuracies in healthcare, such as misdiagnoses and inappropriate treatments, demand a careful and precise approach. MDM also involves interpreting complex and multi-modal data, including imaging, electronic health records, signals, and genetic information, highlighting unique challenges in this field. MDAgents unfolds in four stages: 1) Medical Complexity Check - The system evaluates the intricacies of the medical query, stratifying it into low, moderate, or high categories. 2) Expert Recruitment - Depending on the complexity level, the framework activates appropriate diagnostic methods, either a simple solo approach or a multidisciplinary team (MDT) or integrated care team (ICT) for more complex scenarios. 3) Inference Process - For solo queries, the framework uses prompting techniques to provide answers. For MDTs, multiple LLM agents collaborate to form a consensus. For ICTs, diverse domain information is synthesized to produce a comprehensive report. 4) Decision Making - The final decision is synthesized from the outputs of different complexity levels, employing ensemble techniques and decision strategies like majority and weighted voting. The paper evaluates MDAgents using state-of-the-art LLMs across multiple medical benchmarks in Solo, Group, and Adaptive settings. Experiments highlight the framework's dynamic performance, demonstrating robustness and efficiency by modulating agent numbers and temperature. Results show a beneficial convergence of agent opinions in collaborative settings. Ablation studies further elucidate the individual contributions of agents and strategies within the system, revealing the critical components and interactions that drive the framework's success. The framework opens new possibilities for enhancing LLM-assisted medical diagnosis systems, pushing the boundaries of automated clinical reasoning.The paper introduces MDAgents, a novel framework designed to enhance the utility of Large Language Models (LLMs) in complex medical decision-making by dynamically structuring effective collaboration models. MDAgents emulates real-world medical decision-making processes, adapting the collaboration structure (solo or group) based on the complexity of the medical task. The framework is evaluated across seven challenging medical benchmarks, achieving the best performance in five out of seven benchmarks that require multi-modal medical reasoning. Ablation studies reveal that MDAgents excels in adapting the number of collaborating agents to optimize efficiency and accuracy, showcasing its robustness in diverse scenarios. The framework also explores the dynamics of group consensus, providing insights into how collaborative agents behave in complex clinical team dynamics. The code for MDAgents is available at <https://github.com/mitmedialab/MDAgents>. Medical Decision-Making (MDM) involves clinicians navigating vast and diverse sources of information to arrive at precise conclusions under complexity. Large Language Models (LLMs) have shown potential in transforming MDM by digesting medical literature and clinical information, supporting probabilistic and causal reasoning processes crucial to medical practice. However, the severe implications of inaccuracies in healthcare, such as misdiagnoses and inappropriate treatments, demand a careful and precise approach. MDM also involves interpreting complex and multi-modal data, including imaging, electronic health records, signals, and genetic information, highlighting unique challenges in this field. MDAgents unfolds in four stages: 1) Medical Complexity Check - The system evaluates the intricacies of the medical query, stratifying it into low, moderate, or high categories. 2) Expert Recruitment - Depending on the complexity level, the framework activates appropriate diagnostic methods, either a simple solo approach or a multidisciplinary team (MDT) or integrated care team (ICT) for more complex scenarios. 3) Inference Process - For solo queries, the framework uses prompting techniques to provide answers. For MDTs, multiple LLM agents collaborate to form a consensus. For ICTs, diverse domain information is synthesized to produce a comprehensive report. 4) Decision Making - The final decision is synthesized from the outputs of different complexity levels, employing ensemble techniques and decision strategies like majority and weighted voting. The paper evaluates MDAgents using state-of-the-art LLMs across multiple medical benchmarks in Solo, Group, and Adaptive settings. Experiments highlight the framework's dynamic performance, demonstrating robustness and efficiency by modulating agent numbers and temperature. Results show a beneficial convergence of agent opinions in collaborative settings. Ablation studies further elucidate the individual contributions of agents and strategies within the system, revealing the critical components and interactions that drive the framework's success. The framework opens new possibilities for enhancing LLM-assisted medical diagnosis systems, pushing the boundaries of automated clinical reasoning.