This paper introduces a dynamic expert selection framework for Mixture of Experts (MoE) models, aiming to enhance computational efficiency and model performance by adjusting the number of activated experts based on input difficulty. Unlike traditional MoE approaches that use fixed Top-K routing, which activates a predetermined number of experts regardless of input complexity, the proposed method dynamically selects experts based on the confidence level in expert selection for each input. This allows for more efficient use of computational resources, activating more experts for complex tasks requiring advanced reasoning and fewer for simpler tasks. The method is evaluated across various benchmarks, achieving an average improvement of 0.7% with less than 90% activated parameters. Analysis shows that the model dispatches more experts to tasks requiring complex reasoning, such as BBH, confirming its ability to dynamically allocate computational resources. The findings also highlight variations in the number of experts needed across different layers of the transformer model, offering insights into designing heterogeneous MoE frameworks. The code and models are available at https://github.com/ZhenweiAn/Dynamic_MoE. The method outperforms conventional Top-K routing in terms of efficiency and performance, demonstrating its effectiveness in dynamically allocating experts based on input complexity. The dynamic routing mechanism is efficient in both training and inference, outperforming Top-2 routing while activating fewer experts. The study also reveals that the number of experts needed varies across different layers of the transformer model, suggesting potential for designing heterogeneous MoE frameworks. The dynamic routing mechanism is efficient in both training and inference, outperforming Top-2 routing while activating fewer experts. The method is evaluated across various benchmarks, achieving an average improvement of 0.7% with less than 90% activated parameters. The findings also highlight variations in the number of experts needed across different layers of the transformer model, offering insights into designing heterogeneous MoE frameworks. The code and models are available at https://github.com/ZhenweiAn/Dynamic_MoE.This paper introduces a dynamic expert selection framework for Mixture of Experts (MoE) models, aiming to enhance computational efficiency and model performance by adjusting the number of activated experts based on input difficulty. Unlike traditional MoE approaches that use fixed Top-K routing, which activates a predetermined number of experts regardless of input complexity, the proposed method dynamically selects experts based on the confidence level in expert selection for each input. This allows for more efficient use of computational resources, activating more experts for complex tasks requiring advanced reasoning and fewer for simpler tasks. The method is evaluated across various benchmarks, achieving an average improvement of 0.7% with less than 90% activated parameters. Analysis shows that the model dispatches more experts to tasks requiring complex reasoning, such as BBH, confirming its ability to dynamically allocate computational resources. The findings also highlight variations in the number of experts needed across different layers of the transformer model, offering insights into designing heterogeneous MoE frameworks. The code and models are available at https://github.com/ZhenweiAn/Dynamic_MoE. The method outperforms conventional Top-K routing in terms of efficiency and performance, demonstrating its effectiveness in dynamically allocating experts based on input complexity. The dynamic routing mechanism is efficient in both training and inference, outperforming Top-2 routing while activating fewer experts. The study also reveals that the number of experts needed varies across different layers of the transformer model, suggesting potential for designing heterogeneous MoE frameworks. The dynamic routing mechanism is efficient in both training and inference, outperforming Top-2 routing while activating fewer experts. The method is evaluated across various benchmarks, achieving an average improvement of 0.7% with less than 90% activated parameters. The findings also highlight variations in the number of experts needed across different layers of the transformer model, offering insights into designing heterogeneous MoE frameworks. The code and models are available at https://github.com/ZhenweiAn/Dynamic_MoE.