The paper "More Agents Is All You Need" explores the effectiveness of increasing the number of instantiated agents in large language models (LLMs) to improve their performance. The authors find that a simple sampling-and-voting method, where multiple LLM instances generate answers and the most consistent answer is selected, scales well with the number of agents. This approach is orthogonal to existing complex methods and can enhance their performance, especially in tasks with varying difficulty levels. Comprehensive experiments on various benchmarks show that adding more agents can lead to significant accuracy improvements, even for smaller LLMs compared to larger ones. The study also analyzes the correlation between task difficulty and performance gains, identifying three dimensions: inherent difficulty, the number of reasoning steps, and the prior probability of the correct answer. Based on these findings, the authors propose optimization strategies to further enhance the effectiveness of the sampling-and-voting method. The paper concludes by highlighting the potential risks associated with LLMs, such as hallucinations, and emphasizes the need for responsible deployment.The paper "More Agents Is All You Need" explores the effectiveness of increasing the number of instantiated agents in large language models (LLMs) to improve their performance. The authors find that a simple sampling-and-voting method, where multiple LLM instances generate answers and the most consistent answer is selected, scales well with the number of agents. This approach is orthogonal to existing complex methods and can enhance their performance, especially in tasks with varying difficulty levels. Comprehensive experiments on various benchmarks show that adding more agents can lead to significant accuracy improvements, even for smaller LLMs compared to larger ones. The study also analyzes the correlation between task difficulty and performance gains, identifying three dimensions: inherent difficulty, the number of reasoning steps, and the prior probability of the correct answer. Based on these findings, the authors propose optimization strategies to further enhance the effectiveness of the sampling-and-voting method. The paper concludes by highlighting the potential risks associated with LLMs, such as hallucinations, and emphasizes the need for responsible deployment.