18 May 2024 | Md. Ashraful Islam, Mohammed Eunus Ali, Md Rizwan Parvez
MapCoder is a multi-agent code generation framework designed for competitive problem-solving. It leverages four LLM agents—retrieval, planning, coding, and debugging—to emulate the human programming cycle. The retrieval agent generates relevant examples, the planning agent creates step-by-step plans, the coding agent translates plans into code, and the debugging agent fixes errors. The framework uses dynamic traversal to iteratively improve code generation by adjusting the confidence of generated plans as reward scores. MapCoder achieves state-of-the-art results on benchmarks like HumanEval (93.9%), MBPP (83.1%), APPS (22.0%), CodeContests (28.5%), and xCodeEval (45.3%). It outperforms existing methods across various programming languages and problem difficulties. The framework is open-sourced and evaluated using multiple LLMs, including ChatGPT, GPT-4, and Gemini Pro. Ablation studies show that the debugging agent has the most significant impact on performance, while the planning agent is the second most important. MapCoder also demonstrates strong performance across different programming languages and difficulty levels, and it is effective in handling complex algorithmic problems. The framework is designed to be robust and efficient, with performance gains achieved through careful tuning of hyperparameters like k and t. Overall, MapCoder provides a novel approach to code generation that significantly improves the accuracy and efficiency of code generation in competitive programming scenarios.MapCoder is a multi-agent code generation framework designed for competitive problem-solving. It leverages four LLM agents—retrieval, planning, coding, and debugging—to emulate the human programming cycle. The retrieval agent generates relevant examples, the planning agent creates step-by-step plans, the coding agent translates plans into code, and the debugging agent fixes errors. The framework uses dynamic traversal to iteratively improve code generation by adjusting the confidence of generated plans as reward scores. MapCoder achieves state-of-the-art results on benchmarks like HumanEval (93.9%), MBPP (83.1%), APPS (22.0%), CodeContests (28.5%), and xCodeEval (45.3%). It outperforms existing methods across various programming languages and problem difficulties. The framework is open-sourced and evaluated using multiple LLMs, including ChatGPT, GPT-4, and Gemini Pro. Ablation studies show that the debugging agent has the most significant impact on performance, while the planning agent is the second most important. MapCoder also demonstrates strong performance across different programming languages and difficulty levels, and it is effective in handling complex algorithmic problems. The framework is designed to be robust and efficient, with performance gains achieved through careful tuning of hyperparameters like k and t. Overall, MapCoder provides a novel approach to code generation that significantly improves the accuracy and efficiency of code generation in competitive programming scenarios.