MAGIS: An LLM-Based Multi-Agent Framework for GitHub Issue Resolution This paper presents MAGIS, a novel LLM-based multi-agent framework for resolving GitHub issues. The framework consists of four agents: Manager, Repository Custodian, Developer, and Quality Assurance Engineer. These agents collaborate to enhance the resolution of GitHub issues, which involves not only incorporating new code but also maintaining existing code. The study shows that LLMs face challenges in resolving GitHub issues, particularly at the repository level. MAGIS leverages the collaboration of various agents in the planning and coding process to unlock the potential of LLMs to resolve GitHub issues. In experiments, MAGIS can resolve 13.94% of GitHub issues, significantly outperforming the baselines. Specifically, MAGIS achieves an eight-fold increase in resolved ratio over the direct application of GPT-4. The framework's effectiveness is attributed to the planning of code change, locating lines within the code file, and code review process. The main contributions of this study include an empirical analysis of LLMs in resolving GitHub issues, the proposal of a novel LLM-based multi-agent framework, and a comparison of the framework with other strong LLM competitors. The results show that MAGIS significantly outperforms these competitors. The framework's design is effective and necessary for resolving GitHub issues. The study also identifies key factors affecting LLMs' issue resolution, including the complexity of the code change, the number of modified files, and the accuracy of line localization. The framework's performance is evaluated on the SWE-bench dataset, which is the latest dataset specifically designed for evaluating the performance of GitHub issue resolution. The results demonstrate that MAGIS significantly outperforms other LLMs in resolving GitHub issues. The framework's effectiveness is attributed to the collaboration of various agents in the planning and coding process, which enables the resolution of GitHub issues more efficiently. The study also highlights the importance of code review in ensuring the quality of the code changes. The framework's design is effective and necessary for resolving GitHub issues, and it provides a promising direction for integrating LLMs into software evolution workflows.MAGIS: An LLM-Based Multi-Agent Framework for GitHub Issue Resolution This paper presents MAGIS, a novel LLM-based multi-agent framework for resolving GitHub issues. The framework consists of four agents: Manager, Repository Custodian, Developer, and Quality Assurance Engineer. These agents collaborate to enhance the resolution of GitHub issues, which involves not only incorporating new code but also maintaining existing code. The study shows that LLMs face challenges in resolving GitHub issues, particularly at the repository level. MAGIS leverages the collaboration of various agents in the planning and coding process to unlock the potential of LLMs to resolve GitHub issues. In experiments, MAGIS can resolve 13.94% of GitHub issues, significantly outperforming the baselines. Specifically, MAGIS achieves an eight-fold increase in resolved ratio over the direct application of GPT-4. The framework's effectiveness is attributed to the planning of code change, locating lines within the code file, and code review process. The main contributions of this study include an empirical analysis of LLMs in resolving GitHub issues, the proposal of a novel LLM-based multi-agent framework, and a comparison of the framework with other strong LLM competitors. The results show that MAGIS significantly outperforms these competitors. The framework's design is effective and necessary for resolving GitHub issues. The study also identifies key factors affecting LLMs' issue resolution, including the complexity of the code change, the number of modified files, and the accuracy of line localization. The framework's performance is evaluated on the SWE-bench dataset, which is the latest dataset specifically designed for evaluating the performance of GitHub issue resolution. The results demonstrate that MAGIS significantly outperforms other LLMs in resolving GitHub issues. The framework's effectiveness is attributed to the collaboration of various agents in the planning and coding process, which enables the resolution of GitHub issues more efficiently. The study also highlights the importance of code review in ensuring the quality of the code changes. The framework's design is effective and necessary for resolving GitHub issues, and it provides a promising direction for integrating LLMs into software evolution workflows.