MFTCoder is a multi-task fine-tuning framework designed to enhance the performance of code large language models (Code LLMs) by simultaneously adapting them to multiple coding tasks. The framework addresses common challenges in multi-task learning, such as data imbalance, varying task difficulty, and inconsistent convergence speeds, through the use of diverse loss functions and parameter-efficient fine-tuning (PEFT) techniques like LoRA and QLoRA. MFTCoder enables efficient training with optimized data tokenization and parameter management, allowing it to handle a wide range of code-related tasks effectively. The framework was tested on various code-related tasks, including code completion, text-to-code generation, code comment generation, code translation, and unit test case generation. Experimental results showed that MFTCoder outperformed both individual fine-tuning and mixed-task fine-tuning approaches. Specifically, the MFTCoder-fine-tuned CodeFuse-DeepSeek-33B model achieved the top spot on the Big Code Models Leaderboard as of January 30, 2024. MFTCoder is open-sourced and compatible with several mainstream open-source LLMs, including CodeLLama and Qwen. It has been applied to develop CodeFuse, a programming assistant with web and IDE plugins, which has 12,000 weekly active users and generates nearly 80,000 lines of code weekly. The framework's ability to handle diverse tasks and improve performance through shared knowledge and patterns makes it a valuable tool for code-related applications. MFTCoder's approach to multi-task learning also addresses the issue of varying convergence speeds by dynamically adjusting task weights based on validation loss gradients, leading to more balanced and efficient training. The framework's effectiveness has been validated through extensive experiments and comparisons with other models, demonstrating its superiority in code generation and related tasks.MFTCoder is a multi-task fine-tuning framework designed to enhance the performance of code large language models (Code LLMs) by simultaneously adapting them to multiple coding tasks. The framework addresses common challenges in multi-task learning, such as data imbalance, varying task difficulty, and inconsistent convergence speeds, through the use of diverse loss functions and parameter-efficient fine-tuning (PEFT) techniques like LoRA and QLoRA. MFTCoder enables efficient training with optimized data tokenization and parameter management, allowing it to handle a wide range of code-related tasks effectively. The framework was tested on various code-related tasks, including code completion, text-to-code generation, code comment generation, code translation, and unit test case generation. Experimental results showed that MFTCoder outperformed both individual fine-tuning and mixed-task fine-tuning approaches. Specifically, the MFTCoder-fine-tuned CodeFuse-DeepSeek-33B model achieved the top spot on the Big Code Models Leaderboard as of January 30, 2024. MFTCoder is open-sourced and compatible with several mainstream open-source LLMs, including CodeLLama and Qwen. It has been applied to develop CodeFuse, a programming assistant with web and IDE plugins, which has 12,000 weekly active users and generates nearly 80,000 lines of code weekly. The framework's ability to handle diverse tasks and improve performance through shared knowledge and patterns makes it a valuable tool for code-related applications. MFTCoder's approach to multi-task learning also addresses the issue of varying convergence speeds by dynamically adjusting task weights based on validation loss gradients, leading to more balanced and efficient training. The framework's effectiveness has been validated through extensive experiments and comparisons with other models, demonstrating its superiority in code generation and related tasks.