LLAMAFACTORY is a unified framework for efficient fine-tuning of over 100 large language models (LLMs). It provides a web-based interface, LLAMABOARD, allowing users to customize and fine-tune LLMs without coding. The framework integrates various efficient fine-tuning methods, including optimization techniques like LoRA, QLoRA, GaLore, and BAdam, and computation techniques such as mixed precision training, activation checkpointing, and flash attention. It supports a wide range of training approaches, including pre-training, instruction tuning, and preference optimization. LLAMAFACTORY is implemented using PyTorch and leverages open-source libraries like Transformers, PEFT, and TRL. It has been open-sourced under the Apache-2.0 license and has gained significant attention, with over 25,000 stars and 3,000 forks on GitHub. The framework has been used to build numerous open-source models on the Hugging Face Hub and has demonstrated high efficiency and effectiveness in language modeling and text generation tasks. LLAMAFACTORY also provides a flexible web interface for model inference and evaluation, supporting multilingual and distributed training. The framework has been validated through empirical studies showing its effectiveness in reducing memory usage and improving training efficiency. Future work includes expanding support for multimodal models and integrating more parallel training strategies. LLAMAFACTORY aims to democratize LLM fine-tuning by making it accessible to a broader audience.LLAMAFACTORY is a unified framework for efficient fine-tuning of over 100 large language models (LLMs). It provides a web-based interface, LLAMABOARD, allowing users to customize and fine-tune LLMs without coding. The framework integrates various efficient fine-tuning methods, including optimization techniques like LoRA, QLoRA, GaLore, and BAdam, and computation techniques such as mixed precision training, activation checkpointing, and flash attention. It supports a wide range of training approaches, including pre-training, instruction tuning, and preference optimization. LLAMAFACTORY is implemented using PyTorch and leverages open-source libraries like Transformers, PEFT, and TRL. It has been open-sourced under the Apache-2.0 license and has gained significant attention, with over 25,000 stars and 3,000 forks on GitHub. The framework has been used to build numerous open-source models on the Hugging Face Hub and has demonstrated high efficiency and effectiveness in language modeling and text generation tasks. LLAMAFACTORY also provides a flexible web interface for model inference and evaluation, supporting multilingual and distributed training. The framework has been validated through empirical studies showing its effectiveness in reducing memory usage and improving training efficiency. Future work includes expanding support for multimodal models and integrating more parallel training strategies. LLAMAFACTORY aims to democratize LLM fine-tuning by making it accessible to a broader audience.