LLaMA Pro is a new post-pretraining method for large language models (LLMs) that expands Transformer blocks to enhance domain-specific knowledge while preserving general capabilities. The method involves adding new blocks to an existing LLM, which are then fine-tuned using domain-specific data. This approach allows LLaMA Pro to excel in general tasks, programming, and mathematics, as demonstrated by its performance on various benchmarks. LLaMA Pro and its instruction-following counterpart, LLaMA Pro - INSTRUCT, show superior performance compared to existing open models in the LLaMA family. The method effectively balances the model's performance across both general and domain-specific tasks, making it a versatile foundation model. The research also highlights the potential of LLaMA Pro in addressing diverse tasks as an intelligent agent. The findings contribute to the integration of natural and programming languages, laying a foundation for developing advanced language agents. The method is efficient, requiring fewer computational resources and achieving better performance in domain-specific tasks. The results show that LLaMA Pro outperforms other models in both benchmarks and practical applications, demonstrating its effectiveness and potential in broader complex applications.LLaMA Pro is a new post-pretraining method for large language models (LLMs) that expands Transformer blocks to enhance domain-specific knowledge while preserving general capabilities. The method involves adding new blocks to an existing LLM, which are then fine-tuned using domain-specific data. This approach allows LLaMA Pro to excel in general tasks, programming, and mathematics, as demonstrated by its performance on various benchmarks. LLaMA Pro and its instruction-following counterpart, LLaMA Pro - INSTRUCT, show superior performance compared to existing open models in the LLaMA family. The method effectively balances the model's performance across both general and domain-specific tasks, making it a versatile foundation model. The research also highlights the potential of LLaMA Pro in addressing diverse tasks as an intelligent agent. The findings contribute to the integration of natural and programming languages, laying a foundation for developing advanced language agents. The method is efficient, requiring fewer computational resources and achieving better performance in domain-specific tasks. The results show that LLaMA Pro outperforms other models in both benchmarks and practical applications, demonstrating its effectiveness and potential in broader complex applications.