CODE Llama is a family of large language models (LLMs) designed for code generation and infilling, based on the LLAMA 2 architecture. These models offer state-of-the-art performance among open-source models, with capabilities such as infilling, support for large input contexts, and zero-shot instruction following for programming tasks. The models come in four sizes: 7B, 13B, 34B, and 70B parameters, and include foundation models (CODE Llama), Python-specific models (CODE Llama - Python), and instruction-following models (CODE Llama - INSTRUCT). They are trained on sequences of up to 16,384 tokens and can handle inputs up to 100,000 tokens. The models achieve significant improvements on various code benchmarks, outperforming other models in terms of accuracy and efficiency. CODE Llama is released under a permissive license, allowing for both research and commercial use. The paper details the training and fine-tuning processes, evaluates the models on multiple benchmarks, and discusses their safety and responsible AI considerations.CODE Llama is a family of large language models (LLMs) designed for code generation and infilling, based on the LLAMA 2 architecture. These models offer state-of-the-art performance among open-source models, with capabilities such as infilling, support for large input contexts, and zero-shot instruction following for programming tasks. The models come in four sizes: 7B, 13B, 34B, and 70B parameters, and include foundation models (CODE Llama), Python-specific models (CODE Llama - Python), and instruction-following models (CODE Llama - INSTRUCT). They are trained on sequences of up to 16,384 tokens and can handle inputs up to 100,000 tokens. The models achieve significant improvements on various code benchmarks, outperforming other models in terms of accuracy and efficiency. CODE Llama is released under a permissive license, allowing for both research and commercial use. The paper details the training and fine-tuning processes, evaluates the models on multiple benchmarks, and discusses their safety and responsible AI considerations.