Stable Code Technical Report

Stable Code Technical Report

1 Apr 2024 | Nikhil Pinnaparaju, Reshinth Adithyan, Duy Phung, Jonathan Tow, James Baicoianu, Ashish Datta, Maksym Zhuravinskyi, Dakota Mahan, Marco Bellagente, Carlos Riquelme, Nathan Cooper
Stable Code is a new generation code language model introduced by Stability AI, designed as a general-purpose base model for code completion, reasoning, math, and software engineering tasks. It also includes an instruction variant, Stable Code Instruct, which enables natural chat-based interaction for question-answering and instruction-based tasks. The model is trained on a diverse dataset including code repositories, technical documents, mathematical texts, and web data, with additional synthetic data generated to enhance performance. The training process involves multiple stages, including pre-training on a large-scale dataset and fine-tuning for instruction-based tasks. The model is optimized for efficiency, with quantized versions available for use on edge devices. Stable Code achieves strong performance on code completion benchmarks, including the Multi-PL benchmark, and performs comparably to larger models on the same scale. Stable Code Instruct excels in instruction-based tasks and multi-turn dialogues. The model is available on Hugging Face for download and use, and its performance is evaluated across various benchmarks, including code completion, fill-in-the-middle tasks, and SQL performance. The report also includes throughput measurements and quantization details, highlighting the model's efficiency and compatibility with different hardware. Overall, Stable Code and Stable Code Instruct demonstrate strong performance in a range of software engineering tasks, making them valuable tools for developers and researchers.Stable Code is a new generation code language model introduced by Stability AI, designed as a general-purpose base model for code completion, reasoning, math, and software engineering tasks. It also includes an instruction variant, Stable Code Instruct, which enables natural chat-based interaction for question-answering and instruction-based tasks. The model is trained on a diverse dataset including code repositories, technical documents, mathematical texts, and web data, with additional synthetic data generated to enhance performance. The training process involves multiple stages, including pre-training on a large-scale dataset and fine-tuning for instruction-based tasks. The model is optimized for efficiency, with quantized versions available for use on edge devices. Stable Code achieves strong performance on code completion benchmarks, including the Multi-PL benchmark, and performs comparably to larger models on the same scale. Stable Code Instruct excels in instruction-based tasks and multi-turn dialogues. The model is available on Hugging Face for download and use, and its performance is evaluated across various benchmarks, including code completion, fill-in-the-middle tasks, and SQL performance. The report also includes throughput measurements and quantization details, highlighting the model's efficiency and compatibility with different hardware. Overall, Stable Code and Stable Code Instruct demonstrate strong performance in a range of software engineering tasks, making them valuable tools for developers and researchers.
Reach us at info@study.space