**Abstract:** Recent research has explored optimization using large language models (LLMs) through iterative solution generation or direct prompting for optimizers. However, these approaches suffer from low efficiency, high sensitivity to prompt design, and a lack of domain-specific knowledge. We introduce LLaMoCo, the first instruction-tuning framework designed to adapt LLMs for solving optimization problems in a code-to-code manner. We establish a comprehensive instruction set with well-described problem prompts and effective optimization codes. Our two-phase learning strategy includes a contrastive learning-based warm-up procedure before the instruction-tuning phase to enhance convergence behavior. Experimental results show that a CodeGen (350M) model fine-tuned by LLaMoCo achieves superior optimization performance compared to GPT-4 Turbo and other competitors across synthetic and realistic problem sets. **Introduction:** LLMs are increasingly impacting society, particularly in natural language processing. This paper explores the potential of LLMs in solving optimization problems, which are typically challenging for humans. Existing methods either iteratively prompt LLMs for better solutions or directly prompt them for optimization programs. However, these approaches have limitations, including efficiency, prompt design sensitivity, and lack of domain-specific knowledge. LLaMoCo addresses these issues by fine-tuning general-purpose LLMs on a well-formatted instruction set, enabling them to generate expert-level optimizers for specific problems. **Related Works:** - **Fine-tuning LLMs:** Instruction Tuning (IT) and Alignment Tuning (AT) are prominent strategies for fine-tuning LLMs. - **LLMs for Code Generation:** Specialized LLMs like AlphaCode and StarCoder, as well as general LLMs fine-tuned for code generation, have shown promise. - **LLMs as Optimizers:** Several studies explore LLMs as optimizers, either through iterative solution improvement or direct optimization program generation. **LLaMoCo:** - **Instruction Set Construction:** We generate synthetic problem instances and select effective optimizers from literature, competitions, and benchmarks. - **Two-Phase Instruction Tuning:** A contrastive learning-based warm-up phase enhances latent space representations, followed by conventional sequence-to-sequence loss for instruction tuning. - **Results and Discussions:** LLaMoCo outperforms existing methods, demonstrating superior optimization performance and zero-shot generalization on realistic problems. **Conclusion:** LLaMoCo is the first instruction-tuning framework for adapting LLMs to solve optimization problems. It constructs a comprehensive instruction set and employs a two-phase tuning strategy, achieving superior performance compared to existing approaches. Future work includes enhancing the instruction set and further fine-tuning the LLMs.**Abstract:** Recent research has explored optimization using large language models (LLMs) through iterative solution generation or direct prompting for optimizers. However, these approaches suffer from low efficiency, high sensitivity to prompt design, and a lack of domain-specific knowledge. We introduce LLaMoCo, the first instruction-tuning framework designed to adapt LLMs for solving optimization problems in a code-to-code manner. We establish a comprehensive instruction set with well-described problem prompts and effective optimization codes. Our two-phase learning strategy includes a contrastive learning-based warm-up procedure before the instruction-tuning phase to enhance convergence behavior. Experimental results show that a CodeGen (350M) model fine-tuned by LLaMoCo achieves superior optimization performance compared to GPT-4 Turbo and other competitors across synthetic and realistic problem sets. **Introduction:** LLMs are increasingly impacting society, particularly in natural language processing. This paper explores the potential of LLMs in solving optimization problems, which are typically challenging for humans. Existing methods either iteratively prompt LLMs for better solutions or directly prompt them for optimization programs. However, these approaches have limitations, including efficiency, prompt design sensitivity, and lack of domain-specific knowledge. LLaMoCo addresses these issues by fine-tuning general-purpose LLMs on a well-formatted instruction set, enabling them to generate expert-level optimizers for specific problems. **Related Works:** - **Fine-tuning LLMs:** Instruction Tuning (IT) and Alignment Tuning (AT) are prominent strategies for fine-tuning LLMs. - **LLMs for Code Generation:** Specialized LLMs like AlphaCode and StarCoder, as well as general LLMs fine-tuned for code generation, have shown promise. - **LLMs as Optimizers:** Several studies explore LLMs as optimizers, either through iterative solution improvement or direct optimization program generation. **LLaMoCo:** - **Instruction Set Construction:** We generate synthetic problem instances and select effective optimizers from literature, competitions, and benchmarks. - **Two-Phase Instruction Tuning:** A contrastive learning-based warm-up phase enhances latent space representations, followed by conventional sequence-to-sequence loss for instruction tuning. - **Results and Discussions:** LLaMoCo outperforms existing methods, demonstrating superior optimization performance and zero-shot generalization on realistic problems. **Conclusion:** LLaMoCo is the first instruction-tuning framework for adapting LLMs to solve optimization problems. It constructs a comprehensive instruction set and employs a two-phase tuning strategy, achieving superior performance compared to existing approaches. Future work includes enhancing the instruction set and further fine-tuning the LLMs.