This paper addresses the challenges of fine-tuning large language models (LLMs) for domain-specific machine translation (MT). Current LLM-based MT systems face issues such as sensitivity to input translation examples, high inference costs, and over-generation. To overcome these challenges, the authors propose a prompt-oriented fine-tuning method called LlamaIT. LlamaIT leverages task-specific mixed-domain datasets and fine-tunes the LLM using LoRA, a low-rank adaptation technique. This approach eliminates the need for input translation examples and post-processing, while enhancing MT capabilities for domain-specific tasks. The method also incorporates domain-specific bilingual vocabulary to improve the handling of rare words. Extensive experiments on various datasets demonstrate that LlamaIT significantly enhances the MT capabilities of LLMs, particularly in domain-specific tasks, while preserving their zero-shot MT capabilities. The paper also highlights the efficiency benefits of using LoRA, reducing training time and computational costs.This paper addresses the challenges of fine-tuning large language models (LLMs) for domain-specific machine translation (MT). Current LLM-based MT systems face issues such as sensitivity to input translation examples, high inference costs, and over-generation. To overcome these challenges, the authors propose a prompt-oriented fine-tuning method called LlamaIT. LlamaIT leverages task-specific mixed-domain datasets and fine-tunes the LLM using LoRA, a low-rank adaptation technique. This approach eliminates the need for input translation examples and post-processing, while enhancing MT capabilities for domain-specific tasks. The method also incorporates domain-specific bilingual vocabulary to improve the handling of rare words. Extensive experiments on various datasets demonstrate that LlamaIT significantly enhances the MT capabilities of LLMs, particularly in domain-specific tasks, while preserving their zero-shot MT capabilities. The paper also highlights the efficiency benefits of using LoRA, reducing training time and computational costs.