This paper discusses the practical guidelines and recommendations for fine-tuning large language models (LLMs) using proprietary domain knowledge. The authors focus on fine-tuning LLaMA, an open-source LLM, using enterprise documents and code. They propose data preparation methods for both text and code datasets, including forming paragraph chunks, question-answer pairs, and function-summary pairs. The paper also explores the use of parameter-efficient fine-tuning techniques such as LoRA and QLoRA, which reduce computational resources and memory usage. The study evaluates the effectiveness of these methods in improving model performance on domain-specific tasks. The authors also discuss the impact of quantization on model performance and memory usage, and provide guidelines for choosing the appropriate fine-tuning configuration based on available resources. The paper concludes that fine-tuning LLMs can significantly improve their performance on domain-specific tasks, but requires careful consideration of data preparation, model configuration, and resource constraints.This paper discusses the practical guidelines and recommendations for fine-tuning large language models (LLMs) using proprietary domain knowledge. The authors focus on fine-tuning LLaMA, an open-source LLM, using enterprise documents and code. They propose data preparation methods for both text and code datasets, including forming paragraph chunks, question-answer pairs, and function-summary pairs. The paper also explores the use of parameter-efficient fine-tuning techniques such as LoRA and QLoRA, which reduce computational resources and memory usage. The study evaluates the effectiveness of these methods in improving model performance on domain-specific tasks. The authors also discuss the impact of quantization on model performance and memory usage, and provide guidelines for choosing the appropriate fine-tuning configuration based on available resources. The paper concludes that fine-tuning LLMs can significantly improve their performance on domain-specific tasks, but requires careful consideration of data preparation, model configuration, and resource constraints.