The paper provides a comprehensive overview of the training and inference techniques for Large Language Models (LLMs), focusing on cost-efficient methods. It begins by discussing the evolution of language models from statistical to neural and pre-trained models, highlighting the role of the Transformer architecture in scaling up models. The paper then delves into the technical aspects of LLM training, including data preprocessing, model architecture, pre-training tasks, and parallel training. It also covers inference technologies such as model compression, parallel computation, memory scheduling, and structural optimization. The paper explores the utilization of LLMs in various applications and discusses future directions, emphasizing the importance of engineering skills and collaboration in developing LLMs. The introduction of ChatGPT has significantly increased the demand for cost-effective LLM training and deployment, making it a critical area of research. The paper aims to equip researchers with the knowledge needed to develop, deploy, and apply LLMs effectively.The paper provides a comprehensive overview of the training and inference techniques for Large Language Models (LLMs), focusing on cost-efficient methods. It begins by discussing the evolution of language models from statistical to neural and pre-trained models, highlighting the role of the Transformer architecture in scaling up models. The paper then delves into the technical aspects of LLM training, including data preprocessing, model architecture, pre-training tasks, and parallel training. It also covers inference technologies such as model compression, parallel computation, memory scheduling, and structural optimization. The paper explores the utilization of LLMs in various applications and discusses future directions, emphasizing the importance of engineering skills and collaboration in developing LLMs. The introduction of ChatGPT has significantly increased the demand for cost-effective LLM training and deployment, making it a critical area of research. The paper aims to equip researchers with the knowledge needed to develop, deploy, and apply LLMs effectively.