This paper provides a comprehensive overview of Large Language Models (LLMs), covering their training and inference techniques. The introduction highlights the significance of LLMs in natural language processing (NLP) and their evolution from statistical language models (SLMs) to neural language models (NLMs), and subsequently to pre-trained language models (PLMs). The Transformer architecture, with its self-attention mechanism, has become the foundation for LLMs, enabling efficient scaling and performance improvements. The paper discusses the key components of the Transformer model, including the encoder and decoder, self-attention, and positional embeddings. It also explores various positional encoding methods, such as absolute and relative positional encoding, and their applications in different models. Prompt learning is introduced as a method to guide pre-trained models to perform specific tasks through carefully designed prompts. The paper outlines the background, components, and learning strategies of prompt learning, emphasizing its efficiency in adapting to different tasks without retraining the entire model. The training of LLMs involves data collection, preprocessing, and the use of pre-training tasks such as language modeling. The paper discusses the architecture of LLMs, including encoder-decoder and decoder-only architectures, and their respective applications. It also covers parallel training techniques, including data parallelism, model parallelism, and the ZeRO framework, which aim to optimize training efficiency and reduce computational costs. The paper concludes with a discussion on the utilization of LLMs and their future directions, emphasizing the importance of engineering capabilities in developing and deploying LLMs.This paper provides a comprehensive overview of Large Language Models (LLMs), covering their training and inference techniques. The introduction highlights the significance of LLMs in natural language processing (NLP) and their evolution from statistical language models (SLMs) to neural language models (NLMs), and subsequently to pre-trained language models (PLMs). The Transformer architecture, with its self-attention mechanism, has become the foundation for LLMs, enabling efficient scaling and performance improvements. The paper discusses the key components of the Transformer model, including the encoder and decoder, self-attention, and positional embeddings. It also explores various positional encoding methods, such as absolute and relative positional encoding, and their applications in different models. Prompt learning is introduced as a method to guide pre-trained models to perform specific tasks through carefully designed prompts. The paper outlines the background, components, and learning strategies of prompt learning, emphasizing its efficiency in adapting to different tasks without retraining the entire model. The training of LLMs involves data collection, preprocessing, and the use of pre-training tasks such as language modeling. The paper discusses the architecture of LLMs, including encoder-decoder and decoder-only architectures, and their respective applications. It also covers parallel training techniques, including data parallelism, model parallelism, and the ZeRO framework, which aim to optimize training efficiency and reduce computational costs. The paper concludes with a discussion on the utilization of LLMs and their future directions, emphasizing the importance of engineering capabilities in developing and deploying LLMs.