The paper provides a comprehensive survey of Large Language Models (LLMs), focusing on their recent advancements and applications. LLMs, which have gained significant attention since the release of ChatGPT in November 2022, are trained on massive amounts of text data to achieve strong performance in various natural language tasks. The research area of LLMs is evolving rapidly, with new models and techniques being developed at an accelerating pace. The paper reviews three prominent LLM families: GPT, LLaMA, and PaLM, discussing their characteristics, contributions, and limitations. It also covers techniques for building and augmenting LLMs, popular datasets for training and evaluation, and widely used evaluation metrics. The performance of several popular LLMs on representative benchmarks is compared, and open challenges and future research directions are discussed. The introduction highlights the history of language modeling, from early statistical models to neural language models, pre-trained language models, and LLMs. It emphasizes the role of transformer-based models in advancing LLM capabilities, including in-context learning, instruction following, and multi-step reasoning. The paper then delves into the construction of LLMs, covering dominant architectures such as encoder-only, decoder-only, and encoder-decoder models. It also discusses data preparation, tokenization, pre-training, instruction tuning, and alignment techniques. Finally, the paper concludes by summarizing the current state of LLMs and identifying areas for future research, emphasizing the potential of LLMs in developing general-purpose AI agents and artificial general intelligence (AGI).The paper provides a comprehensive survey of Large Language Models (LLMs), focusing on their recent advancements and applications. LLMs, which have gained significant attention since the release of ChatGPT in November 2022, are trained on massive amounts of text data to achieve strong performance in various natural language tasks. The research area of LLMs is evolving rapidly, with new models and techniques being developed at an accelerating pace. The paper reviews three prominent LLM families: GPT, LLaMA, and PaLM, discussing their characteristics, contributions, and limitations. It also covers techniques for building and augmenting LLMs, popular datasets for training and evaluation, and widely used evaluation metrics. The performance of several popular LLMs on representative benchmarks is compared, and open challenges and future research directions are discussed. The introduction highlights the history of language modeling, from early statistical models to neural language models, pre-trained language models, and LLMs. It emphasizes the role of transformer-based models in advancing LLM capabilities, including in-context learning, instruction following, and multi-step reasoning. The paper then delves into the construction of LLMs, covering dominant architectures such as encoder-only, decoder-only, and encoder-decoder models. It also discusses data preparation, tokenization, pre-training, instruction tuning, and alignment techniques. Finally, the paper concludes by summarizing the current state of LLMs and identifying areas for future research, emphasizing the potential of LLMs in developing general-purpose AI agents and artificial general intelligence (AGI).