A Survey of Large Language Models

A Survey of Large Language Models

24 Nov 2023 | Wayne Xin Zhao, Kun Zhou*, Junyi Li*, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie and Ji-Rong Wen
A survey of large language models (LLMs) explores their evolution, capabilities, and applications. Since the Turing Test was proposed in the 1950s, humans have sought to develop AI capable of language understanding. Language modeling has evolved from statistical models to neural models, with pre-trained language models (PLMs) showing strong performance in NLP tasks. Recent advancements in scaling PLMs have led to LLMs, which exhibit emergent abilities like in-context learning not seen in smaller models. LLMs, such as GPT-3 and GPT-4, have revolutionized AI, with applications in chatbots, search engines, and multimodal tasks. The survey discusses key aspects of LLMs, including pre-training, adaptation, utilization, and capacity evaluation. It highlights the scaling laws that govern LLM performance and the challenges in aligning LLMs with human values. The survey also reviews the technical evolution of GPT-series models, from GPT-1 to GPT-4, emphasizing their improvements in capacity, task-solving, and alignment with human preferences. The survey concludes that LLMs represent a significant leap in AI, with ongoing research addressing their limitations and potential for future development.A survey of large language models (LLMs) explores their evolution, capabilities, and applications. Since the Turing Test was proposed in the 1950s, humans have sought to develop AI capable of language understanding. Language modeling has evolved from statistical models to neural models, with pre-trained language models (PLMs) showing strong performance in NLP tasks. Recent advancements in scaling PLMs have led to LLMs, which exhibit emergent abilities like in-context learning not seen in smaller models. LLMs, such as GPT-3 and GPT-4, have revolutionized AI, with applications in chatbots, search engines, and multimodal tasks. The survey discusses key aspects of LLMs, including pre-training, adaptation, utilization, and capacity evaluation. It highlights the scaling laws that govern LLM performance and the challenges in aligning LLMs with human values. The survey also reviews the technical evolution of GPT-series models, from GPT-1 to GPT-4, emphasizing their improvements in capacity, task-solving, and alignment with human preferences. The survey concludes that LLMs represent a significant leap in AI, with ongoing research addressing their limitations and potential for future development.
Reach us at info@study.space
[slides and audio] A Survey of Large Language Models