This paper presents a comprehensive survey of the application of large language models (LLMs) in natural language processing (NLP). It explores how LLMs are currently used in NLP tasks, whether traditional NLP tasks have been solved with LLMs, and the future of LLMs in NLP. The study introduces a unified taxonomy of LLM applications in NLP, including parameter-frozen and parameter-tuning applications, to provide a unified perspective on the current progress of LLMs in NLP. It also summarizes new frontiers and associated challenges in LLMs for NLP, aiming to inspire further breakthroughs. The paper highlights the potential and limitations of LLMs in NLP, and provides a practical guide for building effective LLMs in NLP. The study also presents a curated collection of LLM resources for NLP, including open-source implementations, relevant corpora, and a list of research papers. These resources are available at https://github.com/LightChen233/Awesome-LLM-for-NLP. The paper also discusses various NLP tasks, including natural language understanding and generation, and explores the application of LLMs in these tasks. It highlights the effectiveness of LLMs in tasks such as sentiment analysis, information extraction, dialogue understanding, table understanding, summarization, code generation, machine translation, and mathematical reasoning. The paper also discusses the future work and new frontiers in LLMs for NLP, including multilingual LLMs, multi-modal LLMs, tool-usage in LLMs, X-of-thought in LLMs, and hallucination in LLMs. The study concludes that LLMs offer a unified generative solution paradigm for various NLP tasks, but there is still a gap between LLMs and smaller supervised learning models. Continued fine-tuning of LLMs on NLP tasks brings substantial improvements. The paper also highlights the challenges in hallucination, safety, and multilingual risks in LLMs for NLP. The study provides valuable insights and resources for building effective LLMs in NLP.This paper presents a comprehensive survey of the application of large language models (LLMs) in natural language processing (NLP). It explores how LLMs are currently used in NLP tasks, whether traditional NLP tasks have been solved with LLMs, and the future of LLMs in NLP. The study introduces a unified taxonomy of LLM applications in NLP, including parameter-frozen and parameter-tuning applications, to provide a unified perspective on the current progress of LLMs in NLP. It also summarizes new frontiers and associated challenges in LLMs for NLP, aiming to inspire further breakthroughs. The paper highlights the potential and limitations of LLMs in NLP, and provides a practical guide for building effective LLMs in NLP. The study also presents a curated collection of LLM resources for NLP, including open-source implementations, relevant corpora, and a list of research papers. These resources are available at https://github.com/LightChen233/Awesome-LLM-for-NLP. The paper also discusses various NLP tasks, including natural language understanding and generation, and explores the application of LLMs in these tasks. It highlights the effectiveness of LLMs in tasks such as sentiment analysis, information extraction, dialogue understanding, table understanding, summarization, code generation, machine translation, and mathematical reasoning. The paper also discusses the future work and new frontiers in LLMs for NLP, including multilingual LLMs, multi-modal LLMs, tool-usage in LLMs, X-of-thought in LLMs, and hallucination in LLMs. The study concludes that LLMs offer a unified generative solution paradigm for various NLP tasks, but there is still a gap between LLMs and smaller supervised learning models. Continued fine-tuning of LLMs on NLP tasks brings substantial improvements. The paper also highlights the challenges in hallucination, safety, and multilingual risks in LLMs for NLP. The study provides valuable insights and resources for building effective LLMs in NLP.