This survey provides a comprehensive overview of Financial Large Language Models (FinLLMs), including their history, techniques, performance, and opportunities and challenges. It begins by tracing the evolution of general-domain Pre-trained Language Models (PLMs) to current FinLLMs, highlighting the GPT-series and selected open-source models. The survey then compares five techniques used across financial PLMs and FinLLMs, such as training methods, data sources, and fine-tuning approaches. It summarizes the performance evaluations of six benchmark tasks and datasets, and provides eight advanced financial NLP tasks and datasets for developing more sophisticated FinLLMs. Finally, it discusses the opportunities and challenges facing FinLLMs, including hallucination, privacy, and efficiency. The survey aims to support AI research in finance by compiling accessible datasets and evaluation benchmarks on GitHub.This survey provides a comprehensive overview of Financial Large Language Models (FinLLMs), including their history, techniques, performance, and opportunities and challenges. It begins by tracing the evolution of general-domain Pre-trained Language Models (PLMs) to current FinLLMs, highlighting the GPT-series and selected open-source models. The survey then compares five techniques used across financial PLMs and FinLLMs, such as training methods, data sources, and fine-tuning approaches. It summarizes the performance evaluations of six benchmark tasks and datasets, and provides eight advanced financial NLP tasks and datasets for developing more sophisticated FinLLMs. Finally, it discusses the opportunities and challenges facing FinLLMs, including hallucination, privacy, and efficiency. The survey aims to support AI research in finance by compiling accessible datasets and evaluation benchmarks on GitHub.