This paper presents a comprehensive survey of the latest advancements in pre-trained model-based continual learning (PTM-CL). The authors categorize existing methods into three groups: prompt-based, representation-based, and model mixture-based. They provide a comparative analysis of these approaches, highlighting their similarities, differences, and respective advantages and disadvantages. Additionally, they conduct an empirical study comparing various state-of-the-art methods to address concerns about fairness in comparisons. The source code for reproducing these evaluations is available at https://github.com/sun-hailong/LAMDA-PILOT. Continual learning (CL) aims to enable learning systems to adapt to new data while retaining previously learned knowledge. A critical challenge in CL is catastrophic forgetting, where new knowledge acquisition leads to a decline in performance on previously learned tasks. Pre-trained models (PTMs) have shown great promise for CL due to their strong generalizability and ability to be fine-tuned with minimal changes. This paper discusses how PTMs can be leveraged for CL, including methods such as prompt tuning, representation-based approaches, and model mixture techniques. Prompt-based methods use lightweight trainable modules, such as prompts, to adjust PTMs. These methods preserve the generalizability of PTMs while allowing for task-specific adaptation. Representation-based methods utilize the generalizability of PTMs to construct classifiers, while model mixture-based methods combine multiple models during inference to alleviate catastrophic forgetting. The paper evaluates the performance of these methods on seven benchmark datasets, including CIFAR100, CUB200, ImageNet-R, ImageNet-A, ObjectNet, OmniBenchmark, and VTAB. The results show that representation-based methods, such as ADAM and RanPAC, perform more competitively than other methods. The paper also highlights the importance of fair comparison protocols and the need for new benchmarks that challenge PTMs with significant domain gaps. The authors conclude that PTM-based CL is a promising area of research, with potential applications in various domains, including large language models and multi-modal continual learning. Future directions include exploring the use of PTMs in lifelong learning scenarios and developing more efficient continual learning algorithms for resource-constrained environments.This paper presents a comprehensive survey of the latest advancements in pre-trained model-based continual learning (PTM-CL). The authors categorize existing methods into three groups: prompt-based, representation-based, and model mixture-based. They provide a comparative analysis of these approaches, highlighting their similarities, differences, and respective advantages and disadvantages. Additionally, they conduct an empirical study comparing various state-of-the-art methods to address concerns about fairness in comparisons. The source code for reproducing these evaluations is available at https://github.com/sun-hailong/LAMDA-PILOT. Continual learning (CL) aims to enable learning systems to adapt to new data while retaining previously learned knowledge. A critical challenge in CL is catastrophic forgetting, where new knowledge acquisition leads to a decline in performance on previously learned tasks. Pre-trained models (PTMs) have shown great promise for CL due to their strong generalizability and ability to be fine-tuned with minimal changes. This paper discusses how PTMs can be leveraged for CL, including methods such as prompt tuning, representation-based approaches, and model mixture techniques. Prompt-based methods use lightweight trainable modules, such as prompts, to adjust PTMs. These methods preserve the generalizability of PTMs while allowing for task-specific adaptation. Representation-based methods utilize the generalizability of PTMs to construct classifiers, while model mixture-based methods combine multiple models during inference to alleviate catastrophic forgetting. The paper evaluates the performance of these methods on seven benchmark datasets, including CIFAR100, CUB200, ImageNet-R, ImageNet-A, ObjectNet, OmniBenchmark, and VTAB. The results show that representation-based methods, such as ADAM and RanPAC, perform more competitively than other methods. The paper also highlights the importance of fair comparison protocols and the need for new benchmarks that challenge PTMs with significant domain gaps. The authors conclude that PTM-based CL is a promising area of research, with potential applications in various domains, including large language models and multi-modal continual learning. Future directions include exploring the use of PTMs in lifelong learning scenarios and developing more efficient continual learning algorithms for resource-constrained environments.