The paper addresses the issue of catastrophic forgetting in large language models (LLMs) during continual learning, a common challenge in real-world applications where previous training data may not be available. To mitigate this problem, the authors propose a framework called Self-Synthesized Rehearsal (SSR). SSR uses the LLM itself to generate synthetic instances for rehearsal, which are then refined using the latest LLM to preserve its acquired knowledge. The synthetic instances are selected for future rehearsal stages to ensure diversity and quality. Experimental results on the SuperNI dataset demonstrate that SSR outperforms or matches conventional rehearsal-based approaches in terms of performance and data efficiency. Additionally, SSR effectively preserves the generalization capabilities of LLMs in various domains, showing its practical value and flexibility in real-world scenarios. The framework is particularly useful when real training data is limited or unavailable, making it a promising solution for continual learning in LLMs.The paper addresses the issue of catastrophic forgetting in large language models (LLMs) during continual learning, a common challenge in real-world applications where previous training data may not be available. To mitigate this problem, the authors propose a framework called Self-Synthesized Rehearsal (SSR). SSR uses the LLM itself to generate synthetic instances for rehearsal, which are then refined using the latest LLM to preserve its acquired knowledge. The synthetic instances are selected for future rehearsal stages to ensure diversity and quality. Experimental results on the SuperNI dataset demonstrate that SSR outperforms or matches conventional rehearsal-based approaches in terms of performance and data efficiency. Additionally, SSR effectively preserves the generalization capabilities of LLMs in various domains, showing its practical value and flexibility in real-world scenarios. The framework is particularly useful when real training data is limited or unavailable, making it a promising solution for continual learning in LLMs.