Consistent Prompting for Rehearsal-Free Continual Learning

Consistent Prompting for Rehearsal-Free Continual Learning

14 Mar 2024 | Zhanxin Gao, Jun Cen, Xiaobin Chang
This paper addresses the issue of training-testing inconsistency in prompt-based continual learning methods, which are designed to adapt models to new tasks without forgetting old knowledge. Existing methods often suffer from classifier inconsistency and prompt inconsistency, leading to suboptimal performance. To tackle these issues, the authors propose a novel method called Consistent Prompting (CPrompt). CPrompt consists of two main components: Classifier Consistency Learning (CCL) and Prompt Consistency Learning (PCL). CCL ensures that all classifiers are exposed to prompt training, while PCL enhances prediction robustness and prompt selection accuracy. The multi-key mechanism is also introduced to improve prompt selection accuracy. Extensive experiments on multiple benchmarks show that CPrompt outperforms existing prompt-based methods, achieving state-of-the-art performance. The paper provides detailed analyses to demonstrate the effectiveness of each component and highlights the importance of training-testing consistency in prompt-based methods.This paper addresses the issue of training-testing inconsistency in prompt-based continual learning methods, which are designed to adapt models to new tasks without forgetting old knowledge. Existing methods often suffer from classifier inconsistency and prompt inconsistency, leading to suboptimal performance. To tackle these issues, the authors propose a novel method called Consistent Prompting (CPrompt). CPrompt consists of two main components: Classifier Consistency Learning (CCL) and Prompt Consistency Learning (PCL). CCL ensures that all classifiers are exposed to prompt training, while PCL enhances prediction robustness and prompt selection accuracy. The multi-key mechanism is also introduced to improve prompt selection accuracy. Extensive experiments on multiple benchmarks show that CPrompt outperforms existing prompt-based methods, achieving state-of-the-art performance. The paper provides detailed analyses to demonstrate the effectiveness of each component and highlights the importance of training-testing consistency in prompt-based methods.
Reach us at info@study.space