29 Mar 2024 | Yuwen Tan*, Qinhao Zhou*, Xiang Xiang†, Ke Wang, Yuchuan Wu, Yongbin Li
Semantically-Shifted Incremental Adapter-Tuning is a Continual ViTransformer
This paper proposes a novel method for class-incremental learning (CIL) that improves continual learning capabilities by incrementally tuning shared adapters without parameter constraints. The method avoids catastrophic forgetting and retains the generalization ability of pre-trained models. It employs feature sampling from stored prototypes to retrain a unified classifier, enhancing performance. The method estimates semantic shifts of old prototypes without access to past samples and updates stored prototypes session by session. It surpasses previous pre-trained model-based CIL methods and demonstrates remarkable continual learning capabilities. Experimental results on five CIL benchmarks validate the effectiveness of the approach, achieving state-of-the-art performance.
The method uses an adapter, a lightweight structure that can be incorporated into a pre-trained transformer-based network to facilitate transfer learning. The adapter consists of a downsampled MLP layer, a non-linear activation function, and an upsampled MLP layer. The method also employs semantic shift estimation to retrain the classifier, which involves computing the semantic shift of previous prototypes and using this information to update the classifier. The method avoids retaining any image samples and eliminates the need for constructing an adapter pool.
The method is evaluated on five CIL benchmarks, including CIFAR100, CUB200, ImageNetR, ImageNetA, and VTAB. The results show that the method achieves state-of-the-art performance, outperforming other CIL methods in terms of accuracy and stability. The method is also compared to traditional CIL methods and shows superior performance, particularly in datasets with significant domain gaps from the pre-trained model. The method is found to be more efficient in terms of parameter usage and training time, making it a promising approach for continual learning.Semantically-Shifted Incremental Adapter-Tuning is a Continual ViTransformer
This paper proposes a novel method for class-incremental learning (CIL) that improves continual learning capabilities by incrementally tuning shared adapters without parameter constraints. The method avoids catastrophic forgetting and retains the generalization ability of pre-trained models. It employs feature sampling from stored prototypes to retrain a unified classifier, enhancing performance. The method estimates semantic shifts of old prototypes without access to past samples and updates stored prototypes session by session. It surpasses previous pre-trained model-based CIL methods and demonstrates remarkable continual learning capabilities. Experimental results on five CIL benchmarks validate the effectiveness of the approach, achieving state-of-the-art performance.
The method uses an adapter, a lightweight structure that can be incorporated into a pre-trained transformer-based network to facilitate transfer learning. The adapter consists of a downsampled MLP layer, a non-linear activation function, and an upsampled MLP layer. The method also employs semantic shift estimation to retrain the classifier, which involves computing the semantic shift of previous prototypes and using this information to update the classifier. The method avoids retaining any image samples and eliminates the need for constructing an adapter pool.
The method is evaluated on five CIL benchmarks, including CIFAR100, CUB200, ImageNetR, ImageNetA, and VTAB. The results show that the method achieves state-of-the-art performance, outperforming other CIL methods in terms of accuracy and stability. The method is also compared to traditional CIL methods and shows superior performance, particularly in datasets with significant domain gaps from the pre-trained model. The method is found to be more efficient in terms of parameter usage and training time, making it a promising approach for continual learning.