DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learning

DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learning

2024 | Huiping Zhuang, Run He, Kai Tong, Ziqian Zeng, Cen Chen, Zhiping Lin
The paper introduces a novel approach called Dual-Stream Analytic Learning (DS-AL) to address the challenge of exemplar-free class-incremental learning (CIL). DS-AL consists of two main components: a main stream and a compensation stream. The main stream reformulates the CIL problem into a Concatenated Recursive Least Squares (C-RLS) task, allowing an equivalence between CIL and joint learning. The compensation stream, governed by a Dual-Activation Compensation (DAC) module, improves the under-fitting limitation of the linear mapping used in the main stream by projecting the embedding to the null space of the main stream's linear mapping. Empirical results show that DS-AL, despite being an exemplar-free technique, performs as well as or better than replay-based methods on various datasets, including CIFAR-100, ImageNet-100, and ImageNet-Full. Additionally, DS-AL demonstrates phase-invariant performance, achieving results on a 500-phase CIL ImageNet task that are comparable to those on a 5-phase task. The paper also includes a detailed analysis of the compensation stream's impact on fitting and generalization, as well as hyperparameter analysis and an ablation study.The paper introduces a novel approach called Dual-Stream Analytic Learning (DS-AL) to address the challenge of exemplar-free class-incremental learning (CIL). DS-AL consists of two main components: a main stream and a compensation stream. The main stream reformulates the CIL problem into a Concatenated Recursive Least Squares (C-RLS) task, allowing an equivalence between CIL and joint learning. The compensation stream, governed by a Dual-Activation Compensation (DAC) module, improves the under-fitting limitation of the linear mapping used in the main stream by projecting the embedding to the null space of the main stream's linear mapping. Empirical results show that DS-AL, despite being an exemplar-free technique, performs as well as or better than replay-based methods on various datasets, including CIFAR-100, ImageNet-100, and ImageNet-Full. Additionally, DS-AL demonstrates phase-invariant performance, achieving results on a 500-phase CIL ImageNet task that are comparable to those on a 5-phase task. The paper also includes a detailed analysis of the compensation stream's impact on fitting and generalization, as well as hyperparameter analysis and an ablation study.
Reach us at info@study.space
Understanding DS-AL%3A A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learning