Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning

Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning

18 Mar 2024 | Da-Wei Zhou, Hai-Long Sun, Han-Jia Ye(✉), De-Chuan Zhan
The paper introduces ExpAndable Subspace Ensemble (EASE), a method for Pre-Trained Model (PTM)-based Class-Incremental Learning (CIL). EASE addresses the challenge of learning new classes without overwriting old knowledge, a common issue in CIL. The key idea is to train lightweight adapters for each new task, creating task-specific subspaces that span a high-dimensional feature space. These adapters enable joint decision-making across multiple subspaces, allowing the model to aggregate information from different tasks. To handle the incompatibility between old and new classifiers, EASE employs a semantic-guided prototype complement strategy, synthesizing old class prototypes in the new subspace without using old class instances. Extensive experiments on seven benchmark datasets demonstrate EASE's superior performance compared to state-of-the-art methods, validating its effectiveness in CIL. The method is efficient in terms of parameter cost and memory usage, making it suitable for practical applications.The paper introduces ExpAndable Subspace Ensemble (EASE), a method for Pre-Trained Model (PTM)-based Class-Incremental Learning (CIL). EASE addresses the challenge of learning new classes without overwriting old knowledge, a common issue in CIL. The key idea is to train lightweight adapters for each new task, creating task-specific subspaces that span a high-dimensional feature space. These adapters enable joint decision-making across multiple subspaces, allowing the model to aggregate information from different tasks. To handle the incompatibility between old and new classifiers, EASE employs a semantic-guided prototype complement strategy, synthesizing old class prototypes in the new subspace without using old class instances. Extensive experiments on seven benchmark datasets demonstrate EASE's superior performance compared to state-of-the-art methods, validating its effectiveness in CIL. The method is efficient in terms of parameter cost and memory usage, making it suitable for practical applications.
Reach us at info@study.space