2024 | Grzegorz Rypęś, Sebastian Cygert, Valeriya Khan, Tomasz Trzciński, Bartosz Zieliński & Bartłomiej Twardowski
SEED is a novel method for exemplar-free class-incremental learning (CIL) that uses an ensemble of experts, where only one expert is trained per task. This approach mitigates catastrophic forgetting and encourages diversity among experts while maintaining high stability. Each expert represents classes with Gaussian distributions, and the optimal expert is selected based on the similarity of these distributions. SEED achieves state-of-the-art performance in various scenarios, including equal split and large first task settings, demonstrating the effectiveness of expert diversification in continual learning. The method uses a fixed number of experts, with only one being updated during training, which reduces computational overhead and improves plasticity. SEED outperforms other methods in terms of accuracy and stability, particularly in scenarios with significant data distribution shifts. The method is evaluated on benchmark datasets such as CIFAR-100, ImageNet-Subset, and DomainNet, showing superior performance across different task splits and data distributions. The results indicate that SEED is effective in both task-agnostic and task-aware settings, and its performance is robust across various configurations. The method is implemented using a combination of feature extraction and Gaussian distribution modeling, with a focus on maintaining model stability and adaptability. The experiments demonstrate that SEED achieves high accuracy while maintaining the flexibility needed for continual learning.SEED is a novel method for exemplar-free class-incremental learning (CIL) that uses an ensemble of experts, where only one expert is trained per task. This approach mitigates catastrophic forgetting and encourages diversity among experts while maintaining high stability. Each expert represents classes with Gaussian distributions, and the optimal expert is selected based on the similarity of these distributions. SEED achieves state-of-the-art performance in various scenarios, including equal split and large first task settings, demonstrating the effectiveness of expert diversification in continual learning. The method uses a fixed number of experts, with only one being updated during training, which reduces computational overhead and improves plasticity. SEED outperforms other methods in terms of accuracy and stability, particularly in scenarios with significant data distribution shifts. The method is evaluated on benchmark datasets such as CIFAR-100, ImageNet-Subset, and DomainNet, showing superior performance across different task splits and data distributions. The results indicate that SEED is effective in both task-agnostic and task-aware settings, and its performance is robust across various configurations. The method is implemented using a combination of feature extraction and Gaussian distribution modeling, with a focus on maintaining model stability and adaptability. The experiments demonstrate that SEED achieves high accuracy while maintaining the flexibility needed for continual learning.