This paper presents an end-to-end incremental learning approach for deep neural networks, which addresses the problem of catastrophic forgetting when adding new classes incrementally. The proposed method uses a cross-distilled loss function, combining cross-entropy loss for learning new classes and distillation loss for retaining knowledge from old classes. The model is trained end-to-end, jointly learning the data representation and the classifier, without decoupling the tasks. The approach is evaluated on the CIFAR-100 and ImageNet datasets, achieving state-of-the-art performance. The main contributions of this work include: (1) an end-to-end incremental learning framework that maintains the model's parameters and memory requirements, (2) a representative memory component that stores a small set of samples from old classes to aid in retaining knowledge, and (3) a cross-distilled loss function that combines cross-entropy and distillation losses to improve performance on both old and new classes. The approach is evaluated on the CIFAR-100 and ImageNet datasets, with results showing that the proposed method outperforms existing incremental learning methods such as iCaRL and LwF.MC. The method is also shown to be effective in handling large-scale datasets with many classes, achieving good accuracy even with large incremental steps. The results demonstrate that the proposed approach is robust and effective in incremental learning scenarios.This paper presents an end-to-end incremental learning approach for deep neural networks, which addresses the problem of catastrophic forgetting when adding new classes incrementally. The proposed method uses a cross-distilled loss function, combining cross-entropy loss for learning new classes and distillation loss for retaining knowledge from old classes. The model is trained end-to-end, jointly learning the data representation and the classifier, without decoupling the tasks. The approach is evaluated on the CIFAR-100 and ImageNet datasets, achieving state-of-the-art performance. The main contributions of this work include: (1) an end-to-end incremental learning framework that maintains the model's parameters and memory requirements, (2) a representative memory component that stores a small set of samples from old classes to aid in retaining knowledge, and (3) a cross-distilled loss function that combines cross-entropy and distillation losses to improve performance on both old and new classes. The approach is evaluated on the CIFAR-100 and ImageNet datasets, with results showing that the proposed method outperforms existing incremental learning methods such as iCaRL and LwF.MC. The method is also shown to be effective in handling large-scale datasets with many classes, achieving good accuracy even with large incremental steps. The results demonstrate that the proposed approach is robust and effective in incremental learning scenarios.