This paper addresses the challenge of efficient lifelong learning (LLL), where the learner must quickly adapt to new tasks using a limited amount of training data. The authors introduce a new evaluation protocol that requires learners to observe each example only once and perform hyper-parameter selection on a disjoint set of tasks. They propose a new metric to measure the speed of learning and an improved version of the GEM algorithm, called Averaged GEM (A-GEM), which is both computationally and memory-efficient while maintaining or improving performance over the original GEM. A-GEM is further enhanced by incorporating compositional task descriptors, which improve few-shot learning performance. Experiments on various benchmarks demonstrate that A-GEM offers the best trade-off between accuracy and efficiency, outperforming other LLL methods in terms of average accuracy, computational cost, and memory usage.This paper addresses the challenge of efficient lifelong learning (LLL), where the learner must quickly adapt to new tasks using a limited amount of training data. The authors introduce a new evaluation protocol that requires learners to observe each example only once and perform hyper-parameter selection on a disjoint set of tasks. They propose a new metric to measure the speed of learning and an improved version of the GEM algorithm, called Averaged GEM (A-GEM), which is both computationally and memory-efficient while maintaining or improving performance over the original GEM. A-GEM is further enhanced by incorporating compositional task descriptors, which improve few-shot learning performance. Experiments on various benchmarks demonstrate that A-GEM offers the best trade-off between accuracy and efficiency, outperforming other LLL methods in terms of average accuracy, computational cost, and memory usage.