This paper introduces A-GEM, an improved version of the Gradient Episodic Memory (GEM) algorithm for lifelong learning (LLL). A-GEM achieves the same or better performance than GEM while being more computationally and memory efficient than other regularization-based methods like EWC. The authors propose a new evaluation protocol where learners observe each example only once and hyper-parameter selection is done on a disjoint set of tasks not used for learning. They also introduce a new metric to measure the speed of learning and show that A-GEM has the best trade-off between accuracy and efficiency on several standard LLL benchmarks. A-GEM is also shown to benefit from task descriptors, which allow the model to learn new skills more quickly. The paper also presents a new joint embedding model that uses compositional task descriptors to improve few-shot learning performance in LLL. Experiments on four dataset streams show that A-GEM outperforms other LLL methods in terms of average accuracy, computational efficiency, and memory usage. The results demonstrate that A-GEM is a promising approach for efficient lifelong learning.This paper introduces A-GEM, an improved version of the Gradient Episodic Memory (GEM) algorithm for lifelong learning (LLL). A-GEM achieves the same or better performance than GEM while being more computationally and memory efficient than other regularization-based methods like EWC. The authors propose a new evaluation protocol where learners observe each example only once and hyper-parameter selection is done on a disjoint set of tasks not used for learning. They also introduce a new metric to measure the speed of learning and show that A-GEM has the best trade-off between accuracy and efficiency on several standard LLL benchmarks. A-GEM is also shown to benefit from task descriptors, which allow the model to learn new skills more quickly. The paper also presents a new joint embedding model that uses compositional task descriptors to improve few-shot learning performance in LLL. Experiments on four dataset streams show that A-GEM outperforms other LLL methods in terms of average accuracy, computational efficiency, and memory usage. The results demonstrate that A-GEM is a promising approach for efficient lifelong learning.