Learning to Generalize: Meta-Learning for Domain Generalization

Learning to Generalize: Meta-Learning for Domain Generalization

10 Oct 2017 | Da Li Yongxin Yang Yi-Zhe Song Timothy M. Hospedales
This paper proposes a novel meta-learning method for domain generalization (DG), called MLDG. Unlike previous DG methods that design specific models for domain shift robustness, MLDG uses a model-agnostic training procedure. It simulates train/test domain shift during training by synthesizing virtual testing domains within each mini-batch. The meta-optimization objective requires that steps to improve training domain performance also improve testing domain performance. This approach trains models with good generalization ability to novel domains. The method is evaluated on a recent cross-domain image classification benchmark and two classic reinforcement learning tasks, achieving state-of-the-art results. MLDG is model-agnostic, applicable to both supervised and reinforcement learning, and does not introduce new parameters or constrain the base learner's architecture. It is compared to existing DG methods and shown to outperform them in terms of generalization ability. The method is also analyzed, showing that it encourages alignment of gradients between training and testing domains, leading to better generalization. Several variants of MLDG are proposed, including MLDG-GC and MLDG-GN, which use cosine similarity and gradient norm respectively. Experiments on synthetic data, object recognition, and reinforcement learning tasks demonstrate the effectiveness of MLDG in achieving domain generalization. The results show that MLDG outperforms existing DG methods in terms of performance and generalization ability.This paper proposes a novel meta-learning method for domain generalization (DG), called MLDG. Unlike previous DG methods that design specific models for domain shift robustness, MLDG uses a model-agnostic training procedure. It simulates train/test domain shift during training by synthesizing virtual testing domains within each mini-batch. The meta-optimization objective requires that steps to improve training domain performance also improve testing domain performance. This approach trains models with good generalization ability to novel domains. The method is evaluated on a recent cross-domain image classification benchmark and two classic reinforcement learning tasks, achieving state-of-the-art results. MLDG is model-agnostic, applicable to both supervised and reinforcement learning, and does not introduce new parameters or constrain the base learner's architecture. It is compared to existing DG methods and shown to outperform them in terms of generalization ability. The method is also analyzed, showing that it encourages alignment of gradients between training and testing domains, leading to better generalization. Several variants of MLDG are proposed, including MLDG-GC and MLDG-GN, which use cosine similarity and gradient norm respectively. Experiments on synthetic data, object recognition, and reinforcement learning tasks demonstrate the effectiveness of MLDG in achieving domain generalization. The results show that MLDG outperforms existing DG methods in terms of performance and generalization ability.
Reach us at info@study.space
[slides and audio] Learning to Generalize%3A Meta-Learning for Domain Generalization