Meta Networks

Meta Networks

8 Jun 2017 | Tsendsuren Munkhdalai, Hong Yu
MetaNet is a novel meta learning method that enables neural networks to rapidly generalize from a single example. It learns meta-level knowledge across tasks and uses fast parameterization to achieve rapid generalization. Evaluated on Omniglot and Mini-ImageNet benchmarks, MetaNet achieves near-human performance and outperforms baseline approaches by up to 6% accuracy. The model demonstrates strong generalization and continual learning capabilities. MetaNet consists of a base learner and a meta learner, with an external memory. The base learner operates in the input task space, while the meta learner operates in a task-agnostic meta space. The meta learner generates fast weights based on meta information, which is derived from the base learner's loss gradients. These fast weights are integrated into both the base and meta learners to shift their inductive biases. The model uses a layer augmentation approach to combine slow and fast weights in a neural network. It processes a higher-order meta information to fast parameterize underlying neural networks for rapid generalization. The training procedure involves generating fast weights for each task and updating slow weights through a learning algorithm. MetaNet was tested on one-shot supervised learning tasks across three datasets: Omniglot, Mini-ImageNet, and MNIST. On Omniglot, MetaNet achieved up to 6% higher accuracy than previous methods. On Mini-ImageNet, it improved previous results by up to 6% accuracy. On MNIST, it achieved 74.8% accuracy in a 10-way one-shot classification task. The model also demonstrated strong generalization capabilities, performing well on tasks with different numbers of classes. It showed flexibility in adapting to new tasks and maintaining performance across varying difficulty levels. MetaNet also supported meta-level continual learning, with the ability to retain knowledge from previous tasks even after training on new ones. The model's performance was evaluated in various scenarios, including testing on out-of-domain data and across different task splits. It showed robustness in handling tasks with different numbers of classes and demonstrated the ability to perform reverse transfer learning, where training on one task improved performance on a previously learned task. Overall, MetaNet provides a flexible and effective approach to one-shot learning and continual learning, with strong performance on benchmark tasks and the ability to adapt to new tasks with limited data.MetaNet is a novel meta learning method that enables neural networks to rapidly generalize from a single example. It learns meta-level knowledge across tasks and uses fast parameterization to achieve rapid generalization. Evaluated on Omniglot and Mini-ImageNet benchmarks, MetaNet achieves near-human performance and outperforms baseline approaches by up to 6% accuracy. The model demonstrates strong generalization and continual learning capabilities. MetaNet consists of a base learner and a meta learner, with an external memory. The base learner operates in the input task space, while the meta learner operates in a task-agnostic meta space. The meta learner generates fast weights based on meta information, which is derived from the base learner's loss gradients. These fast weights are integrated into both the base and meta learners to shift their inductive biases. The model uses a layer augmentation approach to combine slow and fast weights in a neural network. It processes a higher-order meta information to fast parameterize underlying neural networks for rapid generalization. The training procedure involves generating fast weights for each task and updating slow weights through a learning algorithm. MetaNet was tested on one-shot supervised learning tasks across three datasets: Omniglot, Mini-ImageNet, and MNIST. On Omniglot, MetaNet achieved up to 6% higher accuracy than previous methods. On Mini-ImageNet, it improved previous results by up to 6% accuracy. On MNIST, it achieved 74.8% accuracy in a 10-way one-shot classification task. The model also demonstrated strong generalization capabilities, performing well on tasks with different numbers of classes. It showed flexibility in adapting to new tasks and maintaining performance across varying difficulty levels. MetaNet also supported meta-level continual learning, with the ability to retain knowledge from previous tasks even after training on new ones. The model's performance was evaluated in various scenarios, including testing on out-of-domain data and across different task splits. It showed robustness in handling tasks with different numbers of classes and demonstrated the ability to perform reverse transfer learning, where training on one task improved performance on a previously learned task. Overall, MetaNet provides a flexible and effective approach to one-shot learning and continual learning, with strong performance on benchmark tasks and the ability to adapt to new tasks with limited data.
Reach us at info@study.space
[slides] Meta Networks | StudySpace