Hierarchical Representations for Efficient Architecture Search

Hierarchical Representations for Efficient Architecture Search

22 Feb 2018 | Hanxiao Liu*, Karen Simonyan, Oriol Vinyals, Chrisantha Fernando, Koray Kavukcuoglu
The paper explores efficient methods for neural architecture search, demonstrating that a simple evolutionary algorithm can discover new architectures with excellent performance. The approach combines a novel hierarchical genetic representation scheme, which imitates the modularized design pattern used by human experts, and an expressive search space that supports complex topologies. The algorithm efficiently discovers architectures that outperform many manually designed models for image classification, achieving a top-1 error of 3.6% on CIFAR-10 and 20.3% on ImageNet, competitive with the best existing neural architecture search approaches. The paper also presents results using random search, achieving 0.3% less top-1 accuracy on CIFAR-10 and 0.1% less on ImageNet while reducing search time from 36 hours to 1 hour. The main contributions include introducing hierarchical representations for neural network architectures, showing that competitive architectures can be obtained even with simplistic random search, and presenting a scalable variant of evolutionary search that achieves the best published results among evolutionary architecture search techniques.The paper explores efficient methods for neural architecture search, demonstrating that a simple evolutionary algorithm can discover new architectures with excellent performance. The approach combines a novel hierarchical genetic representation scheme, which imitates the modularized design pattern used by human experts, and an expressive search space that supports complex topologies. The algorithm efficiently discovers architectures that outperform many manually designed models for image classification, achieving a top-1 error of 3.6% on CIFAR-10 and 20.3% on ImageNet, competitive with the best existing neural architecture search approaches. The paper also presents results using random search, achieving 0.3% less top-1 accuracy on CIFAR-10 and 0.1% less on ImageNet while reducing search time from 36 hours to 1 hour. The main contributions include introducing hierarchical representations for neural network architectures, showing that competitive architectures can be obtained even with simplistic random search, and presenting a scalable variant of evolutionary search that achieves the best published results among evolutionary architecture search techniques.
Reach us at info@study.space