Hierarchical Representations for Efficient Architecture Search

Hierarchical Representations for Efficient Architecture Search

22 Feb 2018 | Hanxiao Liu*, Karen Simonyan, Oriol Vinyals, Chrisantha Fernando, Koray Kavukcuoglu
This paper presents an efficient neural architecture search method that uses a hierarchical genetic representation to discover high-performance neural network architectures. The approach combines a novel hierarchical genetic representation that mimics the modularized design pattern used by human experts and an expressive search space that supports complex topologies. The algorithm efficiently discovers architectures that outperform many manually designed models for image classification, achieving a top-1 error of 3.6% on CIFAR-10 and 20.3% on ImageNet, which is competitive with the best existing neural architecture search approaches. The method also shows that even simple random search can find competitive architectures when combined with a well-designed search space. The hierarchical representation allows for the construction of complex architectures by stacking smaller motifs, enabling the search algorithm to implement powerful hierarchical modules. The paper also presents results using random search, achieving 0.3% less top-1 accuracy on CIFAR-10 and 0.1% less on ImageNet while reducing search time from 36 hours to 1 hour. The experiments show that the proposed method achieves state-of-the-art results on CIFAR-10 and ImageNet, with the best published results among evolutionary architecture search techniques. The hierarchical architecture is visualized, showing that the search process has discovered several skip connections. The paper concludes that the proposed method is efficient and effective for neural architecture search, achieving high performance on image classification tasks.This paper presents an efficient neural architecture search method that uses a hierarchical genetic representation to discover high-performance neural network architectures. The approach combines a novel hierarchical genetic representation that mimics the modularized design pattern used by human experts and an expressive search space that supports complex topologies. The algorithm efficiently discovers architectures that outperform many manually designed models for image classification, achieving a top-1 error of 3.6% on CIFAR-10 and 20.3% on ImageNet, which is competitive with the best existing neural architecture search approaches. The method also shows that even simple random search can find competitive architectures when combined with a well-designed search space. The hierarchical representation allows for the construction of complex architectures by stacking smaller motifs, enabling the search algorithm to implement powerful hierarchical modules. The paper also presents results using random search, achieving 0.3% less top-1 accuracy on CIFAR-10 and 0.1% less on ImageNet while reducing search time from 36 hours to 1 hour. The experiments show that the proposed method achieves state-of-the-art results on CIFAR-10 and ImageNet, with the best published results among evolutionary architecture search techniques. The hierarchical architecture is visualized, showing that the search process has discovered several skip connections. The paper concludes that the proposed method is efficient and effective for neural architecture search, achieving high performance on image classification tasks.
Reach us at info@study.space
Understanding Hierarchical Representations for Efficient Architecture Search