Regularized Evolution for Image Classifier Architecture Search

Regularized Evolution for Image Classifier Architecture Search

16 Feb 2019 | Esteban Real†‡ and Alok Aggarwal† and Yanping Huang† and Quoc V. Le
This paper introduces a novel evolutionary algorithm for image classifier architecture search, called aging evolution, which outperforms existing methods. The algorithm modifies the tournament selection process by introducing an age property to favor younger genotypes, leading to faster convergence and better performance. The method is applied to the NASNet search space, which represents convolutional neural networks as small directed graphs. The algorithm evolves architectures by applying mutations to the graph structure, allowing for efficient exploration of the search space. The evolved model, AmoebaNet-A, achieves state-of-the-art accuracy on ImageNet, surpassing hand-designed models. It is competitive with other automated methods at similar sizes and sets a new state-of-the-art accuracy at larger sizes. The algorithm is compared against reinforcement learning (RL) and random search (RS), showing that evolution can achieve similar results with a simpler method, especially in resource-constrained settings. Evolution is more efficient in terms of compute resources and can explore the search space more effectively than RL. The paper also discusses the benefits of aging evolution, including its ability to prevent overfitting to training noise and improve model diversity. The algorithm is simple to implement and requires minimal tuning, making it a practical solution for architecture search. The study highlights the importance of exploring different search spaces and datasets to validate the generalizability of the results. The findings suggest that evolution is a promising approach for discovering high-quality neural network architectures.This paper introduces a novel evolutionary algorithm for image classifier architecture search, called aging evolution, which outperforms existing methods. The algorithm modifies the tournament selection process by introducing an age property to favor younger genotypes, leading to faster convergence and better performance. The method is applied to the NASNet search space, which represents convolutional neural networks as small directed graphs. The algorithm evolves architectures by applying mutations to the graph structure, allowing for efficient exploration of the search space. The evolved model, AmoebaNet-A, achieves state-of-the-art accuracy on ImageNet, surpassing hand-designed models. It is competitive with other automated methods at similar sizes and sets a new state-of-the-art accuracy at larger sizes. The algorithm is compared against reinforcement learning (RL) and random search (RS), showing that evolution can achieve similar results with a simpler method, especially in resource-constrained settings. Evolution is more efficient in terms of compute resources and can explore the search space more effectively than RL. The paper also discusses the benefits of aging evolution, including its ability to prevent overfitting to training noise and improve model diversity. The algorithm is simple to implement and requires minimal tuning, making it a practical solution for architecture search. The study highlights the importance of exploring different search spaces and datasets to validate the generalizability of the results. The findings suggest that evolution is a promising approach for discovering high-quality neural network architectures.
Reach us at info@study.space
[slides and audio] Regularized Evolution for Image Classifier Architecture Search