DARTS: DIFFERENTIABLE ARCHITECTURE SEARCH

DARTS: DIFFERENTIABLE ARCHITECTURE SEARCH

23 Apr 2019 | Hanxiao Liu*, Karen Simonyan, Yiming Yang
This paper introduces DARTS (Differentiable ARchiTecture Search), a novel algorithm for efficient architecture search in neural networks. Unlike traditional methods that use discrete and non-differentiable search spaces, DARTS relaxes the architecture representation to a continuous space, allowing for gradient-based optimization. This approach significantly reduces the computational cost while maintaining or improving performance compared to state-of-the-art non-differentiable techniques. DARTS is applicable to both convolutional and recurrent architectures and has been shown to achieve competitive results on tasks such as image classification and language modeling. The algorithm's efficiency is demonstrated through extensive experiments on datasets like CIFAR-10, ImageNet, Penn Treebank, and WikiText-2, where it outperforms existing methods with orders of magnitude less computation. The implementation of DARTS is publicly available, facilitating further research in efficient architecture search.This paper introduces DARTS (Differentiable ARchiTecture Search), a novel algorithm for efficient architecture search in neural networks. Unlike traditional methods that use discrete and non-differentiable search spaces, DARTS relaxes the architecture representation to a continuous space, allowing for gradient-based optimization. This approach significantly reduces the computational cost while maintaining or improving performance compared to state-of-the-art non-differentiable techniques. DARTS is applicable to both convolutional and recurrent architectures and has been shown to achieve competitive results on tasks such as image classification and language modeling. The algorithm's efficiency is demonstrated through extensive experiments on datasets like CIFAR-10, ImageNet, Penn Treebank, and WikiText-2, where it outperforms existing methods with orders of magnitude less computation. The implementation of DARTS is publicly available, facilitating further research in efficient architecture search.
Reach us at info@study.space