23 Apr 2019 | Hanxiao Liu*, Karen Simonyan, Yiming Yang
DARTS is a differentiable architecture search method that enables efficient neural network architecture discovery. Unlike traditional methods that rely on discrete and non-differentiable search spaces, DARTS uses a continuous relaxation of the architecture representation, allowing gradient-based optimization for efficient search. The method is applicable to both convolutional and recurrent architectures and outperforms existing non-differentiable techniques in terms of speed and performance. DARTS achieves competitive results on image classification tasks like CIFAR-10 and language modeling tasks like Penn Treebank, using significantly fewer computational resources. It is also more efficient than the ENAS method. The algorithm is based on bilevel optimization, where the architecture and weights are jointly optimized. DARTS uses a continuous relaxation of the search space, allowing the architecture to be optimized with respect to validation performance via gradient descent. The method is able to transfer learned architectures to other tasks like ImageNet and WikiText-2. The implementation of DARTS is available publicly. The paper presents extensive experiments showing that DARTS achieves high performance on various tasks, with results comparable to state-of-the-art methods but with significantly less computational cost. The method is simple and efficient, avoiding the use of controllers, hypernetworks, or performance predictors. The search space is represented as a directed acyclic graph, with continuous relaxation of operations. The algorithm uses a bilevel optimization approach, where the architecture is optimized along with the weights. The method is able to derive discrete architectures from the continuous search space. The paper also discusses the efficiency improvements of DARTS, showing that it can reduce the cost of architecture discovery to a few GPU days. The results show that DARTS is able to design high-performance convolutional and recurrent architectures, outperforming existing methods in terms of efficiency and performance. The method is applicable to both image and language modeling tasks, and the results demonstrate its effectiveness in discovering high-performance architectures.DARTS is a differentiable architecture search method that enables efficient neural network architecture discovery. Unlike traditional methods that rely on discrete and non-differentiable search spaces, DARTS uses a continuous relaxation of the architecture representation, allowing gradient-based optimization for efficient search. The method is applicable to both convolutional and recurrent architectures and outperforms existing non-differentiable techniques in terms of speed and performance. DARTS achieves competitive results on image classification tasks like CIFAR-10 and language modeling tasks like Penn Treebank, using significantly fewer computational resources. It is also more efficient than the ENAS method. The algorithm is based on bilevel optimization, where the architecture and weights are jointly optimized. DARTS uses a continuous relaxation of the search space, allowing the architecture to be optimized with respect to validation performance via gradient descent. The method is able to transfer learned architectures to other tasks like ImageNet and WikiText-2. The implementation of DARTS is available publicly. The paper presents extensive experiments showing that DARTS achieves high performance on various tasks, with results comparable to state-of-the-art methods but with significantly less computational cost. The method is simple and efficient, avoiding the use of controllers, hypernetworks, or performance predictors. The search space is represented as a directed acyclic graph, with continuous relaxation of operations. The algorithm uses a bilevel optimization approach, where the architecture is optimized along with the weights. The method is able to derive discrete architectures from the continuous search space. The paper also discusses the efficiency improvements of DARTS, showing that it can reduce the cost of architecture discovery to a few GPU days. The results show that DARTS is able to design high-performance convolutional and recurrent architectures, outperforming existing methods in terms of efficiency and performance. The method is applicable to both image and language modeling tasks, and the results demonstrate its effectiveness in discovering high-performance architectures.