EfficientNetV2: Smaller Models and Faster Training

EfficientNetV2: Smaller Models and Faster Training

23 Jun 2021 | Mingxing Tan, Quoc V. Le
EfficientNetV2 is a new family of convolutional neural networks that offer faster training and better parameter efficiency compared to previous models. Developed using a combination of training-aware neural architecture search (NAS) and scaling, EfficientNetV2 is optimized to jointly improve training speed and parameter efficiency. The models were searched from an enriched search space that includes new operations like Fused-MBConv. Experiments show that EfficientNetV2 models train significantly faster while being up to 6.8x smaller than state-of-the-art models. To further speed up training, progressive learning is used, which adaptively adjusts regularization along with image size during training. This method improves both training speed and accuracy. EfficientNetV2 outperforms previous models on ImageNet and other datasets, achieving 87.3% top-1 accuracy on ImageNet ILSVRC2012, outperforming the recent ViT by 2.0% accuracy while training 5x-11x faster using the same computing resources. EfficientNetV2 also demonstrates strong results on CIFAR, Cars, and Flowers datasets. The architecture of EfficientNetV2 is designed to address training bottlenecks in EfficientNet, including the use of Fused-MBConv for better performance, non-uniform scaling strategies, and a training-aware NAS approach. The models are trained with progressive learning, which gradually increases image size and regularization during training, leading to improved accuracy and training speed. The paper also highlights the importance of adaptive regularization in training, which dynamically adjusts regularization based on image size. This approach improves accuracy and training efficiency. EfficientNetV2 is shown to be more efficient in both training and inference, with significantly faster training speeds and better parameter efficiency compared to previous models. The results demonstrate that EfficientNetV2 is a highly effective model for image recognition tasks.EfficientNetV2 is a new family of convolutional neural networks that offer faster training and better parameter efficiency compared to previous models. Developed using a combination of training-aware neural architecture search (NAS) and scaling, EfficientNetV2 is optimized to jointly improve training speed and parameter efficiency. The models were searched from an enriched search space that includes new operations like Fused-MBConv. Experiments show that EfficientNetV2 models train significantly faster while being up to 6.8x smaller than state-of-the-art models. To further speed up training, progressive learning is used, which adaptively adjusts regularization along with image size during training. This method improves both training speed and accuracy. EfficientNetV2 outperforms previous models on ImageNet and other datasets, achieving 87.3% top-1 accuracy on ImageNet ILSVRC2012, outperforming the recent ViT by 2.0% accuracy while training 5x-11x faster using the same computing resources. EfficientNetV2 also demonstrates strong results on CIFAR, Cars, and Flowers datasets. The architecture of EfficientNetV2 is designed to address training bottlenecks in EfficientNet, including the use of Fused-MBConv for better performance, non-uniform scaling strategies, and a training-aware NAS approach. The models are trained with progressive learning, which gradually increases image size and regularization during training, leading to improved accuracy and training speed. The paper also highlights the importance of adaptive regularization in training, which dynamically adjusts regularization based on image size. This approach improves accuracy and training efficiency. EfficientNetV2 is shown to be more efficient in both training and inference, with significantly faster training speeds and better parameter efficiency compared to previous models. The results demonstrate that EfficientNetV2 is a highly effective model for image recognition tasks.
Reach us at info@study.space
[slides] EfficientNetV2%3A Smaller Models and Faster Training | StudySpace