GhostNetV3: Exploring the Training Strategies for Compact Models This paper presents a specialized training strategy for compact neural networks, which are designed for efficient inference on edge devices. The strategy focuses on re-parameterization, knowledge distillation, learning schedule, and data augmentation. The authors find that re-parameterization and knowledge distillation are crucial for training high-performance compact models, while some data augmentation techniques like Mixup and CutMix can degrade performance. Experiments on the ImageNet-1K dataset show that the proposed strategy significantly improves the performance of compact models, including GhostNetV3, MobileNetV2, and ShuffleNetV2. GhostNetV3 1.3× achieves a top-1 accuracy of 79.1% with only 269M FLOPs and a latency of 14.46ms on mobile devices, surpassing its ordinarily trained counterpart. The strategy is also applicable to object detection scenarios. The code and checkpoints are available at https://github.com/huawei-noah/Efficient-AI-Backbones. The paper also compares GhostNetV3 with other compact models and shows that it achieves better accuracy and lower latency than many existing models. The results demonstrate the effectiveness of the proposed training strategy in improving the performance of compact models.GhostNetV3: Exploring the Training Strategies for Compact Models This paper presents a specialized training strategy for compact neural networks, which are designed for efficient inference on edge devices. The strategy focuses on re-parameterization, knowledge distillation, learning schedule, and data augmentation. The authors find that re-parameterization and knowledge distillation are crucial for training high-performance compact models, while some data augmentation techniques like Mixup and CutMix can degrade performance. Experiments on the ImageNet-1K dataset show that the proposed strategy significantly improves the performance of compact models, including GhostNetV3, MobileNetV2, and ShuffleNetV2. GhostNetV3 1.3× achieves a top-1 accuracy of 79.1% with only 269M FLOPs and a latency of 14.46ms on mobile devices, surpassing its ordinarily trained counterpart. The strategy is also applicable to object detection scenarios. The code and checkpoints are available at https://github.com/huawei-noah/Efficient-AI-Backbones. The paper also compares GhostNetV3 with other compact models and shows that it achieves better accuracy and lower latency than many existing models. The results demonstrate the effectiveness of the proposed training strategy in improving the performance of compact models.