27 Jun 2018 | Takeru Miyato*,†,‡, Shin-ichi Maeda*,†, Masanori Koyama§,† and Shin Ishii†,‡
The paper introduces a new regularization method called Virtual Adversarial Training (VAT), which is designed to improve the generalization performance of supervised and semi-supervised learning models. VAT is based on the concept of virtual adversarial loss, which measures the local smoothness of the conditional label distribution around each input data point. Unlike traditional adversarial training, VAT defines the adversarial direction without using label information, making it suitable for semi-supervised learning. The method is computationally efficient, requiring only a few pairs of forward and backward propagations for neural networks. Experiments on various benchmark datasets, including MNIST, CIFAR-10, and SVHN, demonstrate that VAT achieves state-of-the-art performance in semi-supervised learning tasks, outperforming other contemporary methods. The paper also discusses the advantages of VAT, such as its applicability to semi-supervised learning, low computational cost, and parameterization-invariant regularization. Additionally, it compares VAT with random perturbation training (RPT) and explores the effects of hyperparameters and the number of power iterations on the model's performance.The paper introduces a new regularization method called Virtual Adversarial Training (VAT), which is designed to improve the generalization performance of supervised and semi-supervised learning models. VAT is based on the concept of virtual adversarial loss, which measures the local smoothness of the conditional label distribution around each input data point. Unlike traditional adversarial training, VAT defines the adversarial direction without using label information, making it suitable for semi-supervised learning. The method is computationally efficient, requiring only a few pairs of forward and backward propagations for neural networks. Experiments on various benchmark datasets, including MNIST, CIFAR-10, and SVHN, demonstrate that VAT achieves state-of-the-art performance in semi-supervised learning tasks, outperforming other contemporary methods. The paper also discusses the advantages of VAT, such as its applicability to semi-supervised learning, low computational cost, and parameterization-invariant regularization. Additionally, it compares VAT with random perturbation training (RPT) and explores the effects of hyperparameters and the number of power iterations on the model's performance.