16 Jan 2019 | Yin Cui1,2*, Menglin Jia1 Tsung-Yi Lin3 Yang Song4 Serge Belongie1,2
This paper introduces a class-balanced loss function that addresses the problem of long-tailed data distribution in deep learning. Long-tailed data distributions are characterized by a few dominant classes with many examples and many minor classes with few examples. Existing methods often use re-sampling or re-weighting based on class frequency, but these approaches can be ineffective due to diminishing returns from additional data points. The authors propose a novel theoretical framework to measure data overlap by associating each sample with a small neighboring region, leading to the concept of effective number of samples. This is calculated using the formula (1 - βⁿ)/(1 - β), where β is a hyperparameter and n is the number of samples.
The class-balanced loss is designed to re-weight the loss function inversely proportional to the effective number of samples for each class. This approach is shown to significantly improve performance on long-tailed datasets, including artificially induced long-tailed CIFAR datasets and large-scale datasets like ImageNet and iNaturalist. The method is model-agnostic and can be applied to various loss functions, including softmax cross-entropy, sigmoid cross-entropy, and focal loss. Experiments demonstrate that the class-balanced loss outperforms traditional methods, particularly in scenarios with severe class imbalance. The framework provides a general solution for handling long-tailed data distributions in visual recognition tasks.This paper introduces a class-balanced loss function that addresses the problem of long-tailed data distribution in deep learning. Long-tailed data distributions are characterized by a few dominant classes with many examples and many minor classes with few examples. Existing methods often use re-sampling or re-weighting based on class frequency, but these approaches can be ineffective due to diminishing returns from additional data points. The authors propose a novel theoretical framework to measure data overlap by associating each sample with a small neighboring region, leading to the concept of effective number of samples. This is calculated using the formula (1 - βⁿ)/(1 - β), where β is a hyperparameter and n is the number of samples.
The class-balanced loss is designed to re-weight the loss function inversely proportional to the effective number of samples for each class. This approach is shown to significantly improve performance on long-tailed datasets, including artificially induced long-tailed CIFAR datasets and large-scale datasets like ImageNet and iNaturalist. The method is model-agnostic and can be applied to various loss functions, including softmax cross-entropy, sigmoid cross-entropy, and focal loss. Experiments demonstrate that the class-balanced loss outperforms traditional methods, particularly in scenarios with severe class imbalance. The framework provides a general solution for handling long-tailed data distributions in visual recognition tasks.