The paper addresses the challenge of training deep learning models on imbalanced datasets, where the minority classes are crucial for generalization but are often underrepresented. To improve performance, the authors propose two novel methods: a label-distribution-aware margin (LDAM) loss and a deferred re-balancing optimization schedule. The LDAM loss is designed to encourage larger margins for minority classes, while the deferred re-balancing schedule defers re-weighting until after the initial training stage to avoid overfitting to minority classes. The theoretical motivation for LDAM is rooted in minimizing a margin-based generalization bound, and it is shown to be orthogonal to re-weighting and re-sampling techniques. The deferred re-balancing schedule leverages the benefits of vanilla empirical risk minimization (ERM) before applying re-weighting, leading to better initial representations and improved generalization. Experiments on various benchmark datasets, including iNaturalist 2018, demonstrate significant improvements over existing techniques, highlighting the effectiveness of the proposed methods.The paper addresses the challenge of training deep learning models on imbalanced datasets, where the minority classes are crucial for generalization but are often underrepresented. To improve performance, the authors propose two novel methods: a label-distribution-aware margin (LDAM) loss and a deferred re-balancing optimization schedule. The LDAM loss is designed to encourage larger margins for minority classes, while the deferred re-balancing schedule defers re-weighting until after the initial training stage to avoid overfitting to minority classes. The theoretical motivation for LDAM is rooted in minimizing a margin-based generalization bound, and it is shown to be orthogonal to re-weighting and re-sampling techniques. The deferred re-balancing schedule leverages the benefits of vanilla empirical risk minimization (ERM) before applying re-weighting, leading to better initial representations and improved generalization. Experiments on various benchmark datasets, including iNaturalist 2018, demonstrate significant improvements over existing techniques, highlighting the effectiveness of the proposed methods.