This paper introduces a novel variant of knowledge distillation (KD) called Transformed Teacher Matching (TTM), which drops the temperature scaling on the student side. By reinterpreting temperature scaling as a power transform of probability distributions, the authors show that TTM introduces an inherent Rényi entropy term in its objective function, serving as an extra regularization term. Extensive experiments demonstrate that TTM leads to better generalization compared to standard KD. To further enhance the student's ability to match the teacher's transformed probability distribution, the authors introduce a sample-adaptive weighting coefficient, resulting in a new distillation approach called Weighted TTM (WTTM). WTTM is shown to be effective, improving upon TTM and achieving state-of-the-art accuracy on various datasets. The paper also provides theoretical and empirical analyses to support these findings, including comparisons with other distillation methods and discussions on the generalized transform framework.This paper introduces a novel variant of knowledge distillation (KD) called Transformed Teacher Matching (TTM), which drops the temperature scaling on the student side. By reinterpreting temperature scaling as a power transform of probability distributions, the authors show that TTM introduces an inherent Rényi entropy term in its objective function, serving as an extra regularization term. Extensive experiments demonstrate that TTM leads to better generalization compared to standard KD. To further enhance the student's ability to match the teacher's transformed probability distribution, the authors introduce a sample-adaptive weighting coefficient, resulting in a new distillation approach called Weighted TTM (WTTM). WTTM is shown to be effective, improving upon TTM and achieving state-of-the-art accuracy on various datasets. The paper also provides theoretical and empirical analyses to support these findings, including comparisons with other distillation methods and discussions on the generalized transform framework.