The article critiques traditional statistical methods, particularly the null-hypothesis test, which relies on p-values to determine statistical significance. It argues that this approach is misleading because it does not account for the real-world relevance of outcomes. Instead, the authors propose a more intuitive approach based on confidence intervals and magnitude-based inferences. Confidence intervals provide a range of likely true values for a statistic, while magnitude-based inferences assess the practical significance of these values by considering whether they are substantially positive, trivial, or substantially negative.
The authors suggest that if the confidence interval overlaps substantially positive and negative values, the outcome is unclear. Otherwise, the true value is inferred to be substantially positive, trivial, or substantially negative. They further refine this by estimating the likelihood that the true value has the observed magnitude, such as "very likely beneficial." This approach allows for more meaningful interpretations of study results by considering the practical implications of the findings rather than just statistical significance.
The article also discusses other approaches to inference, including Bayesian statistics and meta-analysis, but emphasizes the importance of magnitude-based inferences for practical and clinical significance. It highlights the need for researchers to consider the smallest important effect size when justifying the choice of thresholds for meaningful magnitudes. The authors advocate for a more nuanced interpretation of study results that takes into account the potential benefits, triviality, and harm of outcomes, rather than relying solely on p-values or confidence intervals. They also note that while magnitude-based inferences are more informative, they require additional calculations and may be more complex to implement. The article concludes by emphasizing the importance of moving beyond traditional statistical methods to more meaningful and practical interpretations of research findings.The article critiques traditional statistical methods, particularly the null-hypothesis test, which relies on p-values to determine statistical significance. It argues that this approach is misleading because it does not account for the real-world relevance of outcomes. Instead, the authors propose a more intuitive approach based on confidence intervals and magnitude-based inferences. Confidence intervals provide a range of likely true values for a statistic, while magnitude-based inferences assess the practical significance of these values by considering whether they are substantially positive, trivial, or substantially negative.
The authors suggest that if the confidence interval overlaps substantially positive and negative values, the outcome is unclear. Otherwise, the true value is inferred to be substantially positive, trivial, or substantially negative. They further refine this by estimating the likelihood that the true value has the observed magnitude, such as "very likely beneficial." This approach allows for more meaningful interpretations of study results by considering the practical implications of the findings rather than just statistical significance.
The article also discusses other approaches to inference, including Bayesian statistics and meta-analysis, but emphasizes the importance of magnitude-based inferences for practical and clinical significance. It highlights the need for researchers to consider the smallest important effect size when justifying the choice of thresholds for meaningful magnitudes. The authors advocate for a more nuanced interpretation of study results that takes into account the potential benefits, triviality, and harm of outcomes, rather than relying solely on p-values or confidence intervals. They also note that while magnitude-based inferences are more informative, they require additional calculations and may be more complex to implement. The article concludes by emphasizing the importance of moving beyond traditional statistical methods to more meaningful and practical interpretations of research findings.