Moving beyond P values: Everyday data analysis with estimation plots

Moving beyond P values: Everyday data analysis with estimation plots

April 6, 2019 | Joses Ho, Tayfun Tumkaya, Sameer Aryal, Hyungwon Choi, Adam Claridge-Chang
The article discusses the limitations of null-hypothesis significance testing (NHST) and proposes estimation plots as a more informative alternative for data analysis. Estimation plots provide a visual representation of effect sizes and their uncertainty, promoting quantitative reasoning over dichotomous thinking. The authors argue that NHST focuses on whether an effect exists (i.e., "Does it?") rather than how large the effect is (i.e., "How much?"). Estimation plots, in contrast, display the actual effect size and its confidence interval, allowing for a more nuanced interpretation of data. The two-groups design is fundamental in many experiments, and traditional methods like the Student's t-test are often used to analyze such data. However, these methods have limitations, including the inability to show the full dataset and distributional information. Estimation plots address these issues by visualizing all observed values and the effect size side-by-side, providing greater transparency and insight. The authors introduce DABEST, a free software package that enables the creation of estimation plots. DABEST is available in three open-source libraries for Matlab, Python, and R, and also includes a web application for ease of use. The software uses bootstrapping to calculate the sampling distribution and confidence intervals, making it robust and versatile for various experimental designs. Estimation plots are shown to be more informative than traditional NHST plots, as they provide a clearer picture of the effect size and its precision. They also encourage quantitative reasoning, enabling scientists to make domain-specific judgments about the relevance of an effect. The authors conclude that estimation plots represent an elegant and robust framework for presenting data, and that their adoption could significantly improve the quality of scientific research.The article discusses the limitations of null-hypothesis significance testing (NHST) and proposes estimation plots as a more informative alternative for data analysis. Estimation plots provide a visual representation of effect sizes and their uncertainty, promoting quantitative reasoning over dichotomous thinking. The authors argue that NHST focuses on whether an effect exists (i.e., "Does it?") rather than how large the effect is (i.e., "How much?"). Estimation plots, in contrast, display the actual effect size and its confidence interval, allowing for a more nuanced interpretation of data. The two-groups design is fundamental in many experiments, and traditional methods like the Student's t-test are often used to analyze such data. However, these methods have limitations, including the inability to show the full dataset and distributional information. Estimation plots address these issues by visualizing all observed values and the effect size side-by-side, providing greater transparency and insight. The authors introduce DABEST, a free software package that enables the creation of estimation plots. DABEST is available in three open-source libraries for Matlab, Python, and R, and also includes a web application for ease of use. The software uses bootstrapping to calculate the sampling distribution and confidence intervals, making it robust and versatile for various experimental designs. Estimation plots are shown to be more informative than traditional NHST plots, as they provide a clearer picture of the effect size and its precision. They also encourage quantitative reasoning, enabling scientists to make domain-specific judgments about the relevance of an effect. The authors conclude that estimation plots represent an elegant and robust framework for presenting data, and that their adoption could significantly improve the quality of scientific research.
Reach us at info@futurestudyspace.com