MAY 2013 | Katherine S. Button, John P. A. Ioannidis, Claire Mokrysz, Brian A. Nosek, Jonathan Flint, Emma S. J. Robinson and Marcus R. Munafó
Low statistical power in neuroscience studies undermines the reliability of research findings. A study with low power has a reduced chance of detecting a true effect, and it also reduces the likelihood that a statistically significant result reflects a true effect. The average statistical power of studies in the neurosciences is very low, leading to overestimates of effect size and low reproducibility of results. This problem has ethical implications, as unreliable research is inefficient and wasteful. Improving reproducibility in neuroscience is a key priority and requires attention to well-established but often ignored methodological principles.
Low statistical power leads to three main problems: the low probability of finding true effects; the low positive predictive value (PPV) when an effect is claimed; and an exaggerated estimate of the magnitude of the effect when a true effect is discovered. These problems are exacerbated by biases that often accompany low-powered studies. Empirical evidence from neuroscience shows that the median statistical power is 21%, with many studies having very low power. This means that the likelihood that a nominally significant finding actually reflects a true effect is low. The PPV decreases as statistical power decreases, making it difficult to trust findings from low-powered studies.
Low power also leads to the 'winner's curse,' where the effect size is inflated in small, underpowered studies. This inflation makes it difficult to determine an adequate sample size for replication studies, increasing the probability of type II errors. Additionally, low power is associated with several other biases, including vibration of effects, publication bias, and selective reporting of outcomes. These biases further reduce the reliability of evidence obtained in low-powered studies.
Empirical evidence from specific fields, such as neuroimaging and animal model studies, confirms that low statistical power is a widespread issue in neuroscience. The median statistical power for neuroimaging studies is 8%, and for animal model studies, it is 18% and 31% for water maze and radial maze studies, respectively. These results highlight the need for improved research practices, including a priori power calculations, transparent disclosure of methods and findings, pre-registration of study protocols, and increased availability of study materials and data.
Recommendations for future research practice include increasing disclosure, pre-registering study protocols, making study materials and data available, and incentivizing replication. These practices can improve the reliability and reproducibility of neuroscience research.Low statistical power in neuroscience studies undermines the reliability of research findings. A study with low power has a reduced chance of detecting a true effect, and it also reduces the likelihood that a statistically significant result reflects a true effect. The average statistical power of studies in the neurosciences is very low, leading to overestimates of effect size and low reproducibility of results. This problem has ethical implications, as unreliable research is inefficient and wasteful. Improving reproducibility in neuroscience is a key priority and requires attention to well-established but often ignored methodological principles.
Low statistical power leads to three main problems: the low probability of finding true effects; the low positive predictive value (PPV) when an effect is claimed; and an exaggerated estimate of the magnitude of the effect when a true effect is discovered. These problems are exacerbated by biases that often accompany low-powered studies. Empirical evidence from neuroscience shows that the median statistical power is 21%, with many studies having very low power. This means that the likelihood that a nominally significant finding actually reflects a true effect is low. The PPV decreases as statistical power decreases, making it difficult to trust findings from low-powered studies.
Low power also leads to the 'winner's curse,' where the effect size is inflated in small, underpowered studies. This inflation makes it difficult to determine an adequate sample size for replication studies, increasing the probability of type II errors. Additionally, low power is associated with several other biases, including vibration of effects, publication bias, and selective reporting of outcomes. These biases further reduce the reliability of evidence obtained in low-powered studies.
Empirical evidence from specific fields, such as neuroimaging and animal model studies, confirms that low statistical power is a widespread issue in neuroscience. The median statistical power for neuroimaging studies is 8%, and for animal model studies, it is 18% and 31% for water maze and radial maze studies, respectively. These results highlight the need for improved research practices, including a priori power calculations, transparent disclosure of methods and findings, pre-registration of study protocols, and increased availability of study materials and data.
Recommendations for future research practice include increasing disclosure, pre-registering study protocols, making study materials and data available, and incentivizing replication. These practices can improve the reliability and reproducibility of neuroscience research.