Zoltan Dienes argues that non-significant results in statistical testing do not automatically support a null hypothesis but require careful interpretation using methods like power, confidence intervals, or Bayes factors. He emphasizes that Bayes factors provide a more coherent way to assess whether non-significant results support the null hypothesis or merely indicate data insensitivity. Unlike power, which requires specifying a minimal interesting effect size, Bayes factors use the data itself to determine sensitivity and focus on aspects of a theory's predictions that are easiest to specify. This approach allows for a more direct link between data and theory, enabling researchers to accept or reject the null hypothesis on equal footing.
Bayes factors compare the likelihood of data under two hypotheses (e.g., an alternative hypothesis vs. the null hypothesis) and provide a measure of evidence for one hypothesis over the other. They can indicate strong evidence for the alternative, strong evidence for the null, or insensitivity of the data. Jeffreys proposed conventional thresholds for interpreting Bayes factors: values above 3 or below 1/3 indicate substantial evidence, while values between 1/3 and 3 indicate weak or anecdotal evidence. These thresholds align with traditional standards of evidence in scientific research.
Dienes illustrates how Bayes factors can be applied in practice using different distributions to represent theoretical predictions: uniform, normal, and half-normal. For example, a uniform distribution assumes all values within a specified range are equally plausible, while a normal distribution assumes values cluster around a mean. The half-normal distribution is used when values close to zero are most plausible. These distributions help specify the plausibility of different population values given a theory, which is essential for calculating Bayes factors.
Examples show how Bayes factors can be used to interpret non-significant results. For instance, in a study on the effect of mood on learning, a Bayes factor of 0.89 indicated data insensitivity, while a smaller SE could indicate stronger evidence for the null. Similarly, in a study on placebo effects, a Bayes factor of 0.29 indicated substantial evidence for additivity, suggesting the treatments were equivalent in effectiveness.
Dienes also discusses the use of Bayes factors in complex scenarios, such as interactions in psychological experiments, where the direction of effects can vary. He highlights that Bayes factors can provide more nuanced conclusions than traditional significance testing, allowing researchers to better distinguish between data insensitivity and evidence for the null hypothesis. Overall, Bayes factors offer a flexible and coherent approach to interpreting non-significant results, enhancing the ability to draw meaningful conclusions from data.Zoltan Dienes argues that non-significant results in statistical testing do not automatically support a null hypothesis but require careful interpretation using methods like power, confidence intervals, or Bayes factors. He emphasizes that Bayes factors provide a more coherent way to assess whether non-significant results support the null hypothesis or merely indicate data insensitivity. Unlike power, which requires specifying a minimal interesting effect size, Bayes factors use the data itself to determine sensitivity and focus on aspects of a theory's predictions that are easiest to specify. This approach allows for a more direct link between data and theory, enabling researchers to accept or reject the null hypothesis on equal footing.
Bayes factors compare the likelihood of data under two hypotheses (e.g., an alternative hypothesis vs. the null hypothesis) and provide a measure of evidence for one hypothesis over the other. They can indicate strong evidence for the alternative, strong evidence for the null, or insensitivity of the data. Jeffreys proposed conventional thresholds for interpreting Bayes factors: values above 3 or below 1/3 indicate substantial evidence, while values between 1/3 and 3 indicate weak or anecdotal evidence. These thresholds align with traditional standards of evidence in scientific research.
Dienes illustrates how Bayes factors can be applied in practice using different distributions to represent theoretical predictions: uniform, normal, and half-normal. For example, a uniform distribution assumes all values within a specified range are equally plausible, while a normal distribution assumes values cluster around a mean. The half-normal distribution is used when values close to zero are most plausible. These distributions help specify the plausibility of different population values given a theory, which is essential for calculating Bayes factors.
Examples show how Bayes factors can be used to interpret non-significant results. For instance, in a study on the effect of mood on learning, a Bayes factor of 0.89 indicated data insensitivity, while a smaller SE could indicate stronger evidence for the null. Similarly, in a study on placebo effects, a Bayes factor of 0.29 indicated substantial evidence for additivity, suggesting the treatments were equivalent in effectiveness.
Dienes also discusses the use of Bayes factors in complex scenarios, such as interactions in psychological experiments, where the direction of effects can vary. He highlights that Bayes factors can provide more nuanced conclusions than traditional significance testing, allowing researchers to better distinguish between data insensitivity and evidence for the null hypothesis. Overall, Bayes factors offer a flexible and coherent approach to interpreting non-significant results, enhancing the ability to draw meaningful conclusions from data.