3 NOVEMBER 2007 | VOLUME 335 | John Ioannidis, Nikolaos Patsopoulos, and Evangelos Evangelou
The article discusses the uncertainty in heterogeneity estimates in meta-analyses, emphasizing that while meta-analyses aim to assess the similarity or dissimilarity of study results, the estimates of statistical heterogeneity can be highly uncertain. The authors highlight the limitations of popular statistical tests like Cochran’s Q and introduce the I² statistic, which is more widely used and can be compared across different meta-analyses. However, the I² statistic also has its own uncertainties, as demonstrated by its low statistical power with small numbers of studies and large confidence intervals. The article provides examples of how this uncertainty can lead to misconceptions, such as the exclusion of studies with high heterogeneity in systematic reviews. Empirical evaluations of large datasets show that a significant proportion of meta-analyses with I² estimates of 0% have wide confidence intervals indicating substantial heterogeneity. The authors conclude that the clinical implications of these uncertainties are considerable and must be carefully considered, suggesting that 95% confidence intervals should always be reported when interpreting meta-analyses.The article discusses the uncertainty in heterogeneity estimates in meta-analyses, emphasizing that while meta-analyses aim to assess the similarity or dissimilarity of study results, the estimates of statistical heterogeneity can be highly uncertain. The authors highlight the limitations of popular statistical tests like Cochran’s Q and introduce the I² statistic, which is more widely used and can be compared across different meta-analyses. However, the I² statistic also has its own uncertainties, as demonstrated by its low statistical power with small numbers of studies and large confidence intervals. The article provides examples of how this uncertainty can lead to misconceptions, such as the exclusion of studies with high heterogeneity in systematic reviews. Empirical evaluations of large datasets show that a significant proportion of meta-analyses with I² estimates of 0% have wide confidence intervals indicating substantial heterogeneity. The authors conclude that the clinical implications of these uncertainties are considerable and must be carefully considered, suggesting that 95% confidence intervals should always be reported when interpreting meta-analyses.