Quantifying Publication Bias in Meta-Analysis

Quantifying Publication Bias in Meta-Analysis

2018 September ; 74(3): 785–794 | Lifeng Lin and Haitao Chu
Publication bias is a significant issue in systematic reviews and meta-analyses, affecting the validity and generalizability of conclusions. Current methods for addressing publication bias can be categorized into two main classes: selection models and funnel-plot-based methods. Selection models use weight functions to adjust the overall effect size estimate, while funnel-plot-based methods include visual inspection, regression and rank tests, and the nonparametric trim and fill method. However, these methods often lack intuitive interpretations and may have issues with type I error rates and power in certain scenarios. This article introduces a new measure, the skewness of the standardized deviates, to quantify publication bias. This measure describes the asymmetry of the collected studies' distribution and can serve as a test statistic. The large sample properties of the skewness measure are studied, and its performance is evaluated using simulations and three case studies from the Cochrane Database of Systematic Reviews. The simulations show that the skewness-based test has high power in many cases, though it performs poorly for small meta-analyses. A combined test using both the regression and skewness-based tests maintains high power across various settings. The case studies illustrate the application of the proposed methods, highlighting their ability to detect publication bias in different scenarios. The skewness measure has limitations, such as large variation in small meta-analyses and poor interpretation for multi-modal distributions. However, it provides an intuitive interpretation of the asymmetry of study results and can be a valuable tool for assessing publication bias in meta-analyses.Publication bias is a significant issue in systematic reviews and meta-analyses, affecting the validity and generalizability of conclusions. Current methods for addressing publication bias can be categorized into two main classes: selection models and funnel-plot-based methods. Selection models use weight functions to adjust the overall effect size estimate, while funnel-plot-based methods include visual inspection, regression and rank tests, and the nonparametric trim and fill method. However, these methods often lack intuitive interpretations and may have issues with type I error rates and power in certain scenarios. This article introduces a new measure, the skewness of the standardized deviates, to quantify publication bias. This measure describes the asymmetry of the collected studies' distribution and can serve as a test statistic. The large sample properties of the skewness measure are studied, and its performance is evaluated using simulations and three case studies from the Cochrane Database of Systematic Reviews. The simulations show that the skewness-based test has high power in many cases, though it performs poorly for small meta-analyses. A combined test using both the regression and skewness-based tests maintains high power across various settings. The case studies illustrate the application of the proposed methods, highlighting their ability to detect publication bias in different scenarios. The skewness measure has limitations, such as large variation in small meta-analyses and poor interpretation for multi-modal distributions. However, it provides an intuitive interpretation of the asymmetry of study results and can be a valuable tool for assessing publication bias in meta-analyses.
Reach us at info@study.space