MEASURING REPRODUCIBILITY OF HIGH-THROUGHPUT EXPERIMENTS

MEASURING REPRODUCIBILITY OF HIGH-THROUGHPUT EXPERIMENTS

2011, Vol. 5, No. 3, 1752-1779 | BY QUNHUA LI, JAMES B. BROWN, HAIYAN HUANG AND PETER J. BICKEL
The paper proposes a unified approach to measure the reproducibility of findings from replicate experiments in high-throughput studies. Unlike traditional scalar measures, the proposed method creates a curve that quantitatively assesses when findings are no longer consistent across replicates. This curve is fitted using a copula mixture model, from which a quantitative reproducibility score, called the "irreproducible discovery rate" (IDR), is derived. The IDR can be computed at each set of paired replicate ranks and allows for principled threshold setting for assessing reproducibility and combining replicates. The approach is flexible and can handle arbitrary scales for each replicate, providing useful descriptive measures in various situations. The performance of the algorithm is evaluated through simulations, and its effectiveness is demonstrated in a ChIP-seq experiment. The method is particularly useful for identifying suboptimal results and improving the reliability of signals identified by different algorithms.The paper proposes a unified approach to measure the reproducibility of findings from replicate experiments in high-throughput studies. Unlike traditional scalar measures, the proposed method creates a curve that quantitatively assesses when findings are no longer consistent across replicates. This curve is fitted using a copula mixture model, from which a quantitative reproducibility score, called the "irreproducible discovery rate" (IDR), is derived. The IDR can be computed at each set of paired replicate ranks and allows for principled threshold setting for assessing reproducibility and combining replicates. The approach is flexible and can handle arbitrary scales for each replicate, providing useful descriptive measures in various situations. The performance of the algorithm is evaluated through simulations, and its effectiveness is demonstrated in a ChIP-seq experiment. The method is particularly useful for identifying suboptimal results and improving the reliability of signals identified by different algorithms.
Reach us at info@study.space
[slides] Measuring reproducibility of high-throughput experiments | StudySpace