2009-05-26 | Nikolaus Kriegeskorte, W Kyle Simmons, Patrick S F Bellgowan & Chris I Baker
The article discusses the dangers of circular analysis in systems neuroscience, particularly the practice of "double dipping," where the same dataset is used for both selection and selective analysis. This can lead to distorted descriptive statistics and invalid statistical inference if the results statistics are not inherently independent of the selection criteria under the null hypothesis. The authors argue that this practice is common in neuroimaging and electrophysiology and can bias results, making them appear more consistent with the selection criteria. They provide examples from neuroimaging and electrophysiology to illustrate how non-independent selection can distort results and produce spurious significance. To avoid circularity, the authors suggest using independent data for selective analysis or selecting criteria that are inherently independent of the selective analysis. They propose a policy for noncircular analysis, emphasizing the importance of ensuring independence between selection and selective analysis to maintain the validity of neuroscientific results.The article discusses the dangers of circular analysis in systems neuroscience, particularly the practice of "double dipping," where the same dataset is used for both selection and selective analysis. This can lead to distorted descriptive statistics and invalid statistical inference if the results statistics are not inherently independent of the selection criteria under the null hypothesis. The authors argue that this practice is common in neuroimaging and electrophysiology and can bias results, making them appear more consistent with the selection criteria. They provide examples from neuroimaging and electrophysiology to illustrate how non-independent selection can distort results and produce spurious significance. To avoid circularity, the authors suggest using independent data for selective analysis or selecting criteria that are inherently independent of the selective analysis. They propose a policy for noncircular analysis, emphasizing the importance of ensuring independence between selection and selective analysis to maintain the validity of neuroscientific results.