October 25, 2016 | Russell A. Poldrack, Chris I. Baker, Joke Durnez, Krzysztof J. Gorgolewski, Paul M. Matthews, Marcus Munafò, Thomas E. Nichols, Jean-Baptiste Poline, Edward Vul, Tal Yarkoni
The article "Scanning the Horizon: Towards transparent and reproducible neuroimaging research" addresses the challenges and issues in functional neuroimaging research, particularly fMRI. It highlights concerns such as low statistical power, flexibility in data analysis, software errors, and lack of direct replication, which contribute to the high rate of false positive findings. The authors propose several solutions to these problems, including:
1. **Statistical Power**: Advocating for pre-registration of sample sizes and power analyses to ensure adequate statistical power. They suggest using tools like neuropowertools.org and fmripower.org for power analysis.
2. **Flexibility in Data Analysis**: Recommending pre-registration of methods and analysis plans, including sample size, specific analysis tools, predicted outcomes, and ROI definitions. They emphasize the importance of clearly distinguishing exploratory analyses from planned analyses and validating results in independent datasets.
3. **Multiple Comparisons**: Suggesting dual approaches of reporting corrected whole-brain results and sharing unthresholded statistical maps. They recommend using voxel- and cluster-wise inferences with FWE or FDR correction and validating exploratory results.
4. **Software Errors**: Encouraging the use of well-established software packages and implementing good programming practices, including software testing and validation. They recommend sharing custom analysis code upon manuscript submission.
5. **Insufficient Study Reporting**: Advocating for adherence to accepted reporting standards, such as the COBIDAS guidelines, and ensuring that all major claims are supported by statistical evidence.
6. **Lack of Independent Replications**: Encouraging the recognition of replication reports as important research outcomes and requesting replication in reviews for surprising or impactful findings.
The article concludes with a vision for the future of neuroimaging research, emphasizing transparency and reproducibility. It outlines a roadmap for improving the field, including detailed planning, automated workflows, validation, and dissemination practices.The article "Scanning the Horizon: Towards transparent and reproducible neuroimaging research" addresses the challenges and issues in functional neuroimaging research, particularly fMRI. It highlights concerns such as low statistical power, flexibility in data analysis, software errors, and lack of direct replication, which contribute to the high rate of false positive findings. The authors propose several solutions to these problems, including:
1. **Statistical Power**: Advocating for pre-registration of sample sizes and power analyses to ensure adequate statistical power. They suggest using tools like neuropowertools.org and fmripower.org for power analysis.
2. **Flexibility in Data Analysis**: Recommending pre-registration of methods and analysis plans, including sample size, specific analysis tools, predicted outcomes, and ROI definitions. They emphasize the importance of clearly distinguishing exploratory analyses from planned analyses and validating results in independent datasets.
3. **Multiple Comparisons**: Suggesting dual approaches of reporting corrected whole-brain results and sharing unthresholded statistical maps. They recommend using voxel- and cluster-wise inferences with FWE or FDR correction and validating exploratory results.
4. **Software Errors**: Encouraging the use of well-established software packages and implementing good programming practices, including software testing and validation. They recommend sharing custom analysis code upon manuscript submission.
5. **Insufficient Study Reporting**: Advocating for adherence to accepted reporting standards, such as the COBIDAS guidelines, and ensuring that all major claims are supported by statistical evidence.
6. **Lack of Independent Replications**: Encouraging the recognition of replication reports as important research outcomes and requesting replication in reviews for surprising or impactful findings.
The article concludes with a vision for the future of neuroimaging research, emphasizing transparency and reproducibility. It outlines a roadmap for improving the field, including detailed planning, automated workflows, validation, and dissemination practices.