March 2013 | Volume 8 | Issue 3 | e57410 | Matthew J. C. Crump, John V. McDonnell, Todd M. Gureckis
This paper evaluates the use of Amazon Mechanical Turk (AMT) as a tool for conducting experimental behavioral research. The authors replicate a variety of classic cognitive tasks, including the Stroop, Switching, Flanker, Simon, Posner Cuing, attentional blink, subliminal priming, and category learning tasks, to assess the fidelity of AMT for cognitive behavioral experiments. While most replications were successful and validated the approach of collecting data anonymously online, some experiments revealed discrepancies between laboratory and online results. The study highlights the importance of technical considerations, such as browser compatibility and environmental distractions, and provides valuable lessons for researchers considering using AMT for their work. Overall, the results suggest that AMT can be a viable tool for conducting multi-trial designs in behavioral cognitive research, particularly for tasks that rely on millisecond timing and complex instructions.This paper evaluates the use of Amazon Mechanical Turk (AMT) as a tool for conducting experimental behavioral research. The authors replicate a variety of classic cognitive tasks, including the Stroop, Switching, Flanker, Simon, Posner Cuing, attentional blink, subliminal priming, and category learning tasks, to assess the fidelity of AMT for cognitive behavioral experiments. While most replications were successful and validated the approach of collecting data anonymously online, some experiments revealed discrepancies between laboratory and online results. The study highlights the importance of technical considerations, such as browser compatibility and environmental distractions, and provides valuable lessons for researchers considering using AMT for their work. Overall, the results suggest that AMT can be a viable tool for conducting multi-trial designs in behavioral cognitive research, particularly for tasks that rely on millisecond timing and complex instructions.