Conducting behavioral research on Amazon’s Mechanical Turk

Conducting behavioral research on Amazon’s Mechanical Turk

2012 | Winter Mason • Siddharth Suri
Amazon’s Mechanical Turk is an online labor market where requesters post jobs and workers choose which jobs to do for pay. This article demonstrates how to use this platform for behavioral research, lowering the barrier to entry for researchers. It discusses the advantages of using Mechanical Turk, such as access to a large, diverse subject pool, low cost, and faster iteration between theory and experiments. The article also compares the behavior of workers with that of experts and laboratory subjects, and illustrates the mechanics of putting a task on Mechanical Turk, including recruiting subjects, executing the task, and reviewing the work. It provides solutions to common problems, such as conducting synchronous experiments, ensuring high-quality work, maintaining data privacy, and securing code. Mechanical Turk has been used in online studies, including those that show workers' behavior is comparable to laboratory subjects. It is also used for behavioral experiments, such as studying gender, culture, and risk preferences, and social dilemmas over networks. The article highlights the validity of worker behavior, showing that results from Mechanical Turk are comparable to those from other online domains and offline settings. It also discusses the organization of the guide, the basics of Mechanical Turk, the demographics of workers, and the process of becoming a worker or requester. The article explains the anatomy of a HIT, the lifecycle of HITs, and how to improve HIT efficiency by adjusting pay and completion time. It also addresses issues such as attrition and the importance of validating responses to filter out bots and workers not attending to the survey. The article concludes by emphasizing the potential of Mechanical Turk as a powerful tool for researchers, despite its limitations.Amazon’s Mechanical Turk is an online labor market where requesters post jobs and workers choose which jobs to do for pay. This article demonstrates how to use this platform for behavioral research, lowering the barrier to entry for researchers. It discusses the advantages of using Mechanical Turk, such as access to a large, diverse subject pool, low cost, and faster iteration between theory and experiments. The article also compares the behavior of workers with that of experts and laboratory subjects, and illustrates the mechanics of putting a task on Mechanical Turk, including recruiting subjects, executing the task, and reviewing the work. It provides solutions to common problems, such as conducting synchronous experiments, ensuring high-quality work, maintaining data privacy, and securing code. Mechanical Turk has been used in online studies, including those that show workers' behavior is comparable to laboratory subjects. It is also used for behavioral experiments, such as studying gender, culture, and risk preferences, and social dilemmas over networks. The article highlights the validity of worker behavior, showing that results from Mechanical Turk are comparable to those from other online domains and offline settings. It also discusses the organization of the guide, the basics of Mechanical Turk, the demographics of workers, and the process of becoming a worker or requester. The article explains the anatomy of a HIT, the lifecycle of HITs, and how to improve HIT efficiency by adjusting pay and completion time. It also addresses issues such as attrition and the importance of validating responses to filter out bots and workers not attending to the survey. The article concludes by emphasizing the potential of Mechanical Turk as a powerful tool for researchers, despite its limitations.
Reach us at info@study.space
Understanding Conducting behavioral research on Amazon%E2%80%99s Mechanical Turk