Analyzing the Amazon Mechanical Turk Marketplace

Analyzing the Amazon Mechanical Turk Marketplace

| Panagiotis G. Ipeirotis
This paper analyzes the Amazon Mechanical Turk (AMT) marketplace, a crowdsourcing platform where humans complete tasks for payment. AMT allows computers to post tasks, which are then completed by human workers, giving the illusion of automation. Requesters post "HITs" (Human Intelligence Tasks), which are completed by workers for small payments. The paper presents a detailed analysis of the marketplace, including data collected from 2009 to 2010, showing that the top 0.1% of requesters account for over 30% of the marketplace's activity. Transcription is the most common task, followed by classification and categorization. The paper also analyzes the price distribution of HITs, finding that most tasks have very low rewards, with 90% of HITs offering less than 10 cents. The completion time distribution follows a power-law pattern, indicating heavy-tailed behavior, which makes task completion times unpredictable. The paper also examines the dynamics of task posting and completion, finding that the marketplace operates with a high arrival rate of tasks, but completion rates are slightly higher. The analysis suggests that the marketplace could benefit from improved task discovery mechanisms and better task recommendation systems to increase efficiency and predictability. The paper concludes that AMT is a heavy-tailed market, with a long tail of low-activity requesters, and that further research is needed to improve the marketplace's design and effectiveness.This paper analyzes the Amazon Mechanical Turk (AMT) marketplace, a crowdsourcing platform where humans complete tasks for payment. AMT allows computers to post tasks, which are then completed by human workers, giving the illusion of automation. Requesters post "HITs" (Human Intelligence Tasks), which are completed by workers for small payments. The paper presents a detailed analysis of the marketplace, including data collected from 2009 to 2010, showing that the top 0.1% of requesters account for over 30% of the marketplace's activity. Transcription is the most common task, followed by classification and categorization. The paper also analyzes the price distribution of HITs, finding that most tasks have very low rewards, with 90% of HITs offering less than 10 cents. The completion time distribution follows a power-law pattern, indicating heavy-tailed behavior, which makes task completion times unpredictable. The paper also examines the dynamics of task posting and completion, finding that the marketplace operates with a high arrival rate of tasks, but completion rates are slightly higher. The analysis suggests that the marketplace could benefit from improved task discovery mechanisms and better task recommendation systems to increase efficiency and predictability. The paper concludes that AMT is a heavy-tailed market, with a long tail of low-activity requesters, and that further research is needed to improve the marketplace's design and effectiveness.
Reach us at info@study.space