Beyond the Turk: Alternative platforms for crowdsourcing behavioral research

Beyond the Turk: Alternative platforms for crowdsourcing behavioral research

2017 | Peer, Eyal; Brandimarte, Laura; Samat, Sonam; Acquisti, Alessandro
The paper "Beyond the Turk: Alternative platforms for crowdsourcing behavioral research" by Eyal Peer, Laura Brandimarte, Sonam Samat, and Alessandro Acquisti explores the limitations of Amazon Mechanical Turk (MTurk) as a crowdsourcing platform for behavioral research, particularly the issue of participant non-naivety. The authors introduce and evaluate two alternative platforms, CrowdFlower (CF) and Prolific Academic (ProA), which offer access to more naïve and diverse populations. Through two empirical studies, they find that participants on CF and ProA are more naïve and less dishonest compared to MTurk participants. CF provides the best response rate but has higher failure rates on attention-check questions and fails to replicate certain known effects. ProA, while having a slightly lower response rate, produces data quality comparable to MTurk and exhibits higher naivety and lower dishonest behavior among participants. The study concludes that ProA is a viable alternative to MTurk, offering high-quality data with more diverse and naïve participants, despite its slower response rate.The paper "Beyond the Turk: Alternative platforms for crowdsourcing behavioral research" by Eyal Peer, Laura Brandimarte, Sonam Samat, and Alessandro Acquisti explores the limitations of Amazon Mechanical Turk (MTurk) as a crowdsourcing platform for behavioral research, particularly the issue of participant non-naivety. The authors introduce and evaluate two alternative platforms, CrowdFlower (CF) and Prolific Academic (ProA), which offer access to more naïve and diverse populations. Through two empirical studies, they find that participants on CF and ProA are more naïve and less dishonest compared to MTurk participants. CF provides the best response rate but has higher failure rates on attention-check questions and fails to replicate certain known effects. ProA, while having a slightly lower response rate, produces data quality comparable to MTurk and exhibits higher naivety and lower dishonest behavior among participants. The study concludes that ProA is a viable alternative to MTurk, offering high-quality data with more diverse and naïve participants, despite its slower response rate.
Reach us at info@study.space
Understanding Beyond the Turk%3A Alternative Platforms for Crowdsourcing Behavioral Research