Explaining crowdworker behaviour through computational rationality

Crowdsourcing has transformed whole industries by enabling the collection of human input at scale. Attracting high quality responses remains a challenge, however. Several factors affect which tasks a crowdworker chooses, how carefully they respond, and whether they cheat. In this work, we integrate many such factors into a simulation model of crowdworker behaviour rooted in the theory of computational rationality. The root assumption is that crowdworkers are rational and choose to behave in a way that maximises their expected subjective payoffs. The model captures two levels of decisions: (i) a worker's choice among multiple tasks and (ii) how much effort to put into a task. We formulate the worker's decision problem and use deep reinforcement learning to predict worker behaviour in realistic crowdworking scenarios. We examine predictions against empirical findings on the effects of task design and show that the model successfully predicts adaptive worker behaviour with regard to different aspects of task participation, cheating, and task-switching. To support explaining crowdworker actions and other choice behaviour, we make our model publicly available.

众包（Crowdsourcing）通过规模化采集人类输入数据，重塑了诸多行业的格局。然而，获取高质量应答始终是一项核心挑战。诸多因素会影响众包工作者的任务选择、应答细致程度以及是否存在作弊行为。在本研究中，我们将此类影响因素整合至基于计算合理性理论（theory of computational rationality）的众包工作者行为仿真模型中。该模型的核心假设为：众包工作者具备理性决策能力，会选择能够最大化其预期主观收益的行为方式。模型涵盖两类决策层级：(i) 工作者在多项任务间的选择；(ii) 为执行某项任务所投入的努力程度。我们构建了工作者的决策问题框架，并利用深度强化学习（deep reinforcement learning）在真实众包场景中预测工作者行为。我们将模型预测结果与任务设计影响的实证研究结论进行对比，结果表明，该模型可成功预测工作者在任务参与、作弊行为及任务切换等不同维度上的适应性行为。为支撑对众包工作者行为及其他选择行为的解释，我们将该模型公开发布。