Addressing people’s current and future states in a reinforcement learning algorithm for persuading to quit smoking and to be physically active: Data and analysis code

This is the data and analysis code underlying the paper "Addressing people’s current and future states in a reinforcement learning algorithm for persuading to quit smoking and to be physically active" by Nele Albers, Mark A. Neerincx, and Willem-Paul Brinkman. This paper proposes a Reinforcement Learning (RL)-algorithm for persuading people in the context of a virtual coach for quitting smoking and becoming more physically active. Study The paper is based on a longitudinal study on the crowdsourcing platform Prolific run between 20 May 2021 and 30 June 2021. The Human Research Ethics Committee of Delft University of Technology granted ethical approval for the research (Letter of Approval number: 1523). In this study, smokers who were contemplating or preparing to quit smoking interacted with the text-based virtual coach Sam in up to five conversational sessions. In each session, participants were assigned a new preparatory activity for quitting smoking, such as thinking of and writing down reasons for quitting smoking. Since becoming more physically active may make it easier to quit smoking, half of the activities addressed becoming more physically active. The virtual coach chose from five persuasion types to persuade people to do their activity. In the first two sessions, the persuasion type was chosen uniformly at random; in the last three sessions, the persuasion type was determined by a persuasion algorithm. In the next session, participants were asked to indicate the effort they spent on their activity, which served as basis for the reward signal for the persuasion algorithm. The study was pre-registered in the Open Science Framework (OSF): https://osf.io/k2uac. This pre-registration describes the study design, measures, etc. Note that the data we provide here is only a part of the data collected in the study, namely, the data related to studying the effectiveness of the persuasion algorithm. Pointers to further resources: Data on the acceptance of the virtual coach can be found here: https://doi.org/10.4121/19934783.v1. Data on users' needs for a digital smoking cessation application can be found here: https://doi.org/10.4121/20284131.v2. Data on users' action plans for doing the activities (n = 469) and free-text responses to reflective questions about the activities (n = 2026) is available here: https://doi.org/10.4121/21905271.v1. The implementation of the virtual coach Sam is available here: https://doi.org/10.5281/zenodo.6319356. The formulations for the 24 preparatory activities used in the study can be found in the supplementary material of the paper (S8 Appendix). Data We collected four main types of data: Perceived motivational impact and effort. The perceived motivational impact of the conversational sessions and the effort spent on the activities were used to evaluate the effectiveness of the persuasion algorithm. Both were measured during the conversational sessions. Involvement in the activities. We used people's involvement in their activities for an exploratory subgroup analysis comparing the algorithm effectiveness for people with low and high involvement. User characteristics (e.g., age, gender, Big-Five personality, quitter self-identity). This data was collected by means of questionnaires and from participants' Prolific profiles. RL-samples (states, actions, rewards). This data was collected from the conversational sessions. The actions were the five persuasion types (e.g., consensus, action planning, no persuasion), and the reward was based on the effort. Please consult the "Data"-folder for more information on the data we collected.本数据集配套于Nele Albers、Mark A. Neerincx与Willem-Paul Brinkman合著的论文《针对戒烟与提升身体活动量强化学习劝说算法中个体当前与未来状态的处理》（英文原标题：Addressing people’s current and future states in a reinforcement learning algorithm for persuading to quit smoking and to be physically active），包含该论文依托的实验数据与分析代码。 该论文提出了一款面向戒烟与提升身体活动量场景的虚拟教练，其核心为强化学习（Reinforcement Learning，RL）劝说算法。 本研究依托于2021年5月20日至2021年6月30日在众包平台Prolific上开展的纵向研究，该研究已获得代尔夫特理工大学人类研究伦理委员会的伦理批准（批准函编号：1523）。 本研究的招募对象为正处于戒烟思考期或准备阶段的吸烟者，他们将与文本型虚拟教练Sam进行至多5轮对话会话。每一轮会话中，参与者都会获得一项全新的戒烟准备活动，例如思考并写下戒烟理由。由于提升身体活动量有助于戒烟，半数准备活动围绕提升身体活动量展开。虚拟教练可从5种劝说类型中选择其一，引导参与者完成对应活动。前两轮会话中，劝说类型均为随机均匀选取；后三轮会话中，劝说类型则由劝说算法决定。在下一轮会话开始前，参与者需报告其为完成活动所投入的精力，该指标将作为劝说算法奖励信号的计算依据。 本研究已在开放科学框架（Open Science Framework，OSF）上预先注册，注册链接为：https://osf.io/k2uac。该预注册文件详细说明了研究设计、测量指标等内容。需注意，本数据集仅包含研究中与评估劝说算法有效性相关的部分数据，而非全部采集数据。 相关补充资源指引如下： 1. 关于虚拟教练接受度的数据集可通过以下链接获取：https://doi.org/10.4121/19934783.v1 2. 关于数字化戒烟应用用户需求的数据集可通过以下链接获取：https://doi.org/10.4121/20284131.v2 3. 包含用户活动行动计划（n=469）与活动反思问题自由文本回复（n=2026）的数据集可通过以下链接获取：https://doi.org/10.4121/21905271.v1 4. 虚拟教练Sam的实现代码可通过以下链接获取：https://doi.org/10.5281/zenodo.6319356 5. 本研究中使用的24项准备活动的具体表述可参见论文的补充材料（附录S8）。 本研究共采集四类核心数据： 1. 感知动机影响与活动投入精力：会话的感知动机影响程度与参与者为活动投入的精力，被用于评估劝说算法的有效性，两类指标均在对话会话中完成测量。 2. 活动参与度：我们将参与者的活动参与度用于探索性子组分析，以对比算法在高、低参与度人群中的有效性差异。 3. 用户特征信息：包括年龄、性别、大五人格特质、戒烟自我认同等，此类数据通过问卷以及参与者的Prolific个人档案采集获得。 4. 强化学习样本（状态、动作、奖励）：此类数据从对话会话中采集得到，其中动作对应5种劝说类型（例如共识引导、行动计划引导、无劝说），奖励则基于参与者报告的活动投入精力计算。 如需了解更多采集数据的详细信息，请查阅本数据集中的"Data"文件夹。