Addressing people’s current and future states in a reinforcement learning algorithm for persuading to quit smoking and to be physically active: Data and analysis code

This is the data and analysis code underlying the paper "Addressing people’s current and future states in a reinforcement learning algorithm for persuading to quit smoking and to be physically active" by Nele Albers, Mark A. Neerincx, and Willem-Paul Brinkman. This paper proposes a Reinforcement Learning (RL)-algorithm for persuading people in the context of a virtual coach for quitting smoking and becoming more physically active.<strong>Study</strong>The paper is based on a longitudinal study on the crowdsourcing platform Prolific run between 20 May 2021 and 30 June 2021. The Human Research Ethics Committee of Delft University of Technology granted ethical approval for the research (Letter of Approval number: 1523).  In this study, daily smokers who were contemplating or preparing to quit smoking interacted with the text-based virtual coach Sam in up to five conversational sessions. In each session, participants were assigned a new preparatory activity for quitting smoking, such as thinking of and writing down reasons for quitting smoking. Since becoming more physically active may make it easier to quit smoking, half of the activities addressed becoming more physically active. The virtual coach chose from five persuasion types to persuade people to do their activity. In the first two sessions, the persuasion type was chosen uniformly at random; in the last three sessions, the persuasion type was determined by a persuasion algorithm. In the next session, participants were asked to indicate the effort they spent on their activity, which served as basis for the reward signal for the persuasion algorithm.The study was pre-registered in the Open Science Framework (OSF): https://osf.io/k2uac. This pre-registration describes the study design, measures, etc. Note that the data we provide here is only a part of the data collected in the study, namely, the data related to studying the effectiveness of the persuasion algorithm. Pointers to further resources:Data on the acceptance of the virtual coach can be found here: https://doi.org/10.4121/19934783.Data on users' needs for a digital smoking cessation application can be found here: https://doi.org/10.4121/20284131.Data on users' action plans for doing the activities (n = 469) and free-text responses to reflective questions about the activities (n = 2026) is available here: https://doi.org/10.4121/21905271.The implementation of the virtual coach Sam is available here: https://doi.org/10.5281/zenodo.6319356. The formulations for the 24 preparatory activities used in the study can be found in the supplementary material of the paper (S8 Appendix).<strong>Data</strong>We collected four main types of data:<strong>Perceived motivational impact and effort</strong>. The perceived motivational impact of the conversational sessions and the effort spent on the activities were used to evaluate the effectiveness of the persuasion algorithm. Both were measured during the conversational sessions.<strong>Involvement in the activities</strong>. We used people's involvement in their activities for an exploratory subgroup analysis comparing the algorithm effectiveness for people with low and high involvement.<strong>User characteristics </strong>(e.g., age, gender, Big-Five personality, quitter self-identity). This data was collected by means of questionnaires and from participants' Prolific profiles.<strong>RL-samples</strong> (states, actions, rewards). This data was collected from the conversational sessions. The actions were the five persuasion types (e.g., consensus, action planning, no persuasion), and the reward was based on the effort.Please consult the "Data"-folder for more information on the data we collected.<br><strong>Changes in V3</strong>Fixed an error in extract_RL_samples.py: For the computation of feature means used to make the state features binary, not all the last states of participants were considered. This has now been fixed. The change does not alter the results of the paper in any way because the old and new feature means are so similar that the resulting binary state features are exactly the same.

本数据集对应Nele Albers、Mark A. Neerincx与Willem-Paul Brinkman合著的论文《面向戒烟与体育活动劝导的强化学习算法中个体当前与未来状态建模》，包含支撑该研究的实验数据与分析代码。该论文提出了一种强化学习（Reinforcement Learning, RL）算法，用于在虚拟教练场景中劝导用户戒烟并增加体育活动量。
**研究概况**
本研究基于2021年5月20日至2021年6月30日在众包平台Prolific上开展的纵向研究，该研究已获得代尔夫特理工大学人类研究伦理委员会的伦理批准（批准函编号：1523）。
本研究中，处于戒烟思考或准备阶段的每日吸烟者与基于文本的虚拟教练Sam进行最多五轮对话互动。每轮对话后，参与者会被分配一项新的戒烟准备活动，例如思考并写下戒烟理由。由于增加体育活动量有助于戒烟，半数活动围绕提升体育活动量展开。虚拟教练可从五种劝导类型中选择方式，引导参与者完成指定活动。在前两轮对话中，劝导类型随机均匀选取；在后三轮对话中，劝导类型由劝导算法决定。在下一轮对话中，参与者需报告其完成活动所投入的精力，该指标将作为劝导算法奖励信号的计算依据。
本研究已在开放科学框架（Open Science Framework, OSF）上进行预注册，注册链接：https://osf.io/k2uac，预注册文档包含研究设计、测量方案等内容。需说明的是，本数据集仅包含研究中与劝导算法有效性评估相关的部分采集数据。
**后续资源指引**
虚拟教练接受度相关数据可访问：https://doi.org/10.4121/19934783。
用户对数字化戒烟应用的需求相关数据可访问：https://doi.org/10.4121/20284131。
参与者完成活动的行动计划数据（n=469）及活动反思问题的自由文本回复数据（n=2026）可访问：https://doi.org/10.4121/21905271。
虚拟教练Sam的实现代码可访问：https://doi.org/10.5281/zenodo.6319356。
本研究使用的24项准备活动的具体表述可参见论文附录S8的补充材料。
**数据说明**
本研究共采集四类核心数据：
**感知激励影响与活动投入精力**：会话的感知激励影响及参与者完成活动的投入精力用于评估劝导算法的有效性，两类指标均在对话会话中完成采集。
**活动参与度**：参与者的活动参与度数据用于探索性亚组分析，以比较算法在高、低参与度人群中的有效性差异。
**用户人口学与心理特征**：包括年龄、性别、大五人格特质、戒烟自我认同等，数据通过问卷调查及参与者的Prolific平台个人档案采集。
**强化学习样本数据**：包含状态、动作与奖励，数据来源于对话会话。动作即五种劝导类型（例如共识引导、行动计划引导、无劝导），奖励信号基于参与者的活动投入精力计算。
如需了解采集数据的详细说明，请查阅数据文件夹。
**V3版本更新说明**
修复了extract_RL_samples.py中的一处错误：此前在将状态特征二值化的特征均值计算过程中，未纳入所有参与者的最终状态数据，现已完成修复。本次更新未对论文结果产生任何影响，因新旧特征均值差异极小，最终生成的二值化状态特征完全一致。