Data and analysis for the publication: Content-Based Recommender Support System for Counselors in a Suicide Prevention Chat Helpline: Design and Evaluation Study

This dataset contains all data and analysis scripts to the research conducted for the JMIR paper: "Content-based Recommender Support System for Counselors in a Suicide Prevention Chat Helpline: Design and Evaluation Study" <br><b>Background</b>: The working environment of a suicide prevention helpline requires high emotional and cognitive awareness from chat counselors. A shared opinion among counselors is that as a chat conversation becomes more difficult, it takes more effort and a longer amount of time to compose a response, which, in turn, can lead to writer’s block.<br><b>Objective</b>: This study evaluates and then designs supportive technology to determine if a support system that provides inspiration can help counselors resolve writer’s block when they encounter difficult situations in chats with help-seekers. <br><b>Methods</b>: A content-based recommender system with sentence embedding was used to search a chat corpus for similar chat situations. The system showed a counselor the most similar parts of former chat conversations so that the counselor would be able to use approaches previously taken by their colleagues as inspiration. In a within-subject experiment, counselors’ chat replies when confronted with a difficult situation were analyzed to determine if experts could see a noticeable difference in chat replies that were obtained in 3 conditions: (1) with the help of the support system, (2) with written advice from a senior counselor, or (3) when receiving no help. In addition, the system’s utility and usability were measured, and the validity of the algorithm was examined.<br><b>Results</b>: A total of 24 counselors used a prototype of the support system; the results showed that, by reading chat replies, experts were able to significantly predict if counselors had received help from the support system or from a senior counselor <i>(P=</i>.004). Counselors scored the information they received from a senior counselor (M=1.46, SD 1.91) as significantly more helpful than the information received from the support system or when no help was given at all (M=–0.21, SD 2.26). Finally, compared with randomly selected former chat conversations, counselors rated the ones identified by the content-based recommendation system as significantly more similar to their current chats (β=.30<i>,</i> <i>P</i><b>Conclusions</b>:<b> </b>Support given to counselors influenced how they responded in difficult conversations. However, the higher utility scores given for the advice from senior counselors seem to indicate that specific actionable instructions are preferred. We expect that these findings will be beneficial for developing a system that can use similar chat situations to generate advice in a descriptive style, hence helping counselors through writer’s block.<b></b><br>

本数据集包含针对JMIR期刊论文《面向自杀预防聊天热线辅导员的基于内容推荐支持系统：设计与评估研究》的相关研究的全部数据与分析脚本。

**背景**：自杀预防聊天热线的工作环境对聊天辅导员的情绪感知与认知能力要求极高。辅导员群体普遍认为，当聊天对话难度提升时，撰写回复所需的精力与时长均会增加，进而引发写作阻滞（writer’s block）。

**目标**：本研究旨在开发并评估一款辅助支持技术，探究能够提供创作灵感的支持系统，是否可以帮助辅导员在与求助者的棘手聊天对话中克服写作阻滞。

**方法**：本研究采用搭载句子嵌入（sentence embedding）技术的基于内容的推荐系统（content-based recommender system），在聊天语料库中检索相似的聊天场景。该系统会向辅导员展示过往聊天对话中相似度最高的片段，以供辅导员参考同事此前采用的应对思路作为灵感。本研究采用被试内实验设计，分析辅导员在面临棘手场景时的聊天回复，以判断专家能否从三种条件下生成的聊天回复中识别出显著差异：(1) 借助支持系统辅助；(2) 参考资深辅导员的书面建议；(3) 未获得任何辅助。此外，本研究还评估了该系统的实用性与易用性，并验证了其算法的有效性。

**结果**：共有24名辅导员使用了该支持系统的原型产品。结果显示，通过审阅聊天回复，专家能够显著区分出辅导员是否借助了支持系统或资深辅导员的建议（P=0.004）。辅导员对资深辅导员提供的建议的实用性评分（M=1.46，SD=1.91）显著高于支持系统提供的信息，以及未获得任何辅助的情况（M=-0.21，SD=2.26）。最后，相较于随机抽取的过往聊天对话，辅导员对基于内容推荐系统识别出的对话的相似度评分显著更高（β=0.30，P=）。

**结论**：为辅导员提供的辅助支持会影响其在棘手对话中的回复表现。不过，辅导员对资深辅导员提供的建议给出的更高实用性评分，似乎表明他们更倾向于具体可执行的指导意见。本研究结果有望助力开发一类可基于相似聊天场景生成描述性建议的系统，从而帮助辅导员克服写作阻滞。