Materials for: Co-Writing with Opinionated Language Models Affect Users' Views

If large language models like GPT-3 produce some views more often than others, they may influence people's opinions on an unknown scale. This study investigates whether large language models that preferably generate a particular opinion affect what users write and believe. In an online experiment, we asked participants (N=1,506) to reply to a post discussing whether social media is good for society. Treatment group participants saw suggestions from a writing assistant powered by a version of GPT-3, configured to support a specific side of the debate. Following the writing task, participants completed a social media attitude survey and an independent set of judges (N=500) evaluated the opinions expressed in participants' writing. The results show that interacting with an opinionated language model affected not only the opinion participants expressed in writing, but also shifted participants' opinion in a subsequent attitude survey. Drawing on the social influence literature and nudge theory, we discuss how opinionated AI language technologies may influence people's views. We discuss the wider implications of our results and conclude that that the opinions built into large language models need to be monitored and engineered more carefully.

If you wish your data to be removed from the repository, please contact the study's corresponding author.

若大型语言模型如 GPT-3 对某些观点的产出频率高于其他观点，则可能在不为人知的尺度上影响人们的观点。本研究旨在探究偏好生成特定观点的大型语言模型是否会影响用户的写作内容和信仰。在一场在线实验中，我们邀请了 1,506 名参与者就社交媒体是否有利于社会这一话题进行回应。接受处理的参与者见证了由 GPT-3 版本驱动的写作助手提供的建议，该助手被配置为支持辩论的特定一方。在完成写作任务后，参与者填写了社交媒体态度调查问卷，同时一组独立的评审员（500 名）评估了参与者写作中表达的观点。结果显示，与具有观点的语言模型互动不仅影响了参与者所表达的看法，还在随后的态度调查中改变了参与者的观点。借鉴社会影响文献和微促理论，我们探讨了具有观点的 AI 语言技术可能对人们观点的影响。我们讨论了研究结果的广泛影响，并得出结论：大型语言模型中内置的观点需要更加细致地监控和工程化。如若您希望从数据库中移除您的数据，请联系本研究的对应作者。