Structural Topic Models for Open-Ended Survey Responses

Collection and especially analysis of open-ended survey responses are relatively rare in the discipline and when conducted are almost exclusively done through human coding. We present an alternative, semiautomated approach, the structura ltopic model (STM) (Roberts, Stewart, and Airoldi 2013; Roberts et al. 2013), that draws on recent developments in machine learning based analysis of textual data. A crucial contribution of the method is that it incorporates information about the document, such as the author'™s gender, political affiliation, and treatment assignment (if an experimental study). This article focuses on how the STM is helpful for survey researchers and experimentalists. The STM makes analyzing open-ended responses easier, more revealing, and capable of being used to estimate treatment effects. We illustrate these innovations with analysis of text from surveys and experiments.

在该学科领域内，开放式问卷回复的收集工作，尤其是针对此类回复的分析，相对少见；即便开展相关研究，也几乎完全依赖人工编码。我们提出一种替代方案：半自动化分析方法——结构主题模型（structural topic model，STM）（Roberts、Stewart与Airoldi，2013；Roberts等人，2013），该方法依托基于机器学习的文本数据分析领域的最新进展。该方法的一项核心贡献在于，它能够融入文档附属信息，例如作者性别、政治倾向，以及（实验研究中的）干预分配情况。本文重点阐述结构主题模型如何为问卷研究者与实验研究者提供助力。借助该模型，开放式问卷回复的分析工作将更为简便、更具洞察价值，同时还可用于估算干预效应。我们将通过对问卷与实验文本的分析，展示这些方法创新。