Using the Web to Collect Data on Sensitive Behaviours: A Study Looking at Mode Effects on the British National Survey of Sexual Attitudes and Lifestyles

BackgroundInterviewer-administered surveys are an important method of collecting population-level epidemiological data, but suffer from declining response rates and increasing costs. Web surveys offer more rapid data collection and lower costs. There are concerns, however, about data quality from web surveys. Previous research has largely focused on selection biases, and few have explored measurement differences. This paper aims to assess the extent to which mode affects the responses given by the same respondents at two points in time, providing information on potential measurement error if web surveys are used in the future.Methods527 participants from the third British National Survey of Sexual Attitudes and Lifestyles (Natsal-3), which uses computer assisted personal interview (CAPI) and self-interview (CASI) modes, subsequently responded to identically-worded questions in a web survey. McNemar tests assessed whether within-person differences in responses were at random or indicated a mode effect, i.e. higher reporting of more sensitive responses in one mode. An analysis of pooled responses by generalized estimating equations addressed the impact of gender and question type on change.ResultsOnly 10% of responses changed between surveys. However mode effects were found for about a third of variables, with higher reporting of sensitive responses more commonly found on the web compared with Natsal-3.ConclusionsThe web appears a promising mode for surveys of sensitive behaviours, most likely as part of a mixed-mode design. Our findings suggest that mode effects may vary by question type and content, and by the particular mix of modes used. Mixed-mode surveys need careful development to understand mode effects and how to account for them.

背景：访谈员施测的调查是收集人群层面流行病学数据的重要手段，但存在应答率下滑、成本持续走高的局限。网络调查的数据收集速度更快、成本更低，然而其数据质量问题仍受到广泛关注。既往相关研究多聚焦于选择性偏倚，极少针对测量差异展开探讨。本研究旨在评估同一受访者在两个时间点接受不同调查模式时的应答差异程度，为未来采用网络调查可能产生的测量误差提供参考信息。
方法：本研究纳入第三次英国性态度与生活方式全国调查（third British National Survey of Sexual Attitudes and Lifestyles, Natsal-3）的527名参与者，该调查原本采用计算机辅助个人访谈（computer assisted personal interview, CAPI）与计算机辅助自填问卷（computer assisted self-interview, CASI）两种模式，后续参与者需完成一项措辞完全一致的网络调查。通过麦克奈尔检验（McNemar tests）评估受访者内的应答差异属于随机误差，还是由调查模式差异引发的效应——即某一调查模式下敏感内容的报告率更高。采用广义估计方程（generalized estimating equations）对合并数据进行分析，探究性别与问题类型对应答变化的影响。
结果：仅有10%的应答在两次调查间发生改变。不过约三分之一的变量存在调查模式效应，与Natsal-3的调查模式相比，网络调查中敏感内容的报告率普遍更高。
结论：网络调查用于敏感行为类调查颇具应用前景，更适合作为混合模式调查的组成部分。本研究结果显示，调查模式效应可能因问题类型、调查内容以及所采用的特定模式组合而有所差异。混合模式调查需要进行严谨的开发与优化，以明确模式效应并掌握其校正策略。