Assessment of Multiple Membership Multilevel Models: An Application to Interviewer Effects on Nonresponse

Multilevel multiple membership models account for situations where lower level units are nested within multiple higher level units from the same classification. Not accounting correctly for such multiple membership structures leads to biased results. The use of a multiple membership model requires selection of weights reflecting the hypothesized contribution of each level two unit and their relationship to the level one outcome. The Deviance Information Criterion (DIC) has been proposed to identify such weights. For the case of logistic regression, this study assesses, through simulation, the model identification rates of the DIC to detect the correct multiple membership weights, and the properties of model variance estimators for different weight specifications across a range of scenarios. The study is motivated by analyzing interviewer effects across waves in a longitudinal study. Interviewers can substantially influence the behavior of sample survey respondents, including their decision to participate in the survey. In the case of a longitudinal survey several interviewers may contact sample members to participate across different waves. Multilevel multiple membership models are suitable to account for the inclusion of higher-level random effects for interviewers at various waves, and to assess, for example, the relative importance of previous and current wave interviewers on current wave nonresponse. To illustrate the application, multiple membership models are applied to the UK Family and Children Survey to identify interviewer effects in a longitudinal study. The paper takes a critical view on the substantive interpretation of the model weights and provides practical guidance to statistical modelers. The main recommendation is that it is best to specify the weights in a multiple membership model by exploring different weight specifications based on the DIC, rather than prespecifying the weights.

多层多成员模型适用于低层级单元嵌套于同一分类体系下多个高层级单元的场景。若未正确考虑此类多成员结构，则会导致结果出现偏倚。应用多成员模型时，需选取权重以反映各第二层单元的假设性贡献，及其与第一层结果变量的关联。已有研究提出采用偏差信息准则（Deviance Information Criterion，DIC）来筛选此类权重。针对逻辑回归场景，本研究通过模拟实验，评估了DIC识别正确多成员权重的模型检出率，以及不同权重设定下多场景中模型方差估计量的性质。本研究的动因源自对一项纵向研究中不同调查波次访员效应的分析。访员可对抽样调查受访者的行为产生显著影响，包括其参与调查的意愿。在纵向调查中，不同调查波次可能会有多名访员联系样本成员参与调查。多层多成员模型可用于纳入不同波次访员的高层级随机效应，同时可评估例如前序波次与当前波次访员对当前波次无应答情况的相对重要性。为演示模型应用，本研究将多成员模型应用于英国家庭与儿童调查（UK Family and Children Survey），以识别某纵向研究中的访员效应。本文对模型权重的实质解读持批判性视角，并为统计建模人员提供了实操指南。本研究的核心建议为：在多成员模型中设定权重时，最佳实践是基于DIC探索不同权重设定方案，而非预先指定权重。