Paper-and-pencil versus computerized administration mode: Comparison of data quality and risk behavior prevalence estimates in the European school Survey Project on Alcohol and other Drugs (ESPAD)

Purpose The aim of this experimental study was to investigate whether paper-and-pencil and computerized surveys administered in the school setting yield equivalent data quality indicators and risk behavior prevalence estimates. Methods Data were drawn from the European School Survey Project on Alcohol and Other Drugs (ESPAD®) carried out in Italy to monitor drug, alcohol, tobacco use and other risk-behaviors among Italian high school students aged 15–19 years. A sub-sample of schools was recruited for the study (1673 pupils). For each school, two entire randomly selected courses (from the first to the fifth grade) participated and were assigned randomly to the self-administered paper-and-pencil (N = 811) or computerized survey (N = 862). Differences in data quality were assessed using the following indicators: questionnaire completeness (missing gender and/or 50% of missing answers) and internal consistency (repetitive extreme response patterns). Separate logistic regression models were used to estimate the mode effect on the reporting of each risk behavior, controlling for gender and age. Finally, the prevalence estimates of the experimental study were compared to the results of the national ESPAD® study. Results The computerized administration mode produced a higher proportion of invalid questionnaires, but the prevalence estimates generated from responses to the paper-and-pencil and computerized surveys were generally equivalent. Nevertheless, comparing these results with those of the national ESPAD® study, some differences in the prevalence rates were found. Conclusions The findings suggest that in a proctored school setting, the computerized survey mode yields almost the same results as the paper-and-pencil mode. However, because of the reliance on existing informatics facilities until when all schools in the country will be sufficiently equipped for the computerized data collection, they should be given the opportunity to choose between paper-and-pencil and computerized survey modes, in order to avoid a possible selection bias.Purpose 本实验研究旨在探究学校场景下施测的纸笔式与计算机化调查是否能获得等效的数据质量指标及危险行为流行率估计结果。 Methods 本研究数据取自于在意大利开展的欧洲学校酒精与其他药物调查项目（European School Survey Project on Alcohol and Other Drugs, ESPAD®），该项目用于监测意大利15至19岁高中生的药物、酒精、烟草使用情况及其他危险行为。 本研究招募了部分学校作为子样本，共纳入1673名学生。每所学校随机选取两个完整教学班（高一至五年级）参与研究，并将其随机分配至纸笔式自填问卷组（N=811）或计算机化问卷组（N=862）。研究通过以下指标评估数据质量差异：问卷完整性（缺失性别信息或超过50%的作答缺失）与内部一致性（重复出现极端作答模式）。分别构建逻辑回归模型，评估调查方式对各类危险行为报告情况的影响，并控制性别与年龄变量。最后，将本实验研究的流行率估计结果与全国ESPAD®研究的结果进行对比。 Results 计算机化调查方式的无效问卷占比更高，但纸笔式与计算机化问卷所得到的流行率估计结果整体一致。然而，将本研究结果与全国ESPAD®研究结果对比后，发现部分危险行为的流行率存在差异。 Conclusions 研究结果表明，在有监考的学校场景中，计算机化调查方式与纸笔式调查方式的结果几乎一致。但鉴于当前仍需依赖现有信息化设施开展调查，直至全国所有学校均具备足够的计算机化数据采集条件，因此应允许学校自主选择纸笔式或计算机化调查方式，以避免潜在的选择偏倚。