Data_Sheet_3_External validation of the parental attitude about childhood vaccination scale.doc

IntroductionInternal validation techniques alone do not guarantee the value of a model. This study aims to investigate the external validity of the Parental Attitude toward Childhood Vaccination (PACV) scale for assessing parents’ attitude toward seasonal influenza vaccination.
MethodsUsing a snowball sampling approach, an anonymous online questionnaire was distributed in two languages (English and Arabic) across seven countries. To assess the internal validity of the model, the machine learning technique of “resampling methods” was used to repeatedly select various samples collected from Egypt and refit the model for each sample. The binary logistic regression model was used to identify the main determinants of parental intention to vaccinate their children against seasonal influenza. We adopted the original model developed and used its predictors to determine parents’ intention to vaccinate their children in Libya, Lebanon, Syria, Iraq, Palestine, and Sudan. The area under the curve (AUC) indicated the model’s ability to distinguish events from non-events. We visually compared the observed and predicted probabilities of parents’ intention to vaccinate their children using a calibration plot.
ResultsA total of 430 parents were recruited from Egypt to internally validate the model, and responses from 2095 parents in the other six countries were used to externally validate the model. Multivariate regression analysis showed that the PACV score, child age (adolescence), and Coronavirus disease 2019 (COVID-19) vaccination in children were significantly associated with the intention to receive the vaccination. The AUC of the developed model was 0.845. Most of the predicted points were close to the diagonal line, demonstrating better calibration (the prediction error was 16.82%). The sensitivity and specificity of the externally validated model were 89.64 and 37.89%, respectively (AUC = 0.769).
ConclusionThe PACV showed similar calibration and discrimination across the six countries. It is transportable and can be used to assess attitudes towards influenza vaccination among parents in different countries using either the Arabic or English version of the scale.

引言
仅靠内部验证技术无法确保模型的应用价值。本研究旨在检验儿童时期疫苗接种父母态度量表（Parental Attitude toward Childhood Vaccination, PACV）的外部效度，该量表用于评估父母对季节性流感疫苗接种的态度。

方法
本研究采用滚雪球抽样法，面向七个国家以英语及阿拉伯语发放匿名在线问卷。为评估模型的内部效度，本研究采用机器学习中的重采样方法（resampling methods），反复选取从埃及收集的不同样本，并针对每个样本重新拟合模型。本研究采用二元逻辑回归模型，以识别父母愿意为子女接种季节性流感疫苗的主要影响因素。我们采用已开发的原始模型，利用其预测变量，针对利比亚、黎巴嫩、叙利亚、伊拉克、巴勒斯坦及苏丹的父母，分析其为子女接种疫苗的意愿。曲线下面积（area under the curve, AUC）用以表征模型区分事件与非事件的能力。本研究通过校准曲线（calibration plot），可视化比较父母接种意愿的观测概率与预测概率。

结果
共计招募埃及的430名父母以完成模型的内部效度检验，其余六国的2095名父母的问卷回复则用于模型的外部效度检验。多变量回归分析结果显示，PACV得分、儿童年龄（青春期阶段）以及儿童接种新型冠状病毒肺炎（Coronavirus Disease 2019, COVID-19）疫苗，均与父母的疫苗接种意愿显著相关。所构建模型的AUC值为0.845。多数预测点贴近对角线，表明模型校准性能良好（预测误差为16.82%）。经外部效度检验的模型灵敏度为89.64%，特异度为37.89%（AUC=0.769）。

结论
PACV量表在六国中展现出一致的校准性能与区分能力。该量表具备可迁移性，可通过其英语或阿拉伯语版本，用于评估不同国家父母对流感疫苗接种的态度。