A Pseudo-Likelihood Approach to Linear Regression With Partially Shuffled Data

Recently, there has been significant interest in linear regression in the situation where predictors and responses are not observed in matching pairs corresponding to the same statistical unit as a consequence of separate data collection and uncertainty in data integration. Mismatched pairs can considerably impact the model fit and disrupt the estimation of regression parameters. In this article, we present a method to adjust for such mismatches under “partial shuffling” in which a sufficiently large fraction of (predictors, response)-pairs are observed in their correct correspondence. The proposed approach is based on a pseudo-likelihood in which each term takes the form of a two-component mixture density. expectation-maximization schemes are proposed for optimization, which (i) scale favorably in the number of samples, and (ii) achieve excellent statistical performance relative to an oracle that has access to the correct pairings as certified by simulations and case studies. In particular, the proposed approach can tolerate considerably larger fraction of mismatches than existing approaches, and enables estimation of the noise level as well as the fraction of mismatches. Inference for the resulting estimator (standard errors, confidence intervals) can be based on established theory for composite likelihood estimation. Along the way, we also propose a statistical test for the presence of mismatches and establish its consistency under suitable conditions. Supplemental files for this article are available online.

近年来，由于数据单独采集与数据集成过程中存在不确定性，预测变量与响应变量无法按对应同一统计单元的匹配对形式被观测到的线性回归场景受到了广泛关注。错配配对会显著影响模型拟合效果，并干扰回归参数的估计。本文提出一种针对“部分混排（partial shuffling）”场景下错配问题的校正方法，该场景中足够比例的（预测变量，响应变量）配对可被观测到其真实对应关系。所提方法基于伪似然（pseudo-likelihood）构建，其中每一项均采用两分量混合密度的形式。我们提出了用于优化的期望最大化（expectation-maximization）算法框架，该框架（i）在样本量上具备良好的缩放性，（ii）相较于可获取真实配对信息的神谕估计器，其统计性能优异，这一点已通过仿真与案例研究验证。具体而言，所提方法可容忍的错配比例远高于现有方法，同时能够同时估计噪声水平与错配比例。所得估计量的统计推断（标准误、置信区间）可基于成熟的复合似然估计理论完成。此外，本文还提出了一种用于检测错配存在性的统计检验，并在合适的条件下证明了该检验的相合性。本文的补充材料可在线获取。