MacroPCA: An All-in-One PCA Method Allowing for Missing Values as Well as Cellwise and Rowwise Outliers

Multivariate data are typically represented by a rectangular matrix (table) in which the rows are the objects (cases) and the columns are the variables (measurements). When there are many variables one often reduces the dimension by principal component analysis (PCA), which in its basic form is not robust to outliers. Much research has focused on handling rowwise outliers, that is, rows that deviate from the majority of the rows in the data (e.g., they might belong to a different population). In recent years also cellwise outliers are receiving attention. These are suspicious cells (entries) that can occur anywhere in the table. Even a relatively small proportion of outlying cells can contaminate over half the rows, which causes rowwise robust methods to break down. In this article, a new PCA method is constructed which combines the strengths of two existing robust methods to be robust against both cellwise and rowwise outliers. At the same time, the algorithm can cope with missing values. As of yet it is the only PCA method that can deal with all three problems simultaneously. Its name MacroPCA stands for PCA allowing for Missingness And Cellwise & Rowwise Outliers. Several simulations and real datasets illustrate its robustness. New residual maps are introduced, which help to determine which variables are responsible for the outlying behavior. The method is well-suited for online process control.多元数据通常以矩形矩阵（表格）的形式表示，其中行代表研究对象（样本），列代表变量（测量指标）。当变量维度较高时，研究者常通过主成分分析（Principal Component Analysis, PCA）进行降维，但基础形式的PCA对异常值不具备鲁棒性。过往大量研究聚焦于处理行异常值（rowwise outliers），即与数据集中绝大多数行偏离的行（例如，这些行可能属于不同的总体）。近年来，单元格异常值（cellwise outliers）也逐渐受到关注。这类异常值指的是表格中任意位置出现的可疑单元格（数据项）。即便仅存在占比相对较低的异常单元格，也可能污染超过半数的行，进而导致行鲁棒方法失效。本文提出一种全新的PCA方法，该方法融合了两种现有鲁棒方法的优势，可同时抵御单元格异常值与行异常值。与此同时，该算法还能够处理缺失值。截至目前，该方法是唯一可同时解决上述三类问题的PCA方法。其命名为MacroPCA，即允许缺失值且可应对单元格与行异常值的PCA（PCA allowing for Missingness And Cellwise & Rowwise Outliers）。多项模拟实验与真实数据集验证了该方法的鲁棒性。本文还引入了全新的残差图，可辅助定位导致异常表现的变量。该方法非常适用于在线过程控制场景。