Forward Stepwise Deep Autoencoder-Based Monotone Nonlinear Dimensionality Reduction Methods

Dimensionality reduction is an unsupervised learning task aimed at creating a low-dimensional summary and/or extracting the most salient features of a dataset. Principal component analysis is a linear dimensionality reduction method in the sense that each principal component is a linear combination of the input variables. To allow features that are nonlinear functions of the input variables, many nonlinear dimensionality reduction (NLDR) methods have been proposed. In this article, we propose novel NLDR methods based on bottleneck deep autoencoders. Our contributions are 2-fold: (1) We introduce a monotonicity constraint into bottleneck deep autoencoders for estimating a single nonlinear component and propose two methods for fitting the model. (2) We propose a new, forward stepwise deep learning architecture for estimating multiple nonlinear components. The former helps extract interpretable, monotone components when the assumption of monotonicity holds, and the latter helps evaluate reconstruction errors in the original data space for a range of components. We conduct numerical studies to compare different model fitting methods and use two real data examples from the studies of human immune responses to HIV to illustrate the proposed methods. Supplementary materials for this article are available online.降维（Dimensionality Reduction）是一类无监督学习任务，旨在生成数据集的低维摘要，或是提取其中最具显著性的特征。主成分分析（Principal Component Analysis, PCA）是一种线性降维方法，其核心在于每个主成分均为输入变量的线性组合。为了能够建模输入变量的非线性函数特征，诸多非线性降维（Nonlinear Dimensionality Reduction, NLDR）方法已被学界提出。本文提出了基于瓶颈深度自编码器（bottleneck deep autoencoders）的新型非线性降维方法。本文的贡献主要包含两点：其一，针对单非线性成分的估计任务，我们将单调性约束引入瓶颈深度自编码器，并提出了两种模型拟合方法；其二，针对多非线性成分的估计，我们设计了一种全新的前向逐步深度学习架构。当单调性假设成立时，前者可提取具备可解释性的单调成分；后者则可针对一系列成分，在原始数据空间中完成重构误差的评估。我们通过数值实验对比了不同的模型拟合方法，并选取两项关于人体HIV免疫应答研究的真实数据集示例，对所提出的方法进行演示说明。本文的补充材料可在线获取。