Multi-label Random Subspace Ensemble Classification¹

In this work, we develop a new ensemble learning framework, <i>multi-label Random Subspace Ensemble</i> (mRaSE), for multi-label classification. Given a base classifier (e.g., multinomial logistic regression, classification tree, K-nearest neighbors), mRaSE works by first randomly sampling a collection of subspaces, then choosing the best ones that achieve the minimum cross-validation errors and, finally, aggregating the chosen weak learners. In addition to its superior prediction performance, mRaSE also provides a model-free feature ranking depending on the given base classifier. An iterative version of mRaSE is also developed to further improve the performance. A model-free extension is pursued on the iterative version, leading to the so-called <i>Super mRaSE</i>, which accepts a collection of base classifiers as input to the algorithm. We show the proposed algorithms compared favorably with the state-of-the-art classification algorithm including random forest and deep neural network, via extensive simulation studies and two real data applications. The new algorithms are implemented in an updated version of the R package RaSEn.

本研究针对多标签分类任务，开发了一种全新的集成学习框架——多标签随机子空间集成（multi-label Random Subspace Ensemble，简称mRaSE）。在给定基分类器（如多项逻辑回归、分类树、K近邻（K-nearest neighbors））的前提下，mRaSE的工作流程为：首先随机采样一组子空间，随后筛选出交叉验证误差最小的最优子空间，最终对选出的弱学习器进行集成聚合。除具备优异的预测性能外，mRaSE还可基于所选用的基分类器实现无模型特征排序。为进一步提升模型性能，本研究还开发了mRaSE的迭代版本。针对该迭代版本进一步拓展出无模型变体，即所谓的Super mRaSE，该算法可接收多组基分类器作为算法输入。通过大量仿真实验与两项真实数据集应用，本文证明所提出的算法相较于随机森林、深度神经网络等当前前沿分类算法，具备更优的性能表现。上述新型算法已在更新版的R语言工具包RaSEn中完成实现。