Multi-label Random Subspace Ensemble Classification¹

In this work, we develop a new ensemble learning framework, <i>multi-label Random Subspace Ensemble</i> (mRaSE), for multi-label classification. Given a base classifier (e.g., multinomial logistic regression, classification tree, K-nearest neighbors), mRaSE works by first randomly sampling a collection of subspaces, then choosing the best ones that achieve the minimum cross-validation errors and, finally, aggregating the chosen weak learners. In addition to its superior prediction performance, mRaSE also provides a model-free feature ranking depending on the given base classifier. An iterative version of mRaSE is also developed to further improve the performance. A model-free extension is pursued on the iterative version, leading to the so-called <i>Super mRaSE</i>, which accepts a collection of base classifiers as input to the algorithm. We show the proposed algorithms compared favorably with the state-of-the-art classification algorithm including random forest and deep neural network, via extensive simulation studies and two real data applications. The new algorithms are implemented in an updated version of the R package RaSEn.本研究针对多标签分类任务，构建了一种全新的集成学习框架——多标签随机子空间集成（multi-label Random Subspace Ensemble，简称mRaSE）。当以基础分类器（如多项式逻辑回归、分类树、K近邻算法）作为基学习器时，mRaSE的运行流程为：首先随机采样一组子空间，随后筛选出交叉验证误差最小的最优子空间，最终对筛选出的弱学习器进行集成聚合。除具备优异的预测性能外，mRaSE还可基于所选用的基础分类器实现无模型特征排序。为进一步提升模型性能，本研究还开发了mRaSE的迭代版本。针对该迭代版本，我们进一步开展了无模型扩展研究，由此得到名为Super mRaSE的算法，该算法可接收多组基础分类器作为算法输入。通过大量仿真实验与两个真实数据集应用，我们证明所提出的算法性能优于包括随机森林（random forest）、深度神经网络（deep neural network）在内的当前前沿分类算法。上述新型算法已在更新版R语言包RaSEn中实现。