Multiclass Probability Estimation With Support Vector Machines

Multiclass classification and probability estimation have important applications in data analytics. Support vector machines (SVMs) have shown great success in various real-world problems due to their high classification accuracy. However, one main limitation of standard SVMs is that they do not provide class probability estimates, and thus fail to offer uncertainty measure about class prediction. In this article, we propose a simple yet effective framework to endow kernel SVMs with the feature of multiclass probability estimation. The new probability estimator does not rely on any parametric assumption on the data distribution, therefore, it is flexible and robust. Theoretically, we show that the proposed estimator is asymptotically consistent. Computationally, the new procedure can be conveniently implemented using standard SVM softwares. Our extensive numerical studies demonstrate competitive performance of the new estimator when compared with existing methods such as multiple logistic regression, linear discrimination analysis, tree-based methods, and random forest, under various classification settings. Supplementary materials for this article are available online.

多分类任务与概率估计在数据分析领域拥有重要应用价值。支持向量机（SVMs）凭借其优异的分类精度，已在各类实际问题中取得了显著成效。然而，标准支持向量机存在一项核心局限：无法输出类别概率估计结果，因此无法提供分类预测的不确定性度量。本文提出了一种简洁且高效的框架，可为核支持向量机赋予多分类概率估计的能力。该新型概率估计器无需对数据分布做出任何参数化假设，因此具备灵活性与鲁棒性。理论层面，本文证明所提估计器具有渐近一致性；计算层面，该新流程可借助标准支持向量机软件便捷实现。本文通过大量数值实验证明，在多种分类设置下，相较于多重逻辑回归、线性判别分析、树基方法以及随机森林等现有方法，所提估计器的性能具备竞争力。本文补充材料可在线获取。