Performance evaluation of semantic segmentation models: a cross meta-frontier DEA approach

Performance evaluation of semantic segmentation models is an essential task because it helps to identify the best-performing model. Traditional methods, however, are generally concerned with the improvement of a single quality or quantity. Moreover, what causes low performance usually goes unnoticed. To address these issues, a new cross meta-frontier data envelopment analysis (DEA) approach is proposed in this article. For evaluating model performance comprehensively, not only accuracy metrics, but also hardware burden and model structure factors, are taken as DEA outputs and inputs, separately. In addition, the potential inefficiency is attributed to architectures and backbones <i>via</i> efficiency decomposition, so that it can find the sources of inefficiency and provides a direction for performance improvement. Finally, based on the proposed approach, the performance of 16 classical semantic segmentation models on the PASCAL VOC dataset are re-evaluated and explained. The results verify that the proposed approach can be considered as a comprehensive and interpretable performance evaluation technique, which expands the traditional accuracy-based measurement.