Regression Recalibration by Learning PIT Map Values

In an era dominated by large-scale machine learning models, poor calibration severely limits the trustworthiness of the results. As we increasingly rely on complex systems, recalibration becomes essential, where the objective is to find a mapping that improves the model’s probabilistic predictions. Motivated by the well-known quantile recalibration approach, we explore a broad class of recalibration functions based on learning transformations on the probability integral transform values that includes quantile recalibration as a special case. We derive solutions for the optimal mapping under this class of functions, and we propose a novel recalibration method that outperforms quantile recalibration in both calibration and sharpness in our empirical study. Additionally, we demonstrate the utility of our method in a case study on predicting global stratospheric temperatures following Mt. Pinatubo’s eruption using a convolutional neural network, where we show that our method can be used to adjust the model’s predictions for the post-eruption climate.

在大规模机器学习模型主导的时代，模型校准不佳会严重制约其输出结果的可信度。随着我们愈发依赖复杂系统，模型重校准已成为不可或缺的关键手段，其核心目标是寻找到能够优化模型概率预测结果的映射关系。受经典的分位数重校准（quantile recalibration）方法启发，我们探索了一类基于概率积分变换（probability integral transform）值学习变换的重校准函数，该类函数将分位数重校准作为其特例。我们推导了该类函数框架下最优映射的解析解，并提出了一种全新的重校准方法，经实证研究验证，该方法在校准性能与分布清晰度两项指标上均优于传统分位数重校准方法。此外，我们通过一项案例研究进一步验证了所提方法的实用价值：该研究基于卷积神经网络（convolutional neural network）预测皮纳图博火山（Mt. Pinatubo）喷发后的全球平流层温度，结果表明我们的方法可有效调整模型对喷发后气候的预测结果。