Marginally-calibrated deep distributional regression

Deep neural network (DNN) regression models are widely used in applications requiring state-of-the-art predictive accuracy. However, until recently there has been little work on accurate uncertainty quantification for predictions from such models. We add to this literature by outlining an approach to constructing predictive distributions that are ‘marginally calibrated’. This is where the long run average of the predictive distributions of the response variable matches the observed empirical margin. Our approach considers a DNN regression with a conditionally Gaussian prior for the final layer weights, from which an implicit copula process on the feature space is extracted. This copula process is combined with a non-parametrically estimated marginal distribution for the response. The end result is a scalable distributional DNN regression method with marginally calibrated predictions, and our work complements existing methods for probability calibration. The approach is first illustrated using two applications of dense layer feed-forward neural networks. However, our main motivating applications are in likelihood-free inference, where distributional deep regression is used to estimate marginal posterior distributions. In two complex ecological time series examples we employ the implicit copulas of convolutional networks, and show that marginal calibration results in improved uncertainty quantification. Our approach also avoids the need for manual specification of summary statistics, a requirement that is burdensome for users and typical of competing likelihood-free inference methods.深度神经网络（Deep Neural Network，DNN）回归模型广泛应用于追求当前最优预测精度的各类场景。然而直至近期，针对此类模型的预测结果开展精准不确定性量化的研究仍较为匮乏。 本研究为该领域补充了新的研究成果，提出了一种构建具备「边际校准」特性的预测分布的方法。所谓边际校准，即响应变量预测分布的长期均值与观测到的经验边际分布相一致。 我们的方法采用最终层权重服从条件高斯先验的DNN回归框架，从中提取特征空间上的隐式Copula过程（copula），并将该过程与响应变量的非参数估计边际分布相结合。最终得到一种具备可扩展性的分布型DNN回归方法，其预测结果具备边际校准特性，本研究也为现有的概率校准方法提供了有益补充。 本方法首先通过两个稠密层前馈神经网络的应用案例进行演示。 但本研究的核心应用场景为无似然推断，该场景下利用分布型深度回归方法估计边际后验分布。在两个复杂的生态时间序列案例中，我们采用卷积神经网络的隐式Copula过程，并证实边际校准可有效提升不确定性量化的效果。 此外，本方法无需手动指定汇总统计量——这一要求不仅给使用者带来沉重负担，也是现有同类无似然推断方法的普遍痛点。