Predictive Distribution Modeling Using Transformation Forests

Regression models for supervised learning problems with a continuous response are commonly understood as models for the conditional mean of the response given predictors. This notion is simple and therefore appealing for interpretation and visualization. Information about the whole underlying conditional distribution is, however, not available from these models. A more general understanding of regression models as models for conditional distributions allows much broader inference, for example, the computation of prediction intervals or probabilistic predictions for exceeding certain thresholds. Several random forest-type algorithms aim at estimating conditional distributions, most prominently quantile regression forests. We propose a novel approach based on a parametric family of distributions characterized by their transformation function. A dedicated novel “transformation tree” algorithm able to detect distributional changes is developed. Based on these transformation trees, we introduce “transformation forests” as an adaptive local likelihood estimator of conditional distribution functions. The resulting predictive distributions are fully parametric yet very general and allow inference procedures, such as likelihood-based variable importances, to be applied in a straightforward way. Supplemental files for this article are available online.

针对带有连续响应变量的监督学习任务，回归模型通常被视作给定预测特征时响应变量的条件均值模型。该概念简洁直观，便于解释与可视化。然而，此类模型无法提供关于底层完整条件分布的相关信息。若将回归模型更广义地定义为条件分布模型，则可支持更为广泛的推断任务，例如计算预测区间或针对超出特定阈值的概率预测。现有多种随机森林类算法致力于估计条件分布，其中最具代表性的当属分位数回归森林（quantile regression forests）。本文提出一种基于以变换函数为特征的参数化分布族的全新方法，并开发了专门的新型“变换树”算法以检测分布变化。基于这些变换树，我们引入“变换森林”作为条件分布函数的自适应局部似然估计器。所得到的预测分布兼具完全参数化特性与极强的通用性，可直接应用基于似然的变量重要性等推断流程。本文的补充材料可在线获取。