Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation

This article presents Individual Conditional Expectation (ICE) plots, a tool for visualizing the model estimated by any supervised learning algorithm. Classical partial dependence plots (PDPs) help visualize the average partial relationship between the predicted response and one or more features. In the presence of substantial interaction effects, the partial response relationship can be heterogeneous. Thus, an average curve, such as the PDP, can obfuscate the complexity of the modeled relationship. Accordingly, ICE plots refine the partial dependence plot by graphing the functional relationship between the predicted response and the feature for <i>individual</i> observations. Specifically, ICE plots highlight the variation in the fitted values across the range of a covariate, suggesting where and to what extent heterogeneities might exist. In addition to providing a plotting suite for exploratory analysis, we include a visual test for additive structure in the data generating model. Through simulated examples and real data sets, we demonstrate how ICE plots can shed light on estimated models in ways PDPs cannot. Procedures outlined are available in the R package ICEbox.

本文介绍了个体条件期望（Individual Conditional Expectation, ICE）图——一种可用于可视化任意监督学习算法所估计模型的可视化工具。经典偏依赖图（Partial Dependence Plots, PDPs）用于刻画预测响应与单个或多个特征之间的平均偏依赖关系。当存在显著交互效应时，偏响应关系往往呈现异质性，此时诸如偏依赖图这类平均曲线可能会掩盖建模关系的复杂特性。有鉴于此，个体条件期望图对偏依赖图进行了优化：针对个体观测样本绘制预测响应与特征之间的函数关系图。具体而言，ICE图可展示协变量取值范围内拟合值的分布差异，从而揭示异质性可能存在的位置与程度。除了为探索性分析提供可视化工具集之外，本文还提出了一种用于检验数据生成模型中加性结构的可视化检验方法。通过模拟示例与真实数据集，本文展示了个体条件期望图如何以偏依赖图无法实现的方式，帮助研究者剖析已估计的模型。本文所述方法可通过R包ICEbox实现。