Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experiments*

Researchers are increasingly turning to machine learning (ML) algorithms to investigate causal heterogeneity in randomized experiments. Despite their promise, ML algorithms may fail to accurately ascertain heterogeneous treatment effects under practical settings with many covariates and small sample size. In addition, the quantification of estimation uncertainty remains a challenge. We develop a general approach to statistical inference for heterogeneous treatment effects discovered by a generic ML algorithm. We apply the Neyman’s repeated sampling framework to a common setting, in which researchers use an ML algorithm to estimate the conditional average treatment effect and then divide the sample into several groups based on the magnitude of the estimated effects. We show how to estimate the average treatment effect within each of these groups, and construct a valid confidence interval. In addition, we develop nonparametric tests of treatment effect homogeneity across groups, and rank-consistency of within-group average treatment effects. The validity of our methodology does not rely on the properties of ML algorithms because it is solely based on the randomization of treatment assignment and random sampling of units. Finally, we generalize our methodology to the cross-fitting procedure by accounting for the additional uncertainty induced by the random splitting of data.

研究者正愈发多地借助机器学习（Machine Learning）算法，探究随机实验中的因果异质性。尽管此类算法颇具应用前景，但在协变量繁多、样本量有限的实际场景中，ML算法或无法精准识别异质性处理效应。此外，估计不确定性的量化仍是一项难题。我们针对由通用ML算法识别出的异质性处理效应，提出了一套通用的统计推断方法。我们将内曼重复抽样框架（Neyman's repeated sampling framework）应用于一类常见场景：研究者先用ML算法估计条件平均处理效应（Conditional Average Treatment Effect），再依据估计效应的大小将样本划分为若干组别。我们阐述了如何估算每个组别内的平均处理效应，并构建有效的置信区间。此外，我们还提出了用于检验组间处理效应同质性以及组内平均处理效应秩一致性的非参数检验方法。我们所提方法的有效性不依赖于ML算法的自身性质，因为其仅基于处理分配的随机化与研究单元的随机抽样。最后，我们通过考虑数据随机划分所引入的额外不确定性，将该方法推广至交叉拟合（Cross-fitting）流程中。