Policy Learning with Asymmetric Counterfactual Utilities

Data-driven decision making plays an important role even in high stakes settings like medicine and public policy. Learning optimal policies from observed data requires a careful formulation of the utility function whose expected value is maximized across a population. Although researchers typically use utilities that depend on observed outcomes alone, in many settings the decision maker’s utility function is more properly characterized by the joint set of potential outcomes under all actions. For example, the Hippocratic principle to “do no harm” implies that the cost of causing death to a patient who would otherwise survive without treatment is greater than the cost of forgoing life-saving treatment. We consider optimal policy learning with asymmetric counterfactual utility functions of this form that consider the joint set of potential outcomes. We show that asymmetric counterfactual utilities lead to an unidentifiable expected utility function, and so we first partially identify it. Drawing on statistical decision theory, we then derive minimax decision rules by minimizing the maximum expected utility loss relative to different alternative policies. We show that one can learn minimax loss decision rules from observed data by solving intermediate classification problems, and establish that the finite sample excess expected utility loss of this procedure is bounded by the regret of these intermediate classifiers. We apply this conceptual framework and methodology to the decision about whether or not to use right heart catheterization for patients with possible pulmonary hypertension. Supplementary materials for this article are available online.

即便在医学、公共政策这类高风险应用场景中，数据驱动决策也占据着举足轻重的地位。从观测数据中学习最优策略，需要对效用函数开展严谨的形式化建模，使其期望价值在目标人群中实现最大化。尽管现有研究通常仅采用依赖于观测结果的效用函数，但在诸多实际场景中，决策者的效用函数更合理的表征方式，应当是所有行动下潜在结果的联合集合。例如，希波克拉底"不伤害"原则指出，对本可在无需治疗的前提下存活的患者造成死亡的代价，远高于放弃挽救生命治疗所产生的代价。我们针对这类考虑潜在结果联合集合的非对称反事实效用（counterfactual utility）函数，展开最优策略学习研究。研究表明，非对称反事实效用会导致期望效用函数无法被完全识别，因此我们首先对其进行部分识别（partial identification）。随后基于统计决策理论，通过相较于不同备选策略最小化最大期望效用损失，推导出极小极大决策规则（minimax decision rules）。我们证明，可以通过求解中间分类任务从观测数据中学习极小极大损失决策规则，并证实该方法的有限样本超额期望效用损失，可由这些中间分类器的后悔（regret）值进行界定。我们将这一概念框架与方法论应用于疑似肺动脉高压患者是否实施右心导管术的决策问题。本文的补充材料可在线获取。