Policy Implications of Statistical Estimates: A General Bayesian Decision-Theoretic Model for Binary Outcomes

How should we evaluate the effect of a policy on the likelihood of an undesirable event, such as conflict? The significance test has three limitations. First, relying on statistical significance misses the fact that uncertainty is a continuous scale. Second, focusing on a standard point estimate overlooks the variation in plausible effect sizes. Third, the criterion of substantive significance is rarely explained or justified. A new Bayesian decision-theoretic model, “causal binary loss function model,” overcomes these issues. It compares the expected loss under a policy intervention with the one under no intervention. These losses are computed based on a particular range of the effect sizes of a policy, the probability mass of this effect size range, the cost of the policy, and the cost of the undesirable event the policy intends to address. The model is more applicable than common statistical decision-theoretic models using the standard loss functions or capturing costs in terms of false positives and false negatives. I exemplify the model’s use through three applications and provide an R package. Supplementary materials for this article are available online.

我们应当如何评估一项政策对冲突等不良事件发生概率的影响？显著性检验存在三大局限：其一，仅依赖统计显著性会忽略不确定性是一个连续谱系这一事实；其二，聚焦于标准点估计会忽视合理效应量的变异范围；其三，实质显著性的判定标准极少得到阐释与论证。全新的贝叶斯决策论模型（Bayesian decision-theoretic model）——“因果二元损失函数模型（causal binary loss function model）”可破解上述局限。该模型对比了政策干预与不干预两种情境下的期望损失，损失计算基于政策效应量的特定区间、该效应量区间的概率质量、政策实施成本，以及政策拟应对的不良事件所造成的损失。相较于采用标准损失函数或以假阳性、假阴性量化成本的常见统计决策论模型，本模型具备更强的适用性。文中通过三个应用案例阐释了该模型的使用方法，并提供了配套R包。本文的补充材料可在线获取。