Policy Implications of Statistical Estimates: A General Bayesian Decision-Theoretic Model for Binary Outcomes

How should we evaluate the effect of a policy on the likelihood of an undesirable event, such as conflict? The significance test has three limitations. First, relying on statistical significance misses the fact that uncertainty is a continuous scale. Second, focusing on a standard point estimate overlooks the variation in plausible effect sizes. Third, the criterion of substantive significance is rarely explained or justified. A new Bayesian decision-theoretic model, “causal binary loss function model,” overcomes these issues. It compares the expected loss under a policy intervention with the one under no intervention. These losses are computed based on a particular range of the effect sizes of a policy, the probability mass of this effect size range, the cost of the policy, and the cost of the undesirable event the policy intends to address. The model is more applicable than common statistical decision-theoretic models using the standard loss functions or capturing costs in terms of false positives and false negatives. I exemplify the model’s use through three applications and provide an R package. Supplementary materials for this article are available online.

我们应如何评估一项政策对恶性事件（如冲突）发生概率的影响？显著性检验存在三项局限：其一，依赖统计显著性会忽略不确定性本为连续尺度这一事实；其二，聚焦标准点估计会忽视合理效应量的变异情况；其三，实质显著性的判定标准极少得到阐释与论证。
一款全新的贝叶斯决策论模型——「因果二元损失函数模型（causal binary loss function model）」可克服上述缺陷。该模型对比政策干预与无干预两种场景下的预期损失，损失计算基于政策效应量的特定区间、该效应量区间的概率质量、政策实施成本，以及该政策拟应对的恶性事件的发生成本。相较于采用标准损失函数，或以假阳性、假阴性衡量成本的常见统计决策论模型，本模型具备更强的适用性。本文通过三个应用案例阐释该模型的使用方法，并配套提供了R包。本文的补充材料可在线获取。