The Value of Precision in Probability Assessment: Evidence from a Large-Scale Geopolitical Forecasting Tournament

Scholars, practitioners, and pundits often leave their assessments of uncertainty vague when debating foreign policy, arguing that clearer probability estimates would provide arbitrary detail instead of useful insight. We provide the first systematic test of this claim using a data set containing 888,328 geopolitical forecasts. We find that coarsening numeric probability assessments in a manner consistent with common qualitative expressions—including expressions currently recommended for use by intelligence analysts—consistently sacrifices predictive accuracy. This finding does not depend on extreme probability estimates, short time horizons, particular scoring rules, or individual attributes that are difficult to cultivate. At a practical level, our analysis indicates that it would be possible to make foreign policy discourse more informative by supplementing natural language-based descriptions of uncertainty with quantitative probability estimates. More broadly, our findings advance long-standing debates over the nature and limits of subjective judgment when assessing social phenomena, showing how explicit probability assessments are empirically justifiable even in domains as complex as world politics.

学者、实务从业者与政论评论家在辩论外交政策时，常会对不确定性的评估表述模糊，并主张更精准的概率估算只会带来武断的细节填充，而非具有实用价值的洞见。本研究借助包含888,328条地缘政治预测的数据集（data set），首次对这一主张开展系统性检验。研究发现，以契合常见定性表述（含当前情报分析师推荐使用的表述方式）的路径对数值概率评估进行粗化处理，会持续削弱预测精度。该结论不受极端概率估算、短时间跨度、特定评分规则，或是难以培育的个体特质的影响。从实践层面而言，本研究分析表明，通过以定量概率估算补充基于自然语言的不确定性描述，可提升外交政策讨论的信息含量与参考价值。从更宏观的视角来看，本研究结论推进了关于评估社会现象时主观判断的本质与边界的长期学术争论，证明即便在世界政治这般复杂的领域，明确的概率评估也具备经验层面的合理性。