Coherent Tests for Interval Null Hypotheses

In a celebrated 1996 article, Schervish showed that, for testing interval null hypotheses, tests typically viewed as optimal can be logically incoherent. Specifically, one may fail to reject a specific interval null, but nevertheless—testing at the same level with the same data—reject a larger null, in which the original one is nested. This result has been used to argue against the widespread practice of viewing p-values as measures of evidence. In the current work we approach tests of interval nulls using simple Bayesian decision theory, and establish straightforward conditions that ensure coherence in Schervish’s sense. From these, we go on to establish novel frequentist criteria—different to Type I error rate—that, when controlled at fixed levels, give tests that are coherent in Schervish’s sense. The results suggest that exploring frequentist properties beyond the familiar Neyman–Pearson framework may ameliorate some of statistical testing’s well-known problems.

1996年的一篇经典论文中，舍维什（Schervish）指出，在区间原假设检验场景下，那些通常被视作最优的检验方法可能存在逻辑不相干性。具体而言，研究者可能无法拒绝某一特定的区间原假设，但基于同一数据、采用同一检验水准，却会拒绝包含该原假设的更大区间原假设。这一结论被用于驳斥将p值作为证据衡量标准的普遍实践。本研究采用简易贝叶斯决策理论分析区间原假设检验问题，并推导得到可确保检验符合舍维什所定义的相干性的简明条件。基于上述条件，本研究进一步提出了不同于一类错误率（Type I error rate）的新型频率学派检验准则：当将该准则控制在固定水平时，所得检验方法将满足舍维什定义下的相干性。研究结果表明，跳出广为人知的内曼-皮尔逊框架（Neyman–Pearson framework）探索频率学派检验性质，或可改善统计检验领域若干广为人知的固有问题。