A Simple Algorithm for Exact Multinomial Tests

This work proposes a new method for computing acceptance regions of exact multinomial tests. From this an algorithm is derived, which finds exact <i>p</i>-values for tests of simple multinomial hypotheses. Using concepts from discrete convex analysis, the method is proven to be exact for various popular test statistics, including Pearson’s Chi-square and the log-likelihood ratio. The proposed algorithm improves greatly on the naive approach using full enumeration of the sample space. However, its use is limited to multinomial distributions with a small number of categories, as the runtime grows exponentially in the number of possible outcomes. The method is applied in a simulation study, and uses of multinomial tests in forecast evaluation are outlined. Additionally, properties of a test statistic using probability ordering, referred to as the “exact multinomial test” by some authors, are investigated and discussed. The algorithm is implemented in the accompanying R package ExactMultinom. Supplementary materials for this article are available online.

本研究提出一种全新方法，用于计算精确多项检验（exact multinomial tests）的接受域。基于该方法推导得到的算法，可求解简单多项假设检验的精确p值（p-value）。该方法借助离散凸分析（discrete convex analysis）中的相关概念，经证实可针对皮尔逊卡方（Pearson’s Chi-square）、对数似然比（log-likelihood ratio）等多种常用检验统计量实现精确计算。所提算法相较基于样本空间（sample space）全枚举的朴素方法（naive approach），性能提升极为显著。但该方法的适用范围受限，仅适用于类别数量较少的多项分布，因为其运行时（runtime）随可能结果的数量呈指数级增长。本研究通过模拟研究（simulation study）验证了该方法的有效性，并概述了多项检验在预测评估（forecast evaluation）中的应用场景。此外，本文还针对部分学者所称“精确多项检验”的、基于概率排序（probability ordering）的检验统计量的性质展开了研究与讨论。该算法已在配套的R包（R package）ExactMultinom中实现。本文的补充材料（supplementary materials）可在线获取。