Do multiple outcome measures require p-value adjustment?

BACKGROUND: Readers may question the interpretation of findings in clinical trials when multiple outcome measures are used without adjustment of the p-value. This question arises because of the increased risk of Type I errors (findings of false "significance") when multiple simultaneous hypotheses are tested at set p-values. The primary aim of this study was to estimate the need to make appropriate p-value adjustments in clinical trials to compensate for a possible increased risk in committing Type I errors when multiple outcome measures are used. DISCUSSION: The classicists believe that the chance of finding at least one test statistically significant due to chance and incorrectly declaring a difference increases as the number of comparisons increases. The rationalists have the following objections to that theory: 1) P-value adjustments are calculated based on how many tests are to be considered, and that number has been defined arbitrarily and variably; 2) P-value adjustments reduce the chance of making type I errors, but they increase the chance of making type II errors or needing to increase the sample size. SUMMARY: Readers should balance a study's statistical significance with the magnitude of effect, the quality of the study and with findings from other studies. Researchers facing multiple outcome measures might want to either select a primary outcome measure or use a global assessment measure, rather than adjusting the p-value.

【背景】当临床试验中采用多重结局指标却未对P值（p-value）进行校正时，读者可能会对研究结果的解读提出质疑。这一质疑的根源在于：当基于设定的P值同时检验多个假说时，I类错误（Type I errors，即误判为“有统计学显著性”的假阳性结果）的发生风险会升高。本研究的核心目的，即评估当临床试验使用多重结局指标时，是否需要采取恰当的P值校正措施，以抵消I类错误风险升高可能带来的影响。
【讨论】传统学派认为，随着检验次数的增加，仅因随机误差便获得至少一项具有统计学显著性的检验结果、进而错误地判定存在组间差异的概率会随之升高。理性学派则对该理论提出如下两点反对意见：1）P值校正的计算依赖于待检验的数量，而该数量的定义具有任意性且缺乏统一标准；2）P值校正虽可降低I类错误的发生概率，但同时会提升II类错误（Type II errors）的发生风险，或是需要增大样本量。
【总结】读者应将研究的统计学显著性、效应量大小、研究质量以及其他同类研究的结果进行综合权衡。当研究者面临多重结局指标的选择时，不妨优先选定一项主要结局指标，或采用整体评估指标，而非直接对P值进行校正。