Supplementary data for the paper "Why psychologists should not default to Welch’s t-test instead of Student’s t-test (and why the Anderson–Darling test is an underused alternative)"

The choice between Student’s <em>t</em>-test (IT) and Welch’s t-test (WT) represents a central debate in statistical practice. This paper provides a re-evaluation of their performance by examining the combined effects of unequal variances, skewed distributions, and disparate sample sizes. For normal distributions, we confirm that the WT maintains the false positive rate close to the nominal level when sample sizes and standard deviations are unequal. However, the WT was found to yield inflated false positive rates under skewed distributions with unequal sample sizes. A complementary empirical study based on gender differences in two psychological scales corroborated these findings. Finally, we contend that the null hypothesis of unequal variances together with equal means is often implausible to begin with, and that empirically, a difference in means typically coincides with differences in variance and skewness. An additional analysis using the Kolmogorov-Smirnov and Anderson-Darling tests shows that examining entire distributions, rather than just their means, can provide a suitable alternative when facing unequal variances or skewed distributions. Given these results, researchers should remain cautious with software defaults favoring Welch’s test.

学生t检验（Student’s t-test，简称IT）与韦尔奇t检验（Welch’s t-test，简称WT）的选择，是统计实践中核心的争议议题之一。本研究通过考察方差不齐、分布偏态与样本量不均的联合影响，重新评估了两种检验的性能。针对正态分布场景，我们证实了当样本量与标准差均不相等时，韦尔奇t检验可将假阳性率维持在接近名义水平的范围。然而，研究发现，在样本量不均且分布偏态的场景下，韦尔奇t检验会出现假阳性率膨胀的问题。一项基于两类心理量表性别差异的补充实证研究，验证了上述结论。最后，我们提出：方差不齐且均值相等的原假设，在初始条件下通常并不合理；且从实证层面来看，均值差异往往伴随方差与偏度的差异。额外采用柯尔莫哥洛夫-斯米尔诺夫检验（Kolmogorov-Smirnov）与安德森-达令检验（Anderson-Darling）开展的分析显示，当面临方差不齐或分布偏态的情况时，考察完整分布而非仅关注均值，可作为一种合适的替代方案。基于上述结果，研究人员对青睐韦尔奇检验的软件默认设置，应保持审慎态度。