Supplementary data for the paper "Why psychologists should not default to Welch’s t-test instead of Student’s t-test (and why the Anderson–Darling test is an underused alternative)"

The choice between Student’s <em>t</em>-test (IT) and Welch’s t-test (WT) represents a central debate in statistical practice. This paper provides a re-evaluation of their performance by examining the combined effects of unequal variances, skewed distributions, and disparate sample sizes. For normal distributions, we confirm that the WT maintains the false positive rate close to the nominal level when sample sizes and standard deviations are unequal. However, the WT was found to yield inflated false positive rates under skewed distributions with unequal sample sizes. A complementary empirical study based on gender differences in two psychological scales corroborated these findings. Finally, we contend that the null hypothesis of unequal variances together with equal means is often implausible to begin with, and that empirically, a difference in means typically coincides with differences in variance and skewness. An additional analysis using the Kolmogorov-Smirnov and Anderson-Darling tests shows that examining entire distributions, rather than just their means, can provide a suitable alternative when facing unequal variances or skewed distributions. Given these results, researchers should remain cautious with software defaults favoring Welch’s test.