Supplementary data for the paper "Why psychologists should not default to Welch’s t-test instead of Student’s t-test (and why the Anderson–Darling test is an underused alternative)"

This paper evaluates the claim that Welch’s <em>t</em>-test (WT) should replace the independent-samples <em>t</em>-test (IT) as the default approach for comparing sample means. Through simulations of unequal and equal variances, skewed distributions, and varying sample sizes, we confirm that the WT effectively controls false positives when the smaller sample is drawn from the population with the larger standard deviation. However, the WT is found to yield inflated false positive rates under skewed distributions, even for relatively large sample sizes. By contrast, IT exhibits higher statistical power when standard deviations are equal, and avoids the inflation of false positives under skew. A complementary empirical study based on gender differences in two psychological scales corroborates these findings. Furthermore, an additional analysis using the Kolmogorov–Smirnov and Anderson–Darling tests demonstrated that examining entire distributions rather than just their means can provide a more suitable alternative when facing unequal variances or skewed data. Given these results, researchers should remain cautious with software defaults, such as R favoring Welch’s test.