Distributions and Sampling Variance for Five Effect Size Measures of Variance Overlap

Recent research indicates that Cohen’s d follows a non-central t-distribution, even as the central limit theorem approximates normal, contrary to the assumption that effects match the distribution of their test statistic (Cumming, 2012; Kelley, 2007; Smithson, 2003), and little research to date has focused on the variance overlap family of effects. Likewise, meta-analytic techniques also depend on accurate sampling variance estimations of effect sizes. To address this issue, we simulated 1,000 multivariate normal datasets for 1,152 combinations of varying sample sizes, standard deviations, levels, and level correlations. Eta, generalized eta (Olejnik & Algina 2003), omega, and all partial statistics were calculated for between and within subjects ANOVAs. Simulation results indicated an interactive pattern of mean effects and variances across conditions, with variance overlap measures reliably following a beta/gamma/normal distribution. We discuss the impact of design and data type on the measured effect and variance, as well as provide R scripts for simulations to adequately estimate sampling variance for meta-analyses.

近期研究指出，科恩氏d值遵循非中心t分布，即便中心极限定理近似正态分布，这与效应分布与其检验统计量相匹配的假设相悖（Cumming, 2012；Kelley, 2007；Smithson, 2003），迄今为止，关于方差重叠效应家族的研究尚显不足。同样，元分析方法亦依赖于效应大小的准确抽样方差估计。为解决此问题，我们模拟了1,000个多元正态数据集，涵盖了1,152种不同样本量、标准差、水平和水平相关性的组合。对于主效应和交互效应，我们计算了η、广义η（Olejnik & Algina 2003）、ω以及所有部分统计量。模拟结果表明，在条件之间和条件之内，平均效应和方差呈现出交互模式，方差重叠度量可靠地遵循贝塔/伽马/正态分布。我们探讨了设计和数据类型对测量效应和方差的影响，并提供了R脚本以充分估计元分析的抽样方差。