Data Sheet 5_Change of d-irection: current limitations and future directions in psychological meta-analysis.pdf

Meta-analysis is a statistical tool used to combine the results of multiple studies to answer a research question. In psychology, effects are often measured on different scales (i.e., with different units), and their aggregation is not trivial. The problem is commonly solved using standardized effect sizes such as Cohen’s d. Despite being widely adopted, this approach is flawed. The misunderstanding is that standardized measures are dimensionless by definition—two d do not share the same dimension, as they do not have any. The present work explores alternative approaches to meta-analysis: Multivariate meta-analysis jointly models correlated outcomes while preserving their original unit; Imputation techniques treat the different outcome measures as a missing data problem. We evaluate these approaches through Monte Carlo simulation and an application to real data from psychotherapy studies. Results confirm that under missingness at random, multivariate meta-analysis provides meaningful and precise estimates. Imputation techniques offer an even more flexible alternative for dealing with non-ignorable missing outcome measures. The findings encourage the adoption of multivariate and imputation-based meta-analysis techniques to reduce bias, avoid research waste, and enhance the interpretability of psychological findings.

元分析（Meta-analysis）是一种用于整合多项研究结果以解答特定研究问题的统计学工具。在心理学研究中，效应值常通过不同量表（即采用不同计量单位）进行测量，因此对其进行整合并非易事。学界通常采用科恩d值（Cohen’s d）这类标准化效应量来解决该问题，但该方法虽被广泛应用，却存在缺陷。存在的误区在于，标准化效应量按定义属于无量纲量——但两个科恩d值并不共享同一量纲，究其本质，它们本就不具备任何量纲。本研究探讨了元分析的两类替代方案：其一为多元元分析（Multivariate meta-analysis），可在保留原始计量单位的前提下，对存在相关性的结局指标进行联合建模；其二为插补法（Imputation techniques），即将不同的结局测量方式视为缺失数据问题进行处理。本研究通过蒙特卡洛模拟（Monte Carlo simulation）与心理治疗研究的真实数据集应用，对上述两种方案进行了评估。结果证实，在随机缺失机制下，多元元分析可得到兼具实际意义与精准度的效应估计值；而插补法则为处理不可忽略的缺失结局指标提供了更为灵活的解决方案。本研究结果支持采用多元元分析与基于插补法的元分析技术，以降低统计偏倚、避免研究资源浪费，并提升心理学研究结果的可解释性。