Data Sheet 4_Change of d-irection: current limitations and future directions in psychological meta-analysis.pdf

Meta-analysis is a statistical tool used to combine the results of multiple studies to answer a research question. In psychology, effects are often measured on different scales (i.e., with different units), and their aggregation is not trivial. The problem is commonly solved using standardized effect sizes such as Cohen’s d. Despite being widely adopted, this approach is flawed. The misunderstanding is that standardized measures are dimensionless by definition—two d do not share the same dimension, as they do not have any. The present work explores alternative approaches to meta-analysis: Multivariate meta-analysis jointly models correlated outcomes while preserving their original unit; Imputation techniques treat the different outcome measures as a missing data problem. We evaluate these approaches through Monte Carlo simulation and an application to real data from psychotherapy studies. Results confirm that under missingness at random, multivariate meta-analysis provides meaningful and precise estimates. Imputation techniques offer an even more flexible alternative for dealing with non-ignorable missing outcome measures. The findings encourage the adoption of multivariate and imputation-based meta-analysis techniques to reduce bias, avoid research waste, and enhance the interpretability of psychological findings.