On the Multilevel Nature of Meta-Analysis: A Tutorial, Comparison of Software Programs, and Discussion of Analytic Choices

The term “multilevel meta-analysis” is encountered not only in applied research studies, but in multilevel resources comparing traditional meta-analysis to multilevel meta-analysis. In this tutorial, we argue that the term “multilevel meta-analysis” is redundant since all meta-analysis can be formulated as a special kind of multilevel model. To clarify the multilevel nature of meta-analysis the four standard meta-analytic models are presented using multilevel equations and fit to an example data set using four software programs: two specific to meta-analysis (metafor in R and SPSS macros) and two specific to multilevel modeling (PROC MIXED in SAS and HLM). The same parameter estimates are obtained across programs underscoring that all meta-analyses are multilevel in nature. Despite the equivalent results, not all software programs are alike and differences are noted in the output provided and estimators available. This tutorial also recasts distinctions made in the literature between traditional and multilevel meta-analysis as differences between meta-analytic <i>choices</i>, not between meta-analytic <i>models</i>, and provides guidance to inform choices in estimators, significance tests, moderator analyses, and modeling sequence. The extent to which the software programs allow flexibility with respect to these decisions is noted, with metafor emerging as the most favorable program reviewed.

“多层元分析（multilevel meta-analysis）”这一术语不仅见于应用研究之中，也见于将传统元分析与多层元分析进行对比的多层级相关文献资源中。在本教程中，我们认为“多层元分析”这一表述存在冗余，因为所有元分析均可被构建为一类特殊的多层模型（multilevel model）。为阐明元分析的多层级本质，我们采用多层级方程形式呈现了四种标准元分析模型，并借助四款软件工具将其拟合至同一示例数据集：两款专为元分析设计的工具（R语言中的metafor包与SPSS宏程序），以及两款专为多层建模设计的工具（SAS中的PROC MIXED过程与HLM）。各软件工具所得参数估计结果完全一致，这进一步印证了所有元分析本质上均属于多层级分析范畴。尽管结果等价，但各软件工具并非完全相同，其输出格式与可用估计量均存在显著差异。本教程还将现有文献中传统元分析与多层元分析的差异，重新诠释为元分析「选择」而非元分析「模型」层面的差异，并为估计量选择、显著性检验、调节效应分析与建模流程提供了实操指导。同时我们梳理了各软件工具在上述决策维度上的灵活性差异，最终可见metafor包为本教程所评测的四款工具中最具优势的选择。