Heterogeneity of studies in anesthesiology meta-analyses: A meta-epidemiological review and proposal for evidence mapping.xlsx

Background

Heterogeneity among the
primary studies included in a systematic review is one of the most challenging
considerations for systematic reviewers. 
Current practices in anesthesiology systematic reviews have not been
evaluated, but traditional methods may not provide sufficient information to
evaluate the true nature of these differences. 
We address these issues by examining the practices of systematic
reviewers for evaluating heterogeneity in anesthesiology reviews.  We also propose a mapping method for
presenting heterogeneous aspects of the primary studies in systematic reviews.

Methods

We evaluated heterogeneity practices reported in systematic
reviews published in highly ranked anesthesiology journals. Elements extracted
from the SRs included heterogeneity tests, models used, analyses conducted,
plots used to display heterogeneity, and significant p-values. We also extracted
all I2 values from the SRs to quantify the level of heterogeneity. Additionally,
we selected a systematic review develop an evidence map in order to display
clinical heterogeneity.

Results

Our statistical analysis showed 150/207 SRs reporting a test for
statistical heterogeneity. Of the 150 SRs, 138 reported the use of a plot to
display statistical heterogeneity. Subgroup analyses were the most commonly
reported analysis (54%). Meta-regression and sensitivity analyses were used
sparingly (25%; 23% respectively). A random effects model was most commonly
reported (33%); however, most SRs did not report this information (38%). Heterogeneity
statistics across meta-analyses suggested that, in our sample, the majority (55%)
did not present sufficient heterogeneity to be of great concern.

Conclusions

Many
reviews did not provide sufficient detail regarding heterogeneity. We are
calling for improvement to reporting practices.