Clinical trial generalizability assessment in the big data era: a review

Clinical studies, especially randomized controlled trials, are essential
for generating evidence for clinical practice.  However,
generalizability is a long-standing concern when applying trial results to
real-world patients.  Generalizability assessment is thus
important, nevertheless, not consistently practiced.  We
performed a systematic scoping review to understand the practice of
generalizability assessment.  We identified 187 relevant papers
and systematically organized these studies in a taxonomy with three
dimensions: (1) data availability (i.e., before or after trial [a priori
vs a posteriori generalizability]), (2) result outputs (i.e., score vs
non-score), and (3) populations of interest.  We further reported
disease areas, underrepresented subgroups, and types of data used to
profile target populations.  We observed an increasing trend of
generalizability assessments, but less than 30% of studies reported
positive generalizability results.  As a priori generalizability
can be assessed using only study design information (primarily eligibility
criteria), it gives investigators a golden opportunity to adjust the study
design before the trial starts.  Nevertheless, less than 40% of
the studies in our review assessed a priori generalizability.
 With the wide adoption of electronic health records systems,
rich real-world patient databases are increasingly available for
generalizability assessment; however, informatics tools are lacking to
support the adoption of generalizability assessment practice.