Benchmarking single-arm studies against historical controls from non-small cell lung cancer trials – an empirical analysis of bias

<b>Background:</b> Recent trials of novel agents in ‘rare’ molecular subtypes of non-small cell lung cancer (NSCLC) have used single-arm trial designs and benchmarked outcomes against historical controls. We assessed the consistency of historical control outcomes using docetaxel data from published NSCLC randomized controlled trials (RCTs). <b>Material and methods:</b> Advanced NSCLC RCTs including a docetaxel monotherapy arm were included. Heterogeneity in tumor objective response rates (ORRs), progression-free survival (PFS) and overall survival (OS), and correlations between outcomes and year of trial commencement were assessed. <b>Results:</b> Among 63 trials (<i>N</i> = 10,633) conducted between 2000 and 2017, ORR ranged from 0% to 26% (<i>I</i>² = 76.1%, <i>p</i>_{heterogeneity} I² = 86.0%, <i>p</i>_{heterogeneity} I² = 83.0%, <i>p</i>_{heterogeneity} p = .046), 0.5% (<i>p</i> = .11) and 0.9% (<i>p</i> = .001) improvement in ORR, 3-month PFS and 9-month OS rates, respectively. <b>Conclusions:</b> There was significant heterogeneity and an improving trend in docetaxel outcomes across trials conducted over 20 years. Benchmarking biomarker-targeted agents against historical controls may not be a valid approach to replace RCTs. Innovative study designs involving a concurrent control arm should be considered.