Benchmarking single-arm studies against historical controls from non-small cell lung cancer trials – an empirical analysis of bias

Background: Recent trials of novel agents in ‘rare’ molecular subtypes of non-small cell lung cancer (NSCLC) have used single-arm trial designs and benchmarked outcomes against historical controls. We assessed the consistency of historical control outcomes using docetaxel data from published NSCLC randomized controlled trials (RCTs). Material and methods: Advanced NSCLC RCTs including a docetaxel monotherapy arm were included. Heterogeneity in tumor objective response rates (ORRs), progression-free survival (PFS) and overall survival (OS), and correlations between outcomes and year of trial commencement were assessed. Results: Among 63 trials (N = 10,633) conducted between 2000 and 2017, ORR ranged from 0% to 26% (I2 = 76.1%, pheterogeneity I2 = 86.0%, pheterogeneity I2 = 83.0%, pheterogeneity p = .046), 0.5% (p = .11) and 0.9% (p = .001) improvement in ORR, 3-month PFS and 9-month OS rates, respectively. Conclusions: There was significant heterogeneity and an improving trend in docetaxel outcomes across trials conducted over 20 years. Benchmarking biomarker-targeted agents against historical controls may not be a valid approach to replace RCTs. Innovative study designs involving a concurrent control arm should be considered.