Identifying and mitigating biases in perioperative prognostic models and clinical scoring systems

Digital health innovations have the potential to dramatically alter the way healthcare is delivered, improving access to cutting edge therapies and freeing up clinicians to focus on tasks humans do best. However, they can also exacerbate existing health inequity, and may become a new source of inequity, systematically disadvantaging certain groups in society. Predictive models and clinical scores (PMCS) use health data to make predictions & guide clinicians’ decision-making in diagnosis, treatment planning, prognosis, and other parts of the patient care pathway. Since the 1980s, thousands of PMCS have been developed, and their use has become ubiquitous throughout healthcare. Some PMCS assist clinicians make key decisions about whether to recommend entry to care pathways, for instance by estimating the risks associated with surgery for a particular patient. Other PMCS predict potential for clinical deterioration, and may partially govern referral to critical care services. Clinicians may expect that PMCS perform well for all patients, but we often lack studies to confirm this. A well-known surgical risk prediction score created in the UK had poor ‘calibration’ when tested in a New Zealand cohort, causing it to under-predict risk for patients. A 2020 evidence review by the National Institute for health and Care Excellence (NICE) highlighted similar issues with calibration for other PMCS predicting perioperative risk, though it did not comment on demographic subgroup performances. Given scores’ unpredictable performance at population level, it cannot be assumed that their performance and calibration is equitable across subgroups within populations. In order to safely use PMCS clinicians need to know how well they work for their patients; in particular they need to know when a particular model may work less well for a particular person or group. This project will use retrospective health data to calculate the performance of several widely used PMCS across demographic subgroups.