Considerations on baseline generation for Imaging AI studies illustrated on the CT-based prediction of empyema and outcome assessment.

Considerations on baseline generation for Imaging AI studies illustrated on the CT-based prediction of empyema and outcome assessment.

Introduction: For AI-based classification tasks in computed tomography, a reference standard for evaluating the clinical diagnostic accuracy of individual classes is essential. To enable the implementation of an AI tool in clinical practice, this should be drawn from clinical routine data, using State-of-the-art scanners, evaluated in a blinded manner, and verified with a reference test.

Methods: 2659 consecutive CTs performed between 01/2016 and 01/2021 with reported pleural effusion were retrospectively included. Pathology reports from thoracocentesis or biopsy within 7 days of CT were used as reference standard (n = 335). Two radiologists (4 and 10 PGY) blindly assessed chest CTs (n=335, 81 empyemas) for pleural CT features and ICC was determined. In addition, both pleural CT features and radiological diagnosis were extracted from written radiological reports. If needed, consensus was achieved using an experienced radiologist's opinion (29 PGY). We assessed the correlation of these findings with the following patient outcomes: mortality and median hospital stay.

Results: Specificity and sensitivity for clinical detection of empyema (N=81) were 90.94 (95%-CI 86.55-94.05) and 72.84 (95%-CI: 61.63-81.85%) in all effusions, with moderate to almost perfect interrater agreement for all pleural findings associated with empyema (Cohen's kappa = 0.41-0.82). Features describing pleural enhancement or thickening achieved the highest accuracy with 87.02% and 81.49%, respectively. Empyema was associated with a longer hospital stay (median= 20 versus 14 days), and findings consistent with pleural carcinosis impacted mortality.