Statistical analysis

The integration of AI in the evaluation of Chest X-rays could assist clinicians and enhance accuracy and workflows. We evaluated sensitivity (Se) and specificity (Sp) of an AI software and a radiology resident in interpreting Chest X-ray examinations referred from the Emergency Department (ED) against a senior radiologist (Gold Standard, GS). We also assessed the concordance rate between AI and the resident, described the frequency of doubtful cases in each category and determine how many were considered positive by the GS and evaluated other variables AI is not trained to detect. We conducted an observational, retrospective study, analyzing thoracic radiographs from a sample of 784 patients referred from the ED at our hospital. AI is trained to detect five categorical variables (pulmonary nodule, pulmonary opacity, pleural effusion, pneumothorax, and fracture) and associate them with a confidence label ("positive," "doubtful," or "negative"). Se for fracture and pneumothorax was high (100%), moderate for pulmonary opacity (AI= 76%, resident= 71%), and reasonable for pleural effusion (AI= 60%, resident=67%), with NPV>95% and AUC>0.8. For pulmonary nodule, the resident's Se was moderate (75%), while AI's was low (33%). When AI doubted, only a few of these doubtful diagnoses were considered positive by the GS. The resident doubted less. The Kappa Coefficient between resident and AI was fair (0.3) for all variables, except for pleural effusion, which was moderate (0.5). The prevalence of other variables was: 16% mediastinal abnormalities, 20% surgical material, and 20% other pulmonary findings. AI would be useful to detect most of the aforementioned items, except for pulmonary nodules. Its high NPV stands out, making it useful for screening. When AI doubts, most of the findings are negative. It would be useful to train AI to detect additional findings.