Comparison of models performance metrics.

Nigeria, with the second-largest HIV epidemic globally, faces challenges in achieving its HIV epidemic control goals by 2030, with interruptions in treatment (IIT) a significant challenge. Machine learning (ML) models can help HIV programs implement targeted interventions to improve the quality of care, develop effective early interventions, and provide insights into optimal resource allocation and program sustainability. This paper aims to identify predictors and measure the performance of models used to predict the risk of IIT among People Living with HIV (PLHIV) on antiretroviral therapy (ART). We trained multiple supervised ML algorithms on de-identified client-level electronic medical records data from a cohort of PLHIV across four Nigerian states. Merged demographic, clinic, pharmacy, and laboratory data were included as potential predictor variables in multiple models. The study analyzed data from 41,394 PLHIV, with 266,520 observations receiving treatment across four Nigerian states. The overall IIT rate was 33.7%, ranging from 17.7% in Cross River State to 42.4% in Niger State. The AdaBoost model demonstrated the best performance, with a sensitivity of 69.2%, specificity of 82.3%, F1 score of 0.678, and PR-AUC and ROC-AUC values of 0.563 and 0.843, respectively. Key predictors included PLHIV prior behavior, visit history, and geographic factors, while demographic features played a lesser role. This study highlights the utility of ML, particularly the AdaBoost model, in stratifying PLHIV by the risk of IIT. By leveraging ML, HIV programs can implement data-driven, targeted interventions to improve care continuity. However, further research is needed to address data biases and contextual challenges in resource-constrained settings.