Geopackages for Model A utilized in analysis.

Earthquakes and other disasters often cause substantial damage to health facilities, impacting short-term response capacity and long-term health system needs. Identifying health facility damage following disasters is therefore crucial for coordinating response, but ground-based evaluations require substantial time and labor. Artificial intelligence (AI) models trained on satellite imagery can estimate building damage and could be used to generate rapid health facility damage reports. There is little published about methods of generating these estimates, testing real-world accuracy, or exploring error. This study presents a novel method of overlaying model damage outputs with health facility location data to generate health facility damage estimates following the February 2023 earthquake in Turkey. Two models were compared for agreement, accuracy, and errors. Building-level damage estimates were obtained for Model A (Microsoft neural network model), and Model B (Google AI model), and overlaid with health facility location data to identify facilities with significant damage. Model agreement, sensitivity and specificity for damage detection were calculated. A descriptive review of common error sources based on selected satellite imagery was conducted. A spatially aggregated damage estimation, based on proportion of buildings damaged in a 0.125km2 area, was also generated and assessed for each model. Twenty-five hospitals, 13 dialysis facilities, and 454 pharmacies were evaluated across three cities. Estimated damage was higher for Model A (10.4%) than Model B (4.3%). Cohen’s kappa was 0.32, indicating fair agreement. Sensitivity was low for both models at 42.9%, while specificity was high (A:93.6%, B:96.8%). Agreement and sensitivity were best for hospitals. Common errors included building identification and underestimation of damage for destroyed buildings. Spatially aggregated damage estimates yielded higher sensitivity (A:71.4%, B:57.1%) and agreement (Cohen’s kappa 0.38). Leveraging remote-sensing models for health facility damage assessment is feasible but currently lacks the sensitivity to replace ground evaluations. Improving building identification, damage detection for destroyed buildings, and spatially aggregating results may improve the performance and utility of these models for use in disaster response settings.