JABUTI-SQL: A Portuguese Benchmark for Text-to-SQL Evaluation in Real Healthcare Scenarios

JABUTI-SQL (Joint Annotated Benchmark for User-centric Text-to-SQL In Real Health Data) is a Brazilian benchmark for Text-to-SQL built from real public healthcare data provided by DATASUS, the Brazilian public health data platform. The benchmark integrates multiple heterogeneous sources related to pharmaceutical assistance, public procurement, hospital infrastructure, and regional healthcare organization into a unified relational schema. Unlike synthetic benchmarks, JABUTI-SQL preserves characteristics commonly found in real governmental databases, including incomplete records, semantic inconsistencies, heterogeneous structures, and complex inter-table relations. The benchmark contains 69 Portuguese question–SQL pairs covering three user personas and multiple difficulty levels, aiming to support the evaluation of Text-to-SQL systems under more realistic conditions. The dataset integrates data from: Banco de Preços em Saúde (BPS), Hospitais e Leitos, BNAFAR (pharmaceutical assistance), CNES (national health establishment registry), and regional health divisions (Macrorregião and Região de Saúde). Code available at: https://github.com/BotelhoMDS/JABUTI-SQL