Replication Package: Natural Language Interfaces for Databases — What Changes for Users? (SQL-LLM vs. Snowflake User Study)

Data and analysis code for a mixed-method, between-subjects user study comparing SQL-LLM, a natural-language database interface backed by GPT-4o, with Snowflake, a conventional SQL editor. Twenty SQL-literate participants each completed 12 querying tasks drawn from the BIRD Text-to-SQL benchmark across three relational schemas (Books, Mondial Geo, Legislator), with screen recordings, think-aloud protocols, and timing logs. The package contains the raw study inputs (cleaned timing/SQL spreadsheet, hand-coded behavioral proxies, 11 think-aloud transcripts, per-participant notebooks and reports), derived and graded artifacts (execution-based correctness grades, three LLM-judge grades, BIRD gold answers, model/timing CSVs, LLM inter-coder reliability files), and the full analysis pipeline (figure notebook, execution grader, gold-answer fetcher, difficulty-validation and effect-size scripts). See README.md for the layout and steps. The BIRD gold SQLite databases are not included but are regenerated from the public BIRD train split via the provided script. Code is released under the MIT License; data files under CC BY 4.0.