Earth Observation GeoSQL Query Set for Natural Language to PostGIS Agentic Evaluation

Earth Observation GeoSQL Query Set Overview The Earth Observation GeoSQL Query Set is a curated evaluation dataset designed to benchmark the performance of Large Language Models (LLMs) and agentic systems in translating Natural Language to complex Spatial SQL (PostGIS). Unlike standard generic Text-to-SQL benchmarks (e.g., Spider, BIRD), this dataset focuses on the specific challenges of geospatial databases, including topological relationships, metric constraints requiring `geography` casting, coordinate reference system (SRID) awareness, and spatial aggregations. This dataset was created to evaluate GeoAgent, an LLM-agnostic, graph-based agentic architecture for PostGIS interaction. Database Schema Context To successfully execute the expected SQL queries, the target PostgreSQL/PostGIS database utilizes the following simplified relational schema: `categories`: A lookup table for simulation types Columns: `id` (PK), `name` (e.g., 'Aree verdi urbane', 'Disboscamento'). `simulations`: The main table storing simulation metadata and bounding boxes. Columns: `id` (PK), `category_id` (FK), `name`, `geometry` (PostGIS Geometry, SRID 4326), `created_at`. `simulations_layers`: Stores the actual, highly detailed granular geometries (results) of the simulations. Columns: `id` (PK), `simulations_id` (FK), `geometry` (PostGIS Geometry, SRID 4326). Dataset Structure The dataset is provided as a JSON array of objects. Each object represents a single evaluation query and contains the following fields: `id`: Unique identifier for the query. `question`: The natural language prompt simulating a user's request. `category`: The type of spatial operation required (e.g., `spatial_filter`, `spatial_aggregation`, `spatial_optimization`). `complexity`: An estimated difficulty level (`simple`, `medium`, `high`, `very_high`). `has_spatial`: Boolean indicating the presence of a spatial predicate. `expected_sql`: The target deterministic PostGIS SQL query used as the ground truth for execution accuracy. `notes`: Contextual information detailing the specific PostGIS functions or edge cases tested by the query (e.g., use of `<->` KNN operator, implicit geocoding, `ST_DWithin` casting). Usage This dataset is intended to be used with an execution-based evaluation metric (Execution Accuracy). Systems should be evaluated on whether the generated SQL produces the exact same result set as the `expected_sql` when executed against a populated database following the schema above. Citation If you use this dataset in your research, please cite our accompanying paper:> Piccolo, G., & Sada, C. (2026). A Graph-based and LLM-agnostic Agentic Approach for Natural Language Interaction with PostGIS Databases.