Real-time Spatial Retrieval Augmented Generation for Urban Environments

Real-time Spatial Retrieval Augmented Generation for Urban Environments Overview This repository contains the datasets, code, and results associated with the paper: Real-time Spatial Retrieval Augmented Generation for Urban EnvironmentsDavid Nazareno Campo, Javier Conde, Álvaro Alonso, Gabriel Huecas, Joaquín Salvachúa, and Pedro ReviriegoACM Transactions on Intelligent Systems and Technology (TIST), June 2025ETSI de Telecomunicación — Universidad Politécnica de Madrid (UPM), Spain The paper proposes a real-time spatial RAG architecture for integrating Large Language Models (LLMs) and Urban Foundation Models (UFMs) into smart city environments. The architecture is implemented using FIWARE, an open-source ecosystem for smart city solutions, and validated through two use cases in the city of Madrid: a tourism assistant and a traffic light and luminaire management system. Repository Structure . ├── docker-compose.yml # Qdrant vector database deployment ├── dbs_qdrant/ # Pre-built Qdrant collection snapshots │ ├── # === DATASETS === ├── poi_10.json # Points of Interest — 10 entities ├── poi_100.json # Points of Interest — 100 entities ├── poi_650.json # Points of Interest — 650 entities ├── traffic_light_10.json # Traffic lights — 10 entities ├── traffic_light_100.json # Traffic lights — 100 entities ├── traffic_light_650.json # Traffic lights — 650 entities ├── luminaries_10.json # Luminaires — 10 entities ├── luminaries_100.json # Luminaires — 100 entities ├── luminaries_650.json # Luminaires — 650 entities ├── traffic_light_and_luminaries_20.json # Combined dataset — 20 entities ├── traffic_light_and_luminaries_200.json # Combined dataset — 200 entities ├── traffic_light_and_luminaries_1300.json # Combined dataset — 1300 entities │ ├── # === QUESTION SETS === ├── questions_pois.json # Evaluation questions for the PoI use case ├── questions_lights_luminaries.json # Evaluation questions for traffic lights & luminaires │ ├── # === GROUND TRUTH === ├── ground_truth/ # Ground truth answer sets per question and dataset size ├── final_results/ # Original question spreadsheets (.xlsx) │ ├── # === RETRIEVAL RESULTS === ├── results/ │ ├── all-MiniLM-L6-v2_384/ # Retrieval results using all-MiniLM-L6-v2 (384-dim) │ └── text-embedding-3-large_1024/ # Retrieval results using text-embedding-3-large (1024-dim) ├── precision_recall_analysis.xlsx # Aggregated precision & recall results │ ├── # === NOTEBOOKS & SCRIPTS === ├── z_00_process_questions_poi.ipynb # Generate PoI ground truth ├── z_00_process_questions_traffic_light_and_luminaires.js # Generate traffic light & luminaire ground truth ├── z_01_vector_rag_qdrant_text-embedding-3-large_1024.ipynb # Index with text-embedding-3-large ├── z_01_vector_rag_qdrant-MiniLM-L6-v2.ipynb # Index with all-MiniLM-L6-v2 ├── z_02_retrieval_qdrant_text-embedding-3-large_1024.ipynb # Semantic retrieval (OpenAI model) ├── z_02_retrieval_qdrant-MiniLM-L6-v2.ipynb # Semantic retrieval (MiniLM model) ├── z_03_calculate_precission_recall.js # Compute precision & recall metrics ├── z_04_graphs.ipynb # Generate result figures │ └── figs/ # Output figures (precision, recall, latency, etc.) Use Cases Use Case 1 — Tourism Assistant (PoIs) A real-time tourist assistant for the city of Madrid, built over 1,088 Points of Interest loaded into a FIWARE Context Broker. Each PoI entity includes: Unique NGSI-LD identifier Title, description, and relevance level (1 = very famous, 5 = not very relevant) Geospatial location (WGS-84, Point geometry) Price (single value or range) Current capacity and occupancy (number of people) A map-based interface allows users to select an area of interest and ask questions in natural language. The system retrieves geospatially filtered PoI entities and feeds them to an LLM to generate contextually rich responses. Use Case 2 — Traffic Light and Luminaire Management A management system for 650 traffic lights and 650 luminaires in Madrid. Each entity includes static metadata (district, neighborhood, description, location) and dynamic attributes (current status, crowd density). The system enables operators to query the city infrastructure in natural language and receive real-time status reports. Datasets All datasets represent NGSI-LD entities from the city of Madrid, serialized as JSON arrays. Three region sizes are provided for each entity type to evaluate system behavior under different load conditions: Dataset Small Medium Large Points of Interest (PoI) 10 100 650 Traffic Lights 10 100 650 Luminaires 10 100 650 Traffic Lights + Luminaires 20 200 1300 Evaluation LLMs Evaluated The following models were tested across all scenarios: Model Type GPT-5.2 (gpt-5.2-2025-12-11) Commercial GPT-4.1 (gpt-4.1-2025-04-14) Commercial GPT-4.1 mini (gpt-4.1-mini-2025-04-14) Commercial Llama 3.1 70B Instruct Open-weight Llama 3.1 8B Instruct Open-weight Qwen 2.5 7B Instruct Open-weight DeepSeek V3 685B Open-weight RAG Configurations Compared Five retrieval configurations were benchmarked: Semantic RAG — Vector similarity search using text-embedding-3-large (1024-dim), top-k = 25 Semantic RAG + Reranking — Same as above with ms-marco-MiniLM-L-6-v2 reranker, top-n = 10 Spatial RAG — Geospatial filtering via FIWARE Context Broker only (all region entities sent to LLM) Spatial RAG + Semantic RAG — Geospatial filtering followed by semantic retrieval Spatial RAG + Semantic RAG + Reranking — Full pipeline with spatial, semantic, and reranking stages Embedding Models Model Dimensions Provider all-MiniLM-L6-v2 384 Sentence Transformers (HuggingFace) text-embedding-3-large 1024 OpenAI Metrics Precision — Fraction of retrieved entities that are relevant Recall — Fraction of relevant entities that were retrieved Response time — End-to-end latency broken down by component (Context Broker, Semantic RAG, Reranking, LLM) Correctness — Classified as OK / MEDIUM / WRONG based on true positive retrieval and hallucinations Results Pre-computed results are included in the repository and can be inspected without re-running any experiment. Retrieval Results Raw per-query retrieval outputs are stored in results/, organized by embedding model: results/ ├── all-MiniLM-L6-v2_384/ # One JSON file per (dataset, size, question) combination └── text-embedding-3-large_1024/ # Same structure for the OpenAI embedding model Each file follows the naming convention {dataset}_{size}_{question}_result.json and contains the list of entity IDs returned by the retrieval stage for that query. Precision & Recall Analysis Aggregated metrics across all models, dataset sizes, and RAG configurations are available in precision_recall_analysis.xlsx. This spreadsheet contains precision, recall, true positives, total retrieved, and total ground truth counts for every evaluated combination. Figures All figures from the paper are available in figs/: File Description models_performance.png Correctness (OK / MEDIUM / WRONG) by model and region size models_response_time.png End-to-end response time per model across region sizes precision_by_region_size_and_scenario.png Precision across the five RAG configurations and three region sizes recall_by_region_size_and_scenario.png Recall across the five RAG configurations and three region sizes total_retrieved_by_region_size_and_scenario.png Number of entities retrieved per configuration and region size time_ms_by_scenario_stacked.png Stacked response time by system component (Context Broker, Semantic RAG, Reranking, LLM) Replicate the results Requirements Docker and Docker Compose Python 3.x with Jupyter Notebook Node.js (for .js scripts) OpenAI API key (for text-embedding-3-large and GPT models) 1. Start the Qdrant Vector Database docker compose up -d Qdrant will be available at http://localhost:6333. The dashboard can be accessed at http://localhost:6333/dashboard. 2. Load Qdrant Snapshots Pre-built collection snapshots are available in dbs_qdrant/. Load them via the Qdrant REST API or dashboard to skip re-indexing. Snapshots are named following the pattern {dataset}_{size}_{embedding_model}.snapshot, for example: poi_650_text-embedding-3-large_1024.snapshot traffic_light_100_all-MiniLM-L6-v2_384.snapshot 3. Generate Ground Truth For traffic lights and luminaires: node z_00_process_questions_traffic_light_and_luminaires.js For PoIs, run the notebook: jupyter notebook z_00_process_questions_poi.ipynb Ground truth files will be written to ground_truth/. 4. Index Entities (Optional — if not using snapshots) Run one of the following notebooks to embed and index the datasets into Qdrant: jupyter notebook z_01_vector_rag_qdrant_text-embedding-3-large_1024.ipynb # or jupyter notebook z_01_vector_rag_qdrant-MiniLM-L6-v2.ipynb 5. Run Retrieval Execute the retrieval notebooks to generate result files in results/: jupyter notebook z_02_retrieval_qdrant_text-embedding-3-large_1024.ipynb 6. Calculate Precision and Recall node z_03_calculate_precission_recall.js Results are saved to precision_recall_analysis.xlsx. 7. Generate Figures jupyter notebook z_04_graphs.ipynb Output figures are saved to figs/. FAIR Data Statement This dataset is designed to comply with the FAIR data principles (Findable, Accessible, Interoperable, Reusable): Findable The dataset is deposited in Zenodo, a publicly accessible research data repository operated by CERN, which assigns a globally unique and persistent Digital Object Identifier (DOI). Rich metadata (title, authors, description, keywords, license, funding) is registered with the Zenodo record, enabling discovery through Zenodo search, DataCite, and major academic search engines. Keywords include: Large Language Models, Retrieval Augmented Generation, Smart City, Urban Foundation Models, FIWARE, NGSI-LD, spatial data, Points of Interest, traffic lights, luminaires. Accessible All datasets, code, results, and figures are freely downloadable from Zenodo without registration. The repository follows an open access policy: no embargo, no access restrictions. Data are stored in widely supported, open formats: JSON (datasets, ground truth, results), XLSX (aggregated metrics), PNG (figures), IPYNB (Jupyter notebooks), and JS (Node.js scripts). The Qdrant vector database is deployed via Docker Compose, ensuring that the full computational environment can be reproduced without proprietary tooling. Interoperable All entity datasets conform to the NGSI-LD standard (ETSI GS CIM 009), the linked-data format adopted by the FIWARE ecosystem and endorsed by ETSI and the European Commission for smart city interoperability. Entity types follow the FIWARE Smart Data Models ontologies (TrafficLightSignal, PoI, Streetlight), which are publicly documented and reusable across urban platforms. Geographic coordinates use the WGS-84 reference system (GeoJSON Point geometry), the international standard for geospatial data exchange. The question and ground truth files use plain JSON, readable by any programming language or data tool. Reusable The dataset is released under a Creative Commons license, clearly stated in the Zenodo metadata. Detailed provenance is provided: entity types, attribute semantics, dataset sizes, and the city (Madrid) are fully documented in this README and in the associated paper. Evaluation questions (questions_pois.json, questions_lights_luminaries.json) and ground truth files (ground_truth/) are included, enabling direct benchmarking of new RAG systems or LLMs against the same experimental setup without re-running the full pipeline. Pre-built Qdrant vector database snapshots (dbs_qdrant/) are included so the retrieval infrastructure can be restored in minutes, lowering the barrier to reuse. The code is modular and documented through numbered notebooks (z_00 → z_04), covering the full pipeline from data preprocessing to figure generation. Citation If you use this dataset or code in your research, please cite: @article{campo2025realtimespatialrag, title = {Real-time Spatial Retrieval Augmented Generation for Urban Environments}, author = {Campo, David Nazareno and Conde, Javier and Alonso, Álvaro and Huecas, Gabriel and Salvachúa, Joaquín and Reviriego, Pedro}, journal = {ACM Transactions on Intelligent Systems and Technology}, year = {2025}, month = {June}, volume = {1}, number = {1}, pages = {1--25}, doi = {XXXXXXX.XXXXXXX}, publisher = {ACM} }