Supplementary materials for "Can urban planners rely on Large Language Models? Prompt-framing experiments with frontier models on two contested masterplans"

Supplementary materials Aristotelis Vartholomaios 1 and Dionysia Georgia Ch. Perperidou 2 1. Department of Planning and Regional Development, University of Thessaly, Pedion Areos, 38334 Volos, Greece; [e-mail]2. Department of Surveying and Geoinformatics Engineering, University of West Attica, Egaleo Park, Ag. Spyridonos Str., 12243 Athens, Greece, [e-mail] This repository accompanies the study "Can urban planners rely on Large Language Models? Prompt-framing experiments with frontier models on two contested masterplans". The stury performed a structured prompt–response experiment iwith four frontier LLMs (Spring 2026: ChatGPT 5.1, Claude Opus 4.6, Gemini 3.0 Pro, Grok 4) were asked to appraise two contested Greek urban regeneration projects (Hellinikon, TIF-HELEXPO) under four prompt framings, yielding a 32-run matrix (2 cases x 4 prompts x 4 models). The repository contains every artefact produced in the study: the prompts as administered, the factsheets given to the models, the verbatim model responses, the validation rubrics and analysis outputs. *methodologically, the UI of the LLMs does not allow specifying a temperature of zero for full deterministic results. Replications of the experiment will likely yield similar but NOT identical responses. Repository layout prompts/ Prompt templates as administered (P1–P4 × HEL, TIF) factsheets/ Factsheets given to the models (one per case) runs/ Verbatim model responses, one Markdown file per run (32 files total: 2 cases × 4 models × 4 prompts) coding/ Validation rubrics, coding protocol, fabrication audit, and the raw position-level codings (raw_codings/) references/ Source catalogues backing both the factsheets and the validation rubrics (one file per case) data/ Derived data tables (CSV) behind the paper's figures: descriptive metrics (A1), similarity matrices (A2), lexical-distinctiveness tables (A3), image-reference counts findings/ Narrative results documents describing the analysis Run naming convention {CASE}_{MODEL}_{PROMPT}.md, where: CASE ∈ {HEL (Hellinikon), TIF (TIF-HELEXPO)} MODEL ∈ {CLA (Claude Opus 4.6), GEM (Gemini 3.0 Pro), GPT (ChatGPT 5.1), GRK (Grok 4)} PROMPT ∈ {P1 minimal-impartial, P2 comprehensive-impartial, P3 comprehensive-skeptical, P4 comprehensive-advocating} Factsheet construction The factsheets in factsheets/ are the texts that were actually shown to the models. They were derived from internal full-length case files by removing all named individuals, evaluative language, arguments for and against the project, direct quotes and characterisations. References were also stripped, since several source titles would themselves signal contestation (for example, formal opposition statements from professional bodies) and would defeat the purpose of the impartial baseline prompts P1 and P2. The full un-edited case files are not redistributed in this repository. Validation rubric coding/validation_rubric_HEL.md and coding/validation_rubric_TIF.md enumerate the established positions used to score each model run. Positions were compiled from professional-body statements (Technical Chamber of Greece, Hellenic Society for Urbanism and Regional Planning, Aristotle University Architecture Department), Council of State filings, peer-reviewed scholarship, civic-movement documentation and official proponent communications. The rubric was finalised before any model output was read; the models never saw it. Hellinikon: 18 critical + 15 supportive positions. TIF-HELEXPO: 19 critical + 18 supportive. Combined: 70 positions. Provenance of factual claims and rubric positions Although the factsheets and the validation rubrics carry no inline citations (intentionally, in the case of the factsheets), every factual statement and every rubric position derives from the documentary record. The references/ folder contains a per-case source catalogue (32 numbered entries per case) listing legal and regulatory documents, news coverage, professional-body statements, academic publications, civic-movement documentation and proponent communications, with URLs, dates and short descriptions. Readers wishing to verify a specific factsheet claim or rubric position can trace it to its documentary basis through these catalogues. Licensing Compiled materials and annotations (factsheets, prompts, rubrics, coding protocol and audit, derived data tables): CC-BY 4.0. Raw model responses (runs/): generated by the named models under their respective Terms of Service as of the run date. The compilation, naming and annotation of these responses are CC-BY 4.0; the underlying generated text is provided for non-commercial research replication. Citation If you use these materials, please cite the accompanying article (currently under review).