Source Diversity Audit: what the citations displayed by four generative search products reach, traced on six Virginia data-center policy questions (September 2026)
收藏资源简介:
Capture, tracing and coding data for an audit of what the citations displayed by four generative search products actually reach. Six questions about Virginia data-center policy were put to ChatGPT, Gemini, Perplexity and Grok, twice each, each in the mode the product labels Temporary chat, Incognito or Private, on the evening of 12 September and the morning of 13 September 2026. ChatGPT and Gemini ran on paid tiers, Perplexity and Grok on free tiers. The authoritative record for each question, its date and its denominator were entered before any prompt was issued. Across the 48 answers the products displayed 185 citations. Those citations reached 136 distinct pages, of which 13 were the record itself and 90 could not be followed from the interface, 77 of them because the product displayed a title with no link. No cited page documented its own verification against a record. Twenty-nine of the 48 answers reached no primary record, 16 reached one, and 3 reached more than one. The Terminals sheet registers 144 pages, eight of which are pre-registered records or pilot pages that no captured answer cited; the README defines both counting bases and says which to use for which statement. Contents. The capture and tracing workbook (11 sheets: pre-registration, controlled vocabulary with post-registration additions marked, terminals, citations, captures, summary, sensitivity, adjudication, page evidence, blind second coder). A validator script that enforces the vocabulary and the internal joins. Structured capture records for all 48 answers, recovered full answer text for Grok, the page evidence pass, the blind second coder's codes, and copies of the pre-registered records. Experiment B: a pre-registration frozen 18 September 2026 before its first capture, testing whether an instruction to name the record changes what the same products cite, with its capture workbook. Method note. Tracing, page retrieval and first coding were performed by an AI assistant under the author's direction. A second AI instance coded the 58 unique cited URLs from the definitions and the page evidence alone and agreed on 42 of the 52 it could code. The author ruled on the contested codings, which are recorded verbatim with both codings and a sensitivity table for every headline count. Scope. Six questions, one policy domain, four products, two runs, one 48-hour window. Engine and tier are confounded and no product ranking is claimed. Version 1.1. README restated: counting bases defined, product tiers corrected to those recorded on the Captures sheet, dataset DOI filled. Experiment B pre-registration and capture workbook added. No row of captured, traced or coded data changed from version 1.0.



