Citation Gating in Legal RAG: data, code and labelling protocol (v1)
收藏资源简介:
Data, code and labelling protocol accompanying the working paper "Citation Gating in Legal RAG: Cost, Consistency and Model Agreement of Proposition-Support Checks for Slovenian Law" (N. Fošnarič, Arieta Labs Ltd, version 1, 29 October 2026). The paper studies a citation gate in a deployed legal research pipeline for Slovenian law: a stage that classifies each (proposition, court headnote) pair as supports, complicates, undermines or irrelevant before a citation is shown to the user. This record contains everything needed to reproduce the paper's tables: the five research questions; every recorded analysis, gate output, ranking, label file and cost ledger from the seven evaluation runs (JSON/JSONL); the scripts that produce the tables; the dated protocol files fixed before each run's labels were computed; the 417-pair attorney labelling package (rubric in Slovenian and English, pair files, second-annotator and full-decision subsets, scoring code, and the model-label key that annotators must not see); and the 50 full-decision labels returned by one lawyer. A README describes the layout, the redactions made (local paths, opaque encrypted reasoning payloads in API tapes, a provider contact address) and the licences. All runs used the Valvasor AI pipeline at commit fab119e (27 October 2026). The pipeline code itself is not included; the prompts, request structures and every recorded input and output are. The retrieval corpus (LexAPI) is served live and has no fixed snapshot; decisions are identified by ECLI and file number and are public on sodnapraksa.si and pisrs.si. Code: MIT. Data, reports and labels: CC BY 4.0.



