Free LLMs under-state ERG evacuation distances in hazmat prose
收藏资源简介:
Finding: Free large language models systematically under-state ERG (Emergency Response Guidebook) evacuation distances in hazmat prose. On N=90 (6 models × 15 TIH substances, 'Public Safety & Evacuation' prose, IUPAC representation): of cells stating a distance, 58% (43/74) fall below the ERG night protective-action requirement and 30% (22/74) below the day requirement; class-aware ERG fidelity is 7% (5/74), at chance level (14% baseline). The direction is robust across all six models. Method: pre-registered (hypothesis, metric, tolerance, and a hard extractor instrument-gate locked before running). Ground truth = ERG 2024 (US DOT/PHMSA, public), text-layer verified. Model outputs reused from ChEmREF Task II (arXiv:2511.10027, MIT). Includes an ALCOA+ correction where adversarial verification reversed the initial conclusion. Contents: finding write-up, pre-registered method, ERG ground-truth appendix, and raw + corrected per-cell classification data (JSON). Rights holder: EcoDeco AS.



