遇见数据集

Recognition Without Binding: RWB Frozen Eval v0.2B Public Package

收藏
Zenodo2026-05-23 更新2026-05-26 收录
官方服务:

资源简介:

This public package releases RWB Frozen Eval v0.2B, benchmark package for Recognition Without Binding (RWB): a provisional-action failure mode in administrative and compliance-oriented LLM decision support. RWB occurs when a model recognizes an incomplete evidence or authorization gate but nevertheless opens a temporary, reversible, conditional, or limited action path before the gate is complete. The package includes a frozen 80-case evaluation set, four prompt conditions, Workbench code for running and auditing model outputs, raw model-run archives, synthetic validation outputs, cross-model summaries, historical development artifacts, and documentation. The primary evaluated models are GPT-4o-mini, Gemini 2.5 Flash, and DeepSeek V4 Pro. The central benchmark result is condition-sensitive: confirmed RWB failures appear in the unscaffolded condition for some models, while explicit evidence/authorization scaffolding reduces confirmed RWB failures in the tested runs. This release should be interpreted as an AI-assisted benchmark and technical report package, not as a human-ground-truth benchmark, not as a real-world prevalence estimate, and not as proof of a universal LLM failure mode. Automated audit labels are treated as screening signals and are separated from synthetic validation/adjudication results.

提供机构:
Zenodo
创建时间:
2026-05-22
二维码
社区交流群
二维码
科研交流群
商业服务