遇见数据集

ErBioBench v1.0: benchmark data, prompts, model outputs, scoring rules and analysis code for deterministic grounding in occupational biological-risk assessment

收藏
Zenodo2026-10-29 更新2026-11-05 收录
官方服务:

资源简介:

ErBioBench is a controlled multi-model benchmark evaluating the reliability of generative artificial intelligence for legitimate occupational biological-risk assessment. The frozen REP5 experiment crossed 15 synthetic benchmark cases, 12 model slots from four provider families, three experimental conditions and five repeated API executions, yielding 2,700 nominal runs. The experimental conditions were: C1, LLM-only assessment; C2, assessment with structured ErBio procedural guidance; and C3, assessment additionally supplied with deterministic output generated by ErBioR. All conditions used the same frozen response schema. The archive contains the benchmark protocol, cases, prompts, response schema, scoring and safety rules, model manifest, raw model responses, canonical outcomes, automated scoring datasets, secondary safety and traceability outcomes, reproducibility outputs, final audit materials, analysis scripts and figure-generation scripts. The final canonical reconstruction contains 2,263 scientifically valid completed responses, 290 refusals and 147 invalid or non-valid technical responses. ErBioBench evaluates legitimate biosafety and occupational-health decision support. It does not evaluate malicious biological capability. The C3 condition should not be interpreted as a test of unaided LLM reasoning, because the deterministic ErBioR computation is supplied to the model; C3 evaluates the fidelity of a hybrid architecture separating deterministic scientific computation from generative interpretation and communication.

提供机构:
Zenodo
创建时间:
2026-10-29
二维码
社区交流群
二维码
科研交流群
商业服务