遇见数据集

FailureScope Adversarial Corpus (NeurIPS 2026 E&D)

收藏
Zenodo2026-05-05 更新2026-05-26 收录
官方服务:

资源简介:

FailureScope adversarial corpus: 630 multi-turn agent traces across three attack families (split-action, cognitive-load, authority-impersonation) and three frontier models (Claude Sonnet 4.6, GPT-5.4, DeepSeek-v3.2) against a sandboxed code-execution environment. Records contain full agent trajectories, per-step refusal/compliance labels, LLM-judge attack-success-rate (ASR) labels, and ground-truth network-execution outcomes. Surfaces a 73 to 100 percentage-point gap between LLM-judge ASR and real network execution on cognitive-load attacks, plus per-model behavioral profiles including a Claude Sonnet 4.6 selective-refusal signal (0/30 judge ASR with 21/30 step-2 refusals) on split-action attacks where GPT-5.4 fails at 92%. Includes 1,203 generated adversarial task variants used during attack-template development. This dataset is one of three components of the FailureScope release; see related identifiers on the umbrella record DOI 10.5281/zenodo.20037167.

提供机构:
Zenodo
创建时间:
2026-05-05
二维码
社区交流群
二维码
科研交流群
商业服务