AI45Research/ATBench

ATBench是一个用于评估代理在现实、长期交互中安全性的轨迹级别基准。数据集包含500个标注的执行轨迹（250个安全/250个不安全），具有多轮交互（平均8.97轮）和1,575个独特工具。基准提供了基于分类的细粒度安全标注，能够进行精确的风险归因和诊断，而不仅仅是二进制的安全/不安全标签。每个样本对应一个完整的代理执行轨迹，并标注了轨迹级别的二进制安全标签（0表示安全，1表示不安全）。此外，数据集还采用了统一的三维安全分类法，从风险来源、失败模式和现实世界危害三个维度对风险进行组织。

ATBench is a trajectory-level benchmark for evaluating agentic safety in realistic, long-horizon interactions. It contains 500 annotated execution trajectories (250 safe / 250 unsafe) with multi-turn interactions (avg. 8.97 turns) and 1,575 unique tools. The benchmark provides taxonomy-grounded, fine-grained safety annotations, enabling precise risk attribution and diagnosis beyond binary safe/unsafe labels. Each sample corresponds to one complete agent execution trajectory and is annotated with a trajectory-level binary safety label (0 for safe, 1 for unsafe). Additionally, the dataset adopts a unified, three-dimensional safety taxonomy that organizes risks along the axes of risk source, failure mode, and real-world harm.