AI45Research/ATBench-Codex

ATBench-Codex是一个面向Codex的代理轨迹安全基准数据集，源自ATBench，作为AI代理安全和安全诊断护栏框架AgentDoG的基准配套。它设计用于可执行编码代理设置中的轨迹级安全评估，重点关注在诸如shell执行、工作区变更、仓库变更、MCP工具调用或长时工具链等操作实际执行之前必须做出安全决策的关键点。与原始ATBench相比，此版本围绕Codex特定的操作语义构建，包括多工具编码工作流、结构化展开事件、仓库和工件操作、MCP服务器供应链表面以及仅在针对实时工作区执行操作时才可见的指令遵循失败。该数据集包含500个样本，每个样本包含用户对话、结构化的Codex执行轨迹、二进制安全判断和细粒度分类标签，适用于对话级安全检测和工具驱动代理轨迹的深入分析。

ATBench-Codex is a Codex-oriented benchmark release derived from ATBench and serves as a benchmark companion to AgentDoG, our diagnostic guardrail framework for AI agent safety and security. It is designed for trajectory-level safety evaluation in executable coding-agent settings, with a focus on the point where safety decisions must be made before actions such as shell execution, workspace mutation, repository changes, MCP tool invocation, or long-horizon tool chaining are actually carried out. Compared with the original ATBench, this release is built around Codex-specific operational semantics, including multi-tool coding workflows, structured rollout events, repository and artifact manipulation, MCP server supply-chain surfaces, and instruction-following failures that only become visible once actions are executed against a live workspace. This 500-example release preserves the current Codex export schema directly, including a normalized conversation, a structured codex_rollout, top-level safety taxonomy fields, and per-example tool metadata, making it suitable for both conversation-level safety detection and deeper analysis of tool-driven agent trajectories.