遇见数据集

Replication Package for "Silent Failures in Tool-Using LLM Agents Querying Incomplete Enterprise Aggregates"

收藏
Zenodo2026-11-04 更新2026-11-05 收录
官方服务:

资源简介:

This record contains the replication package for the paper “Silent Failures in Tool-Using LLM Agents Querying Incomplete Enterprise Aggregates.” It includes the BRIEF generator of synthetic corporate worlds for a fictitious mining group, the generated test worlds (data warehouse tables, documents and data catalog), gold answers, the run logs of all reported conditions for Claude Sonnet 5.5 and GPT-6.1 Sol (plus Claude Haiku 5.5 and Claude Opus 5.5 in an exploratory study), and the source code for the agents, grading and analysis. All data are synthetic; no real company data or human participants are involved. Contents of the ZIP archive src/: source code (world generator, tool layer, single and router agent architectures, scripted oracle and saboteur controls, deterministic grader, runner and reporting), one package per study tools/: scripts that compute the paper’s tables and figure from the run logs data*/, gold*/: generated test worlds and gold answers with the fact ledgers manifests/, e3/, r/, e4r/, g3/: data manifests and batch records runs/: raw run logs (4,002 agent runs) reports/paper/: tables and figure reported in the paper Algorithm_Description.pdf: description of the algorithms implemented in the code README.txt, LICENSE.txt, CITATION.cff, CHECKSUMS.sha256 Preregistration records are available from the authors on request. Code is licensed under MIT; data and logs under CC BY 4.0. Access is restricted. Please submit an access request stating your name, affiliation and intended use.

提供机构:
Zenodo
创建时间:
2026-11-04
二维码
社区交流群
二维码
科研交流群
商业服务