遇见数据集

Agent failure-mode benchmarks: raw measurement records

收藏
Zenodo2026-10-30 更新2026-11-05 收录
官方服务:

资源简介:

Raw per-run records behind four small benchmarks on how agent tooling fails: a streamed tool-call argument truncated mid-token that gets repaired into a call the user never wrote; a degenerate repetition loop streamed to the end of the budget; the same tool call executed twice after a retry; and a retrieval system that answers where it should abstain. 721 files, copied byte-for-byte from four public repositories at the commits named in the README. Each record names the client, its version, the scenario and the outcome, so every number in the corresponding write-up can be re-derived instead of trusted. Nothing here is analysis, and nothing here is a ranking: the clients were run at different versions on different days and the scenario counts differ per benchmark.

提供机构:
Zenodo
创建时间:
2026-10-30
二维码
社区交流群
二维码
科研交流群
商业服务