Agent failure-mode benchmarks: raw measurement records
收藏资源简介:
Raw per-run records behind four small benchmarks on how agent tooling fails: a streamed tool-call argument truncated mid-token that gets repaired into a call the user never wrote; a degenerate repetition loop streamed to the end of the budget; the same tool call executed twice after a retry; and a retrieval system that answers where it should abstain. 721 files, copied byte-for-byte from four public repositories at the commits named in the README. Each record names the client, its version, the scenario and the outcome, so every number in the corresponding write-up can be re-derived instead of trusted. Nothing here is analysis, and nothing here is a ranking: the clients were run at different versions on different days and the scenario counts differ per benchmark.



