遇见数据集

Study records for "Recovering from Tool Errors at the Inference Engine: Pre-Registered Studies of Error Delivery for LLM Agents"

收藏
Zenodo2026-10-29 更新2026-11-05 收录
官方服务:

资源简介:

Records of the three studies in the manuscript "Recovering from Tool Errors at the Inference Engine: Pre-Registered Studies of Error Delivery for LLM Agents" (2026): plans and registrations with the SHA-256 lists that fixed them, stimuli, scripts, raw outputs and analyses. Study 1 (six open models, plus two 32B models) compares where and how a tool error is delivered to the model. Study 2 (seven small and six large models) tests an append-only guidance turn; its small tier was registered on OSF (https://osf.io/7gujd). Study 3 evaluates the engine's tool-error layer in real use against llama.cpp, vLLM and SGLang on the Loud/Quiet and ToolMaze benchmarks and with the Qwen Code agent. The layer is part of the open-source engine GeulPU (https://doi.org/10.5281/zenodo.23548056). ToolMaze content is not redistributed because the benchmark has no license: its conversations were removed and its evaluation files reduced to task, mode and pass. README.md explains the files, what is not included, the redactions and how to check the hashes.

提供机构:
Zenodo
创建时间:
2026-10-29
二维码
社区交流群
二维码
科研交流群
商业服务