Study records for "Recovering from Tool Errors at the Inference Engine: Pre-Registered Studies of Error Delivery for LLM Agents"
收藏资源简介:
Records of the three studies in the manuscript "Recovering from Tool Errors at the Inference Engine: Pre-Registered Studies of Error Delivery for LLM Agents" (2026): plans and registrations with the SHA-256 lists that fixed them, stimuli, scripts, raw outputs and analyses. Study 1 (six open models, plus two 32B models) compares where and how a tool error is delivered to the model. Study 2 (seven small and six large models) tests an append-only guidance turn; its small tier was registered on OSF (https://osf.io/7gujd). Study 3 evaluates the engine's tool-error layer in real use against llama.cpp, vLLM and SGLang on the Loud/Quiet and ToolMaze benchmarks and with the Qwen Code agent. The layer is part of the open-source engine GeulPU (https://doi.org/10.5281/zenodo.23548056). ToolMaze content is not redistributed because the benchmark has no license: its conversations were removed and its evaluation files reduced to task, mode and pass. README.md explains the files, what is not included, the redactions and how to check the hashes.



