SkillAudit: Skill-Centered Evaluation Traces for LLM Coding Agents
收藏Zenodo2026-06-12 更新2026-05-26 收录
下载链接:
https://zenodo.org/doi/10.5281/zenodo.20253169
下载链接
链接失效反馈官方服务:
资源简介:
Skill-centered evaluation traces for LLM coding agents. SkillAudit evaluates 226 real-world agent skill packages (Anthropic-style /skill markdown + supporting files) across 6 agent-model configurations for utility and 4 for security. Each skill isexercised in matched with-skill and without-skill conditions inside an isolated Harbor Docker sandbox, producing per-trial trajectories plus per-skill aggregate reports of pass_rate_gain, time_saving, token_saving and safety_score.
This dataset accompanies the NeurIPS 2026 Datasets and Benchmarks Track submission "SkillAudit: From Task-First Evaluation to Skill-Centered Assessment".
DOUBLE-BLIND DRAFT: author identity will be filled at camera-ready. Top-level license: CDLA-Permissive-2.0; per-skill licenses pending the legal review. All credentials and identifying paths have been scrubbed by sanitize_traces_v2.py v2.7; an independentpost-scrub scan (detect-secrets v1.5.0 + pure-regex JWT verifier) across 983,059 published text files reports 0 JWT and 0 real credentials. See INDEPENDENT_SCAN_NOTES.md.
Croissant 1.0 + RAI/1.0 metadata is included as croissant.json; mlcroissant.Dataset(jsonld='croissant.json') validates with no errors and no warnings.
提供机构:
Zenodo创建时间:
2026-05-17



