Homelander456/SkillMismatch-Bench
收藏Hugging Face2026-04-23 更新2026-04-26 收录
下载链接：
https://hf-mirror.com/datasets/Homelander456/SkillMismatch-Bench
下载链接
链接失效反馈官方服务：
资源简介：
SkillMismatch-Bench是一个用于诊断技能增强编码代理失败模式的基准测试。首次公开发布包含最小基准核心：17个稳定的仓库级错误修复任务、54个技能卡片和4个真实开源Python仓库（psf/requests、pallets/flask、pytest-dev/pytest、sphinx-doc/sphinx），以及2个基准测试套件（first8和phaseb）。数据集设计用于控制技能暴露条件下的评估，适用于具有仓库访问权限的编码代理，而非仅静态提示聊天模型。
SkillMismatch-Bench is a benchmark for diagnosing failure modes in skill-augmented coding agents. This first public release contains the minimal benchmark core: 17 stable repository-level bug-fix tasks, 54 skill cards, and 4 real open-source Python repositories (psf/requests, pallets/flask, pytest-dev/pytest, sphinx-doc/sphinx), along with 2 benchmark suites (first8 and phaseb). The benchmark is designed for controlled skill exposure and is intended for coding agents with repository access, not just static prompt-only chat models.
提供机构：
Homelander456
