anon-skillsalign-26/skill_align

SkillsAlign是一个包含245,000个技能的野生数据集，专注于AI代理技能包中的跨上下文错位检测。当技能包的元数据描述与其指令体或随附的资源文件不匹配时，即发生错位。该数据集支持训练和评估检测器，以在代理加载错位技能包之前进行标记。数据集包括多个子集：normalized（完整爬取的语料库，经过规范化处理，包含解析的前端元数据和渲染的M/I/R块，共264,937行）、labeled（人工标注的评估切片，包含552个技能，其中491个对齐和61个错位，提供两种文本变体di和dic）、pairs_di和pairs_dic（合成错位训练对，分别基于M+I和M+I+R变体，每个包含40,000个训练行、2,000个验证行和2,000个测试行，对齐和错位行各占50%）。错位类型包括T2（捐赠跨度交换）、T3a（LLM重写跨度）、T3b（捐赠跨度替换）和T3c（基于规则的标识符替换）。数据集的schema涵盖pair_id、anchor_skill_id、text（带标签的M/I/R块）、label（aligned或misaligned）、corruption_type、split等列。标签通过三信号管道生成：静态规则扫描、LLM审计器集成和沙箱执行（Docker容器监控运行时行为）。数据集推荐用于在合成错位对上训练检测器，并在人工标注集上评估性能，需报告每类精确率、召回率、F1分数和AUC。数据集遵循CC-BY 4.0许可证，但原始包树继承上游源仓库的许可证（如MIT或Apache-2.0）。伦理考虑包括数据来源于公共技能市场列表（2025-09至2026-04）、尊重作者隐私（不重新分发作者元数据）、排除攻击性网络剧本（AUP阻止），以及强调错位通常为无心之失而非恶意软件。

SkillsAlign is a 245K-skill in-the-wild dataset for cross-context misalignment in AI agent skill packages — when a skills metadata description does not match its instruction body or the resource files it ships. The dataset supports training and evaluation of detectors that flag misaligned packages before an agent loads them. It includes multiple subsets: normalized (the full crawled corpus, normalized with parsed front-matter and rendered M/I/R blocks, 264,937 rows), labeled (a human-labelled evaluation slice of 552 skills with 491 aligned and 61 misaligned, available in two text variants di and dic), and pairs_di and pairs_dic (synthetic typed-corruption training pairs based on M+I and M+I+R variants, each with 40,000 train, 2,000 val, and 2,000 test rows, balanced 50/50 aligned vs misaligned). Misalignment types include T2 (donor-span swap), T3a (LLM-rewritten span), T3b (donor-span replace), and T3c (rule-based identifier substitution). The schema includes columns such as pair_id, anchor_skill_id, text (tagged M/I/R block), label (aligned or misaligned), corruption_type, and split. Labels are generated via a three-signal pipeline: static rule scan, LLM auditor ensemble, and sandboxed execution (Docker container with runtime monitoring). The dataset is recommended for training detectors on synthetic corruption pairs and evaluating on the human-labelled set, with reporting of per-class precision, recall, F1, and AUC. It is licensed under CC-BY 4.0, though raw package trees inherit upstream repository licenses (e.g., MIT or Apache-2.0). Ethical considerations cover data sources from public skill-marketplace listings (2025-09 to 2026-04), author handle privacy, exclusion of offensive-cyber playbooks (AUP-blocked), and emphasis that misalignment often reflects honest mistakes rather than malware.