遇见数据集

Axiom 2026 evaluation dataset

收藏
Zenodo2026-09-25 更新2026-10-01 收录
官方服务:

资源简介:

Evaluation records and reproduction scripts for T. Chaudhry, "From Context to Code: Codifying Cross-System Reasoning for LLM-Assisted Operations," IEEE Access, 2026. Contains the 270-cell dataset (9 models × 3 passes × 5 queries × 2 conditions) with primary-judge verdicts, correctness rubrics, second-rater (DeepSeek) verdicts for every cell, per-run harness configuration, per-cell tool-usage flags, the baseline-condition skill documents, and scripts that regenerate every quantitative result in the paper. v1.1.0 adds the second-rater verdicts and agreement script (paper §VI-B), harness configuration, tool-usage flags, and baseline skills; no cell was re-run and no primary verdict changed. Raw agent traces are withheld because they contain customer identifiers from a production system.

提供机构:
Zenodo
创建时间:
2026-09-25
二维码
社区交流群
二维码
科研交流群
商业服务