anonymous-md/ProseOnlyRepair_Codes

ProseOnlyRepair 是一个用于评估分析脚本的数据集，旨在重现《Repair-First》论文（双盲评审中）中的表格和图形。该数据集包含多个Python脚本，用于生成头条表格（如每个任务的准确率、Paloma BPB、修复增量）、8变体和12变体的修复效果分层分析（包括POST-only变体），以及从评估JSON文件重新生成论文图1和图2。脚本基于lm-evaluation-harness 0.4.12版本，使用vLLM后端，评估28个任务的散文套件和11个语料库的Paloma BPB面板，解码采用贪婪策略，最大长度一般为256（CoQA/SQuADv2为512）。数据集支持CPU-only运行，具有确定性输出，并附带评估结果JSON文件（12个变体 × 散文和paloma套件），这些文件在论文的补充材料中发布。数据集匿名发布，用于双盲评审，评审后将重新发布包含作者和机构元数据的版本。ProseOnlyRepair is a dataset of evaluation/analysis scripts designed to reproduce the tables and figures in the *Repair-First* paper (under double-blind review). It includes multiple Python scripts for generating headline tables (e.g., per-task accuracy, Paloma BPB, repair deltas), 8-variant and 12-variant repair-effect-by-tier analyses (including POST-only variants), and recreating Figures 1 and 2 from evaluation JSON files. The scripts are based on lm-evaluation-harness version 0.4.12 with a vLLM backend, evaluating a 28-task prose suite and an 11-corpus Paloma BPB panel, using greedy decoding with a maximum length of 256 (except 512 for CoQA/SQuADv2). The dataset supports CPU-only execution, provides deterministic outputs, and is accompanied by evaluation result JSON files (12 variants × prose and paloma suites) released in the papers supplementary materials. It is anonymized for double-blind review and will be re-released with author and affiliation metadata after the review period.