VeriGrade replication package: synthetic benchmark, generator, and direct-LLM baselines for compiled document grading
收藏资源简介:
Version 1.0.3. Every headline figure in the paper is recomputable from this package: the compiled arm's per-submission totals for both exams (the MAE = 0 result) and for the seed-2027 fresh-seed replication, the feature-attribution ablation outputs behind the 41%/44% template-inherited figures, per-instance near-miss verdicts (1,669 for the compiled arm; 256 paired instances for the two direct arms) with the scripts that recompute the reported tests, and per-submission records for all four direct-LLM repeated-run arms. Non-ASCII filenames carry the UTF-8 flag, so the archive extracts correctly on Windows. See CHANGELOG.md. Replication package for the VeriGrade paper. Contains two synthetic benchmarks of mutated office-document submissions (Word / Excel / PowerPoint / filesystem, 120 submissions each, seed 2026) with constructive ground truth, the mutant generator and per-exam recipes, the direct-LLM baseline runner and prompts for every reported arm, the near-miss kill-rate and held-out-operator analyses, the model-provenance probe, the agreement-statistics script, and the sanitizer applied to the package. All submissions are synthetic, produced by programmatic mutation of instructor-authored worked samples. No real student submission and no verbatim exam text is included. Ground truth is constructive: the generator declares which sub-checks each mutation breaks, so agreement with it measures a grader rather than defining the answer. Sanitization was verified to be behaviour-preserving: re-running the platform sub-check code over the sanitized submissions agrees with constructive truth on 4440/4440 cells (Exam 1) and 1800/1800 cells (Exam 2).



