VeriGrade replication package: synthetic benchmark, generator, and direct-LLM baselines for compiled document grading
收藏资源简介:
Replication package for the VeriGrade paper. Contains two synthetic benchmarks of mutated office-document submissions (Word / Excel / PowerPoint / filesystem, 120 submissions each, seed 2026) with constructive ground truth, the mutant generator and per-exam recipes, the direct-LLM baseline runner and prompts for every reported arm, the near-miss kill-rate and held-out-operator analyses, the model-provenance probe, the agreement-statistics script, and the sanitizer applied to the package. All submissions are synthetic, produced by programmatic mutation of instructor-authored worked samples. No real student submission and no verbatim exam text is included. Ground truth is constructive: the generator declares which sub-checks each mutation breaks, so agreement with it measures a grader rather than defining the answer. Sanitization was verified to be behaviour-preserving: re-running the platform sub-check code over the sanitized submissions agrees with constructive truth on 4440/4440 cells (Exam 1) and 1800/1800 cells (Exam 2). Data are licensed CC BY 4.0; the code under code/ and release_sanitize.py are licensed MIT (see LICENSE-CODE in the package).



