Class Judge AI v2.0: PlantUML Benchmark, LLM Run Logs, and Code for Evaluating Quantized Open-Weight LLMs as UML Class Diagram Graders

Reproducibility package for the study Evaluating Quantized Open-Weight LLMs for UML Class Diagram Pre-Screening under Controlled Structural and Semantic Perturbations (manuscript under review). Contents dataset_puml_benchmark/: six reference PlantUML class diagrams (S01–S06; 13–41 classes, 158 classes and 1,263 model elements in total) and 84 controlled test variants. Suite A removes 0/25/50/75% of classes with three nested random masks per system. Suite B applies attribute defects (B1), method defects (B2), relationship inversions and deletions (B3), and synonym renaming (B4). Metadata gives the parser-derived retention targets, the removal masks, and the mutation audit log. experiment_results/: raw run logs (per-run summaries and full JSONL audit logs with raw model responses) for Gemma-4-E4B and Qwen3.5-9B on Ollama (Kaggle, 2× Tesla T4) and LM Studio (Mac Mini M4, 16 GB), including a complete re-execution and a three-seed repeatability grid, plus the consolidated CSV used in the analysis. class_judge_app/: Java 21 / Spring Boot source code of Class Judge AI v2.0 (PlantUML parser, deterministic exact and fuzzy baselines, structured and raw prompt modes, LLM clients, headless benchmark API, 34 JUnit tests); build with Maven. scripts/: benchmark generator, headless runner, Kaggle notebook, and analysis scripts that recompute the reported statistics, figures, metamorphic-relation checks, and seed-repeatability results. See README.md in the archive for the folder layout, run settings, and commands. Data: CC BY 4.0. Code: MIT.