CMC Evaluation/Analysis Harness v1.0: RAG + QLoRA Fine-Tuning Pipeline for Validating Language Models in Regulated CMC Work

The evaluation/analysis harness used to produce the CMC-QA v1.0 benchmark evaluation records (https://doi.org/10.5281/zenodo.21177973). Implements: corpus chunking/ embedding and a RAG retrieval index; RAFT training-data synthesis and QLoRA fine-tuning; a four-arm evaluation runner (base / base+RAG / fine-tuned / fine-tuned+RAG) producing per- question JSONL records and a run manifest; a pre-registered paired-statistics module (Wilcoxon/McNemar, effect sizes, bootstrap CIs, Holm/BH correction); a blind SME/judge- agreement export and scorer; and figure-generation scripts. Companion to: Gilbert, P.-A. "A Scientific Methodology for Adapting and Validating Local Open-Source Language Models in Regulated CMC Environments" [DOI/arXiv once available], and Gilbert, P.-A. "Benchmarking Small Open Language Models for Regulatory and CMC Question Answering" [DOI/arXiv once available]. Data produced by this harness (benchmark, evaluation records, statistics, derived metrics) is deposited separately under CC-BY-4.0: https://doi.org/10.5281/zenodo.21177973. Live development: https://github.com/pag992007/LOCAL_LLM_CMC — this record is a frozen snapshot at commit `f95eabb`.