DRFT: A Corpus of Draft–Review–Final Triples for Modeling and Evaluating the Peer-Review Cycle

DRFT: Draft–Review–Final Triples Dataset 📌 Overview DRFT (Draft–Review–Final Triples) is a curated dataset of 225 triplets that capture the peer-review cycle in scientific publishing.Each instance contains: The initial draft (submission) The peer reviews (concatenated comments and/or meta-review) The final published version (arXiv) The dataset is distributed as a single .csv file with standardized schema and evaluation signals. 📊 Key Features 225 aligned draft–review–final triplets (ICLR + PeerRead). Two automatic metrics included: BERTScore (draft ↔ final) – semantic preservation G-Eval (review → final, GPT-5o-mini) – review uptake with justification Compact, high-value benchmark for: Review-conditioned edit planning Evaluating automatic revisions vs. human finals Analyzing review facets and revision magnitude 📝 Schema (Fields) Each row in the .csv dataset contains: base_text → Full text of the initial submission review → Concatenated peer-review comments/meta-review final_text → Final version (arXiv) source → Data source (ICLR/OpenReview, PeerRead) bertscore → Semantic similarity score (draft ↔ final) g_eval_score → Review uptake score (1–10) g_eval_justification → Short justification generated by GPT-5o-mini 📈 Corpus Statistics Base text length (tokens): mean ≈ 4700 Final text length (tokens): mean ≈ 4600 BERTScore: mean ≈ 0.82 G-Eval: mean ≈ 8.31 Correlation: r ≈ –0.33 (complementarity between metrics) 🔧 Usage Typical research use cases include: Review-conditioned edit planningGenerate structured edit plans based on review feedback. Evaluating automatic revisionsBenchmark against human final versions using BERTScore + G-Eval. Analyzing review facetsStudy how specific feedback drives textual changes. 📜 How to Load the Dataset We provide a helper script drft_loader.py to make it easy to load the dataset into a pandas DataFrame. Example: from scripts.drft_loader import load_drft_dataset # Load the dataset df = load_drft_dataset("DRFT_dataset.csv") # Explore print(df.shape) print(df.head())