VeriLoop-E2-Evaluation-Evidence

# VeriLoop E2 Evaluation Evidence **Public evaluation evidence for VeriLoop E2 across nine code, agentic, mathematical, and scientific reasoning benchmarks.** This dataset repository is the canonical public evidence layer for the reported benchmark results of **VeriLoop E2**, a post-trained model based on **Qwen 3.8-27B**. It is designed to separate headline benchmark reporting from the underlying auditable artifacts required to inspect, reproduce, and verify those results. The repository contains benchmark-level summaries together with the corresponding public evaluation artifacts, manifests, configuration records, and integrity hashes. Raw evidence is preserved as released; benchmark summaries are derived from those frozen artifacts rather than edited independently. ## Benchmark Results | Benchmark | VeriLoop E2 | |---|---:| | SWE-bench Pro | **76.20%** | | DeepSWE v1.1 | **64.60%** | | SWE-Marathon v1.1 | **45.00%** | | Terminal-Bench 2.1 | **88.80%** | | Terminal-Bench 3.0 | **29.70%** | | Terminal-Bench 4.0 | **37.90%** | | AIME 2026 | **98.30%** | | GPQA Diamond | **93.94%** | | MathArena Apex 2025 | **89.60%** | ## Scope The nine published evaluations cover four complementary capability regimes: - **Software engineering:** SWE-bench Pro, DeepSWE v1.1, and SWE-Marathon v1.1. - **Agentic terminal interaction:** Terminal-Bench 2.1, Terminal-Bench 3.0, and Terminal-Bench 4.0. - **Mathematical reasoning:** AIME 2026 and MathArena Apex 2025. - **Scientific reasoning:** GPQA Diamond. This repository is an **evidence repository**, not a replacement for the original benchmark implementations. Benchmark definitions, task data, graders, and licensing remain governed by their respective upstream projects. ## Repository Structure ```text VeriLoop-E2-Evaluation-Evidence/ ├── README.md ├── MANIFEST.json ├── SHA256SUMS ├── swe-bench-pro/ │ ├── summary.json │ ├── results/ │ ├── logs/ │ ├── config/ │ └── SHA256SUMS ├── deep-swe-v1.1/ │ ├── summary.json │ ├── results/ │ ├── logs/ │ ├── config/ │ └── SHA256SUMS ├── swe-marathon-v1.1/ │ ├── summary.json │ ├── results/ │ ├── logs/ │ ├── config/ │ └── SHA256SUMS ├── terminal-bench-2.1/ │ ├── summary.json │ ├── results/ │ ├── logs/ │ ├── config/ │ └── SHA256SUMS ├── terminal-bench-3.0/ │ ├── summary.json │ ├── results/ │ ├── logs/ │ ├── config/ │ └── SHA256SUMS ├── terminal-bench-4.0/ │ ├── summary.json │ ├── results/ │ ├── logs/ │ ├── config/ │ └── SHA256SUMS ├── aime-2026/ │ ├── summary.json │ ├── results/ │ ├── logs/ │ ├── config/ │ └── SHA256SUMS ├── gpqa-diamond/ │ ├── summary.json │ ├── results/ │ ├── logs/ │ ├── config/ │ └── SHA256SUMS └── matharena-apex-2025/ ├── summary.json ├── results/ ├── logs/ ├── config/ └── SHA256SUMS ``` ## Evidence Model Each benchmark directory is organized around the same provenance contract: 1. **`summary.json`** records the released aggregate result and benchmark identity. 2. **`results/`** contains the public task- or run-level evaluation outputs retained for audit. 3. **`logs/`** contains the corresponding public execution or grading records where release is permitted. 4. **`config/`** records the evaluation configuration required to interpret the run. 5. **`SHA256SUMS`** binds the released files to immutable content hashes. The repository-level `MANIFEST.json` binds the benchmark set, model identity, release version, and evidence locations into one machine-readable index. ## Integrity and Reproducibility The publication policy for this repository is intentionally strict: - Aggregate scores must be derivable from frozen evaluation artifacts. - Released raw artifacts are not rewritten to match a headline score. - Benchmark identity and version are recorded explicitly. - Public files are bound by cryptographic hashes. - Any redaction required for licensing, privacy, or infrastructure security is performed without altering the reported evaluation outcome. - Corrections are versioned rather than silently replacing previously published evidence. - The evidence repository and the model repository remain logically separate so that model weights, model-card reporting, and benchmark evidence can evolve without conflating their provenance. ## Hugging Face Eval Results The VeriLoop E2 model repository uses Hugging Face's structured evaluation-results format under: ```text .eval_results/ ``` Each supported benchmark entry points back to the corresponding evidence in this dataset repository. This provides two complementary publication layers: **Model repository → structured score registration and Hub presentation** **Evaluation evidence dataset → auditable benchmark artifacts and provenance** ## Model **Model:** VeriLoop E2 **Base model:** Qwen 3.8-27B **Release type:** Post-training **Evaluation evidence:** This repository **Model repository:** `xxxx` **Technical report:** `xxxx` ## Citation If you use VeriLoop E2 or the evaluation evidence released here, please cite the accompanying technical report and this dataset release. ```bibtex @misc{wang2026veriloop_e2, title = {VeriLoop E2}, author = {Libo Wang}, year = {2026}, howpublished = {Hugging Face}, note = {Model and public evaluation evidence} } ``` ## License and Upstream Benchmarks This repository distributes evaluation evidence produced for VeriLoop E2. It does not relicense upstream benchmark datasets, tasks, graders, or third-party artifacts. Files derived from upstream benchmarks remain subject to their original licenses and terms. Model-specific evidence and repository metadata are released under the terms stated in this repository. --- **VeriLoop E2 — public results backed by inspectable evidence.**