agents-last-exam-reference

# Agents Last Exam — Reference (Ground-Truth) Data ⚠️ **Gated dataset.** This repo contains the ground-truth / reference outputs used to score the Agents Last Exam (ALE) benchmark. Access requires login, agreement to the terms on the access-request form, and **manual approval**. ## The Agents Last Exam dataset family ALE is published as three companion HuggingFace datasets: | Dataset | Contents | Access | |---|---|---| | [Task Card Metadata](https://huggingface.co/datasets/agents-last-exam/agents-last-exam) | One row per task: titles, prompts, taxonomy, input-file descriptors | Open | | [Task Input Data](https://huggingface.co/datasets/agents-last-exam/agents-last-exam-data) | The `input/` files each task hands the agent at run start | Open | | [Reference (Ground-Truth) Data](https://huggingface.co/datasets/agents-last-exam/agents-last-exam-reference) | The `reference/` outputs used to score runs | ⚠️ Gated (manual approval) | **You are viewing: Reference (Ground-Truth) Data.** - **Source repository:** https://github.com/rdi-berkeley/agents-last-exam - **Official site:** [https://agents-last-exam.org](https://agents-last-exam.org) ## Terms of Use (summary) - **No training.** Do not use this data to train, fine-tune, or distill models. - **No leakage at eval time.** Do not expose this data to any model/agent while it is being evaluated on ALE. - **No redistribution.** Do not repost or share access. - Access is **logged** (HF username, email, timestamp, submitted fields) and may be **revoked** on misuse. The full terms are presented on the access-request form. ## Layout ``` tasks/ <task_id>/ # e.g. business_finance/taxform_4_1 <variant>/ # variant name as defined in the source repo reference/ # ground-truth outputs for scoring ... ``` ## What's Included / Excluded - **Included:** the `reference/` subdirectory of each task variant. - **Excluded:** `input/` (open data repo in the family table above), `software/`, VM images, and internal artifacts.