boss001/Dolci-DPO-Model-Response-Pool

---
license: odc-by
language:
- en
tags:
- llm-responses
- model-comparison
- synthetic
pretty_name: Dolci DPO Model Response Pool
size_categories:
- 100B<n<1T
configs:
- config_name: gemma3-4b
data_files: gemma3-4b/data.parquet
- config_name: gemma3-12b
data_files: gemma3-12b/data.parquet
- config_name: gemma3-27b
data_files: gemma3-27b/data.parquet
- config_name: gpt-20b
data_files: gpt-20b/data.parquet
- config_name: gpt-120b
data_files: gpt-120b/data.parquet
- config_name: gpt-4.1-2025-04-14
data_files: gpt-4.1-2025-04-14/data.parquet
- config_name: mistral-24b
data_files: mistral-24b/data.parquet
- config_name: olmo2-1b
data_files: olmo2-1b/data.parquet
- config_name: olmo2-7b
data_files: olmo2-7b/data.parquet
- config_name: olmo2-13b
data_files: olmo2-13b/data.parquet
- config_name: olmo2-32b
data_files: olmo2-32b/data.parquet
- config_name: phi4-mini-instruct
data_files: phi4-mini-instruct/data.parquet
- config_name: qwen3-coder-no_reasoning-30b-3a
data_files: qwen3-coder-no_reasoning-30b-3a/data.parquet
- config_name: qwen3-no_reasoning-0.6b
data_files: qwen3-no_reasoning-0.6b/data.parquet
- config_name: qwen3-no_reasoning-1.7b
data_files: qwen3-no_reasoning-1.7b/data.parquet
- config_name: qwen3-no_reasoning-4b
data_files: qwen3-no_reasoning-4b/data.parquet
- config_name: qwen3-no_reasoning-8b
data_files: qwen3-no_reasoning-8b/data.parquet
- config_name: qwen3-no_reasoning-14b
data_files: qwen3-no_reasoning-14b/data.parquet
- config_name: qwen3-no_reasoning-30b-3a
data_files: qwen3-no_reasoning-30b-3a/data.parquet
- config_name: qwen3-no_reasoning-32b
data_files: qwen3-no_reasoning-32b/data.parquet
- config_name: qwen3-reasoning-1.7b
data_files: qwen3-reasoning-1.7b/data.parquet
- config_name: qwen3-reasoning-4b
data_files: qwen3-reasoning-4b/data.parquet
- config_name: qwen3-reasoning-8b
data_files: qwen3-reasoning-8b/data.parquet
- config_name: qwen3-reasoning-14b
data_files: qwen3-reasoning-14b/data.parquet
- config_name: qwen3-reasoning-30b-3a
data_files: qwen3-reasoning-30b-3a/data.parquet
- config_name: qwen3-reasoning-32b
data_files: qwen3-reasoning-32b/data.parquet
- config_name: qwq-32b
data_files: qwq-32b/data.parquet
- config_name: yi-9b
data_files: yi-9b/data.parquet
- config_name: yi-34b
data_files: yi-34b/data.parquet
default_config: gemma3-4b
---

# Dolci DPO Model Response Pool

This dataset contains up to 2.5 million responses for each model in the Olmo 3 DPO model pool, totalling about 71 million prompt, response pairs. Prompts are sourced from [allenai/Dolci-Instruct-SFT](https://huggingface.co/datasets/allenai/Dolci-Instruct-SFT), with additional data from [allenai/WildChat](https://huggingface.co/datasets/allenai/WildChat).

## Dataset Structure

### Configurations

Each model has its own configuration. Load a specific model's responses with:

```python
from datasets import load_dataset

# Load a single model's responses
ds = load_dataset("your-org/your-dataset-name", "gemma3-12b")

# Load multiple models
models = ["gemma3-12b", "qwen3-reasoning-32b", "yi-34b"]
datasets = {m: load_dataset("your-org/your-dataset-name", m) for m in models}
```

### Available Models

| Model Family | Configurations |
|--------------|----------------|
| Gemma 3 | `gemma3-4b`, `gemma3-12b`, `gemma3-27b` |
| GPT | `gpt-20b`, `gpt-120b`, `gpt-4.1-2025-04-14` |
| Mistral | `mistral-24b` |
| OLMo 2 | `olmo2-1b`, `olmo2-7b`, `olmo2-13b`, `olmo2-32b` |
| Phi 4 | `phi4-mini-instruct` |
| Qwen 3 (no reasoning) | `qwen3-no_reasoning-0.6b`, `qwen3-no_reasoning-1.7b`, `qwen3-no_reasoning-4b`, `qwen3-no_reasoning-8b`, `qwen3-no_reasoning-14b`, `qwen3-no_reasoning-30b-3a`, `qwen3-no_reasoning-32b` |
| Qwen 3 (reasoning) | `qwen3-reasoning-1.7b`, `qwen3-reasoning-4b`, `qwen3-reasoning-8b`, `qwen3-reasoning-14b`, `qwen3-reasoning-30b-3a`, `qwen3-reasoning-32b` |
| Qwen 3 Coder | `qwen3-coder-no_reasoning-30b-3a` |
| QwQ | `qwq-32b` |
| Yi 1.5 | `yi-9b`, `yi-34b` |

### Schema

Each configuration contains the following fields:

| Field | Type | Description |
|-------|------|-------------|
| `custom_id` | `string` | Unique identifier for the prompt (shared across models) |
| `dataset` | `string` | Source dataset the prompt came from |
| `prompt` | `string` | The input prompt |
| `response` | `string` | The model's response |
| `model` | `string` | Model name |

### Cross-Model Comparison

To compare responses across models for the same prompts:

```python
from datasets import load_dataset

# Load two models
gemma = load_dataset("your-org/your-dataset-name", "gemma3-12b")
qwen = load_dataset("your-org/your-dataset-name", "qwen3-reasoning-32b")

# Create lookup by custom_id
gemma_by_id = {row['custom_id']: row for row in gemma['train']}
qwen_by_id = {row['custom_id']: row for row in qwen['train']}

# Find shared prompts
shared_ids = set(gemma_by_id.keys()) & set(qwen_by_id.keys())
print(f"Shared prompts: {len(shared_ids)}")

# Compare a specific prompt
example_id = list(shared_ids)[0]
print(f"Prompt: {gemma_by_id[example_id]['prompt'][:200]}...")
print(f"Gemma response: {gemma_by_id[example_id]['response'][:200]}...")
print(f"Qwen response: {qwen_by_id[example_id]['response'][:200]}...")
```

## Coverage Statistics

Not every model has a response for every prompt. The table below shows approximate coverage:

| Model | Number of Responses |
|-------|---------------------|
| `gpt-4.1-2025-04-14` | 2,500,999 |
| `qwen3-coder-no_reasoning-30b-3a` | 2,500,999 |
| `qwen3-no_reasoning-30b-3a` | 2,500,999 |
| `qwen3-no_reasoning-4b` | 2,500,999 |
| `qwen3-reasoning-30b-3a` | 2,500,999 |
| `qwen3-reasoning-4b` | 2,500,999 |
| `phi4-mini-instruct` | 2,500,997 |
| `gemma3-12b` | 2,500,980 |
| `qwen3-no_reasoning-0.6b` | 2,500,075 |
| `qwen3-no_reasoning-14b` | 2,500,075 |
| `qwen3-no_reasoning-32b` | 2,500,075 |
| `mistral-24b` | 2,499,303 |
| `qwen3-no_reasoning-1.7b` | 2,499,075 |
| `qwen3-no_reasoning-8b` | 2,499,075 |
| `gpt-120b` | 2,498,450 |
| `gpt-20b` | 2,495,496 |
| `yi-34b` | 2,492,148 |
| `yi-9b` | 2,492,148 |
| `olmo2-13b` | 2,486,926 |
| `olmo2-1b` | 2,486,926 |
| `olmo2-32b` | 2,486,926 |
| `olmo2-7b` | 2,486,926 |
| `qwen3-reasoning-32b` | 2,460,620 |
| `qwq-32b` | 2,453,635 |
| `gemma3-4b` | 2,451,039 |
| `gemma3-27b` | 2,406,872 |
| `qwen3-reasoning-1.7b` | 2,278,035 |
| `qwen3-reasoning-8b` | 2,233,012 |
| `qwen3-reasoning-14b` | 1,993,762 |

**Total unique prompts:** 2,500,999
**Total models:** 29

## License

This dataset is licensed under ODC-BY. It is intended for research and educational use in accordance with Ai2's [Responsible Use Guidelines](https://allenai.org/responsible-use).

Note: each model has their own terms of use for their outputs. We refer users to the licenses and terms for each model when using portions of this dataset for downstream training.

## Citation

```
@misc{olmo2025olmo3,
title={Olmo 3},
author={Team Olmo and Allyson Ettinger and Amanda Bertsch and Bailey Kuehl and David Graham and David Heineman and Dirk Groeneveld and Faeze Brahman and Finbarr Timbers and Hamish Ivison and Jacob Morrison and Jake Poznanski and Kyle Lo and Luca Soldaini and Matt Jordan and Mayee Chen and Michael Noukhovitch and Nathan Lambert and Pete Walsh and Pradeep Dasigi and Robert Berry and Saumya Malik and Saurabh Shah and Scott Geng and Shane Arora and Shashank Gupta and Taira Anderson and Teng Xiao and Tyler Murray and Tyler Romero and Victoria Graf and Akari Asai and Akshita Bhagia and Alexander Wettig and Alisa Liu and Aman Rangapur and Chloe Anastasiades and Costa Huang and Dustin Schwenk and Harsh Trivedi and Ian Magnusson and Jaron Lochner and Jiacheng Liu and Lester James V. Miranda and Maarten Sap and Malia Morgan and Michael Schmitz and Michal Guerquin and Michael Wilson and Regan Huff and Ronan Le Bras and Rui Xin and Rulin Shao and Sam Skjonsberg and Shannon Zejiang Shen and Shuyue Stella Li and Tucker Wilde and Valentina Pyatkin and Will Merrill and Yapei Chang and Yuling Gu and Zhiyuan Zeng and Ashish Sabharwal and Luke Zettlemoyer and Pang Wei Koh and Ali Farhadi and Noah A. Smith and Hannaneh Hajishirzi},
year={2025},
eprint={2512.13961},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2512.13961},
}
```