omega-500

# Omega-500: Random Sample of Mathematical Problems

This dataset contains a random sample of 500 mathematical problems selected from the comprehensive OMEGA problem families dataset. It provides a diverse, manageable subset for quick evaluation and experimentation across multiple mathematical domains and difficulty levels.

## Overview

Omega-500 is designed for:
- **Quick Evaluation**: Fast assessment of model capabilities across math domains
- **Prototyping**: Testing new approaches before scaling to larger datasets
- **Benchmarking**: Standardized subset for fair model comparisons
- **Research**: Focused analysis on a balanced mathematical problem set

The sample maintains diversity across mathematical domains and difficulty levels while keeping the dataset size manageable for rapid iteration.

## Quick Start

```python
from datasets import load_dataset

# Load the Omega-500 sample
dataset = load_dataset("allenai/omega-500")
problems = dataset["train"]

# Access individual problems
first_problem = problems[0]
print("Problem:", first_problem["messages"][0]["content"])
print("Answer:", first_problem["ground_truth"])
print("Family:", first_problem["family"])
print("Difficulty:", first_problem["difficulty_level"])
```

## Dataset Composition

### Total Problems: 500

### Domain Distribution:
- **Algebra**: 92 problems (18.4%)
- **Arithmetic**: 173 problems (34.6%)
- **Combinatorics**: 84 problems (16.8%)
- **Geometry**: 45 problems (9.0%)
- **Logic**: 61 problems (12.2%)
- **Number Theory**: 45 problems (9.0%)

## Data Fields

Each problem contains:
- `id`: Unique identifier for this sample
- `original_id`: Original identifier from source dataset
- `family`: Problem family (e.g., "algebra_func_area")
- `difficulty_level`: Numeric difficulty level from source
- `source_family`: Source family directory name
- `source_level`: Source difficulty level name
- `messages`: Problem statement in chat format
- `ground_truth`: Correct answer
- `dataset`: Dataset identifier ("OMEGA_500_SAMPLE")

## Citation

If you use this dataset, please cite the original OMEGA work:

```bibtex
@article{sun2024omega,
title = {OMEGA: Can LLMs Reason Outside the Box in Math? Evaluating Exploratory, Compositional, and Transformative Generalization},
author = {Yiyou Sun and Shawn Hu and Georgia Zhou and Ken Zheng and Hannaneh Hajishirzi and Nouha Dziri and Dawn Song},
journal = {arXiv preprint arXiv:2506.18880},
year = {2024},
}
```

## Related Resources

- **Full Problem Families**: See [omega-problems](https://huggingface.co/datasets/allenai/omega-problems) for the complete dataset
- **Explorative Dataset**: See [omega-explorative](https://huggingface.co/datasets/allenai/omega-explorative) for explorative reasoning challenges
- **Compositional Dataset**: See [omega-compositional](https://huggingface.co/datasets/allenai/omega-compositional) for compositional reasoning challenges
- **Transformative Dataset**: See [omega-transformative](https://huggingface.co/datasets/allenai/omega-transformative) for transformative reasoning challenges
- **Paper**: See the full details in [paper](https://arxiv.org/pdf/2506.18880)
- **Code Repository**: See generation code on [github](https://github.com/sunblaze-ucb/math_ood)

# Omega-500：数学问题随机抽样数据集
本数据集从完整的OMEGA问题家族数据集（OMEGA problem families dataset）中随机抽取500道数学问题构建而成，提供了一个多样化且规模可控的子集，可用于跨多个数学领域与难度层级的快速评估与实验研究。

## 概述
Omega-500的设计用途包括：
- **快速评估**：快速评估模型在各数学领域的能力
- **原型开发**：在扩展至更大规模数据集前测试新方法
- **基准测试**：用于公平模型对比的标准化子集
- **研究工作**：针对均衡数学问题集开展聚焦性分析

该抽样样本在保持数学领域与难度层级多样性的同时，将数据集规模控制在便于快速迭代的范围内。

## 快速入门
python
from datasets import load_dataset

# 加载Omega-500抽样数据集
dataset = load_dataset("allenai/omega-500")
problems = dataset["train"]

# 访问单个问题
first_problem = problems[0]
print("Problem:", first_problem["messages"][0]["content"])
print("Answer:", first_problem["ground_truth"])
print("Family:", first_problem["family"])
print("Difficulty:", first_problem["difficulty_level"])

## 数据集构成
### 总问题数：500道
### 领域分布：
- **代数（Algebra）**：92道（18.4%）
- **算术（Arithmetic）**：173道（34.6%）
- **组合数学（Combinatorics）**：84道（16.8%）
- **几何（Geometry）**：45道（9.0%）
- **逻辑学（Logic）**：61道（12.2%）
- **数论（Number Theory）**：45道（9.0%）

## 数据字段
每个问题包含以下字段：
- `id`：该样本的唯一标识符
- `original_id`：源数据集的原始标识符
- `family`：问题家族（例如："algebra_func_area"）
- `difficulty_level`：源数据集给出的数值型难度层级
- `source_family`：源家族目录名称
- `source_level`：源难度层级名称
- `messages`：对话格式的问题陈述
- `ground_truth`：正确答案
- `dataset`：数据集标识符（"OMEGA_500_SAMPLE"）

## 引用说明
若使用本数据集，请引用原始OMEGA研究成果：
bibtex
@article{sun2024omega,
title = {OMEGA: Can LLMs Reason Outside the Box in Math? Evaluating Exploratory, Compositional, and Transformative Generalization},
author = {Yiyou Sun and Shawn Hu and Georgia Zhou and Ken Zheng and Hannaneh Hajishirzi and Nouha Dziri and Dawn Song},
journal = {arXiv preprint arXiv:2506.18880},
year = {2024},
}

## 相关资源
- **完整问题家族数据集**：可访问[omega-problems](https://huggingface.co/datasets/allenai/omega-problems)获取完整数据集
- **探索式推理数据集**：可访问[omega-explorative](https://huggingface.co/datasets/allenai/omega-explorative)获取探索式推理挑战集
- **组合式推理数据集**：可访问[omega-compositional](https://huggingface.co/datasets/allenai/omega-compositional)获取组合式推理挑战集
- **转换式推理数据集**：可访问[omega-transformative](https://huggingface.co/datasets/allenai/omega-transformative)获取转换式推理挑战集
- **研究论文**：详细内容可查阅[论文](https://arxiv.org/pdf/2506.18880)
- **代码仓库**：生成代码可访问[GitHub](https://github.com/sunblaze-ucb/math_ood)