omega-problems

# Mathematical Problem Families by Difficulty

This dataset contains mathematical problems organized by problem families, with each family spanning multiple difficulty levels. This organization allows for studying how mathematical reasoning scales with problem complexity within specific mathematical domains.

## Overview

Each problem family represents a specific type of mathematical problem (e.g., function area calculation, matrix operations, probability calculations) with problems spanning multiple difficulty levels. This structure enables researchers to study:

- **Difficulty Scaling**: How mathematical reasoning changes as problems become more complex
- **Domain-Specific Learning**: How models learn within specific mathematical areas
- **Progressive Training**: Training on easier levels and testing on harder ones
- **Curriculum Learning**: Gradually increasing difficulty within problem types

## Quick Start

```python
from datasets import load_dataset

# Load all problem families
dataset = load_dataset("allenai/omega-prob-families")

# Load a specific problem family with all difficulty levels
func_area_data = load_dataset("allenai/omega-prob-families", "algebra_func_area")
level_1_data = func_area_data["level_1"] # Easiest problems
level_5_data = func_area_data["level_5"] # Hardest problems

# Load just a specific difficulty level
level_3_only = load_dataset("allenai/omega-prob-families", "algebra_func_area", split="level_3")

# Get all available families
available_families = list(dataset.keys())
print(f"Available families: {available_families}")
```

## Dataset Description

Each problem family contains exactly 100 problems per difficulty level (or fewer if the original file contained fewer than 100), allowing for systematic study of how mathematical reasoning scales with complexity. The families cover various mathematical domains including:

- **Algebra**: Function analysis, polynomial operations, linear equations
- **Arithmetic**: Matrix operations, number theory, basic calculations
- **Geometry**: Shape analysis, transformations, spatial reasoning
- **Combinatorics**: Probability, pattern matching, counting problems
- **Logic**: Grid-based puzzles, constraint satisfaction

## Family Structure

Each family is organized as:
- **Family Name**: Descriptive identifier (e.g., `algebra_func_area`)
- **Difficulty Levels**: Progressive levels from 1 (easiest) to N (hardest)
- **Problem Count**: Exactly 100 problems per level (or fewer if original file had fewer)
- **Consistent Format**: Same problem type across all levels, increasing in complexity

## Citation

If you use this dataset, please cite the original work:

```bibtex
@article{sun2024omega,
title = {OMEGA: Can LLMs Reason Outside the Box in Math? Evaluating Exploratory, Compositional, and Transformative Generalization},
author = {Yiyou Sun and Shawn Hu and Georgia Zhou and Ken Zheng and Hannaneh Hajishirzi and Nouha Dziri and Dawn Song},
journal = {arXiv preprint arXiv:2506.18880},
year = {2024},
}
```

## Related Resources

- **Explorative Dataset**: See [omega-explorative](https://huggingface.co/datasets/allenai/omega-explorative) for explorative reasoning challenges
- **Compositional Dataset**: See [omega-compositional](https://huggingface.co/datasets/allenai/omega-compositional) for compositional reasoning challenges
- **Transformative Dataset**: See [omega-transformative](https://huggingface.co/datasets/allenai/omega-transformative) for transformative reasoning challenges
- **Paper**: See the full details in [paper](https://arxiv.org/pdf/2506.18880)
- **Code Repository**: See generation code on [github](https://github.com/sunblaze-ucb/math_ood)

# 按难度分级的数学问题族数据集

本数据集包含按问题族（problem families）组织的数学题目，每个问题族涵盖多个难度层级。该组织方式支持研究者在特定数学领域内，探究数学推理能力随问题复杂度的变化规律。

## 概览

每个问题族代表一类特定的数学题型（例如函数域计算、矩阵运算、概率计算），其包含的题目覆盖多个难度层级。此结构可支持研究者开展以下方向的研究：
- **难度缩放研究**：数学推理能力如何随问题复杂度提升而变化
- **领域专属学习研究**：模型在特定数学领域内的学习规律
- **渐进式训练**：以低难度题目训练、高难度题目测试的训练范式
- **课程学习**：在同一题型内逐步提升任务难度的学习策略

## 快速入门

python
from datasets import load_dataset

# 加载所有问题族数据集
dataset = load_dataset("allenai/omega-prob-families")

# 加载包含所有难度层级的特定问题族
func_area_data = load_dataset("allenai/omega-prob-families", "algebra_func_area")
level_1_data = func_area_data["level_1"] # 难度最低的题目
level_5_data = func_area_data["level_5"] # 难度最高的题目

# 仅加载指定难度层级的数据集
level_3_only = load_dataset("allenai/omega-prob-families", "algebra_func_area", split="level_3")

# 获取所有可用的问题族
available_families = list(dataset.keys())
print(f"可用问题族：{available_families}")

## 数据集详情

每个问题族的每个难度层级均包含恰好100道题目（若原始文件中题目数量不足100，则以实际数量为准），这为系统探究数学推理能力随复杂度的变化规律提供了支撑。本数据集覆盖的数学领域包括：
- **代数学**：函数分析、多项式运算、线性方程求解
- **算术**：矩阵运算、数论、基础计算
- **几何学**：图形分析、几何变换、空间推理
- **组合数学**：概率计算、模式匹配、计数问题
- **逻辑学**：网格谜题、约束满足问题

## 问题族结构

每个问题族的组织形式如下：
- **族名称**：具有描述性的标识符（例如`algebra_func_area`）
- **难度层级**：从1（最低难度）到N（最高难度）的渐进式分级
- **题目数量**：每个难度层级对应100道题目（若原始文件中题目数量不足100，则以实际数量为准）
- **格式统一**：所有层级的题目均为同一题型，仅复杂度随层级提升而增加

## 引用

若您使用本数据集，请引用以下原始文献：

bibtex
@article{sun2024omega,
title = {OMEGA: 大语言模型（Large Language Models, LLMs）能否在数学领域跳出固有思维？评估探索性、组合性与变换性泛化能力},
author = {Yiyou Sun and Shawn Hu and Georgia Zhou and Ken Zheng and Hannaneh Hajishirzi and Nouha Dziri and Dawn Song},
journal = {arXiv预印本 arXiv:2506.18880},
year = {2024},
}

## 相关资源

- **探索性推理数据集**：可访问[omega-explorative](https://huggingface.co/datasets/allenai/omega-explorative)获取探索性推理挑战数据集
- **组合性推理数据集**：可访问[omega-compositional](https://huggingface.co/datasets/allenai/omega-compositional)获取组合性推理挑战数据集
- **变换性推理数据集**：可访问[omega-transformative](https://huggingface.co/datasets/allenai/omega-transformative)获取变换性推理挑战数据集
- **论文原文**：可访问[论文](https://arxiv.org/pdf/2506.18880)获取完整研究细节
- **代码仓库**：可访问[github](https://github.com/sunblaze-ucb/math_ood)获取题目生成代码