B2-UniMath

# B2-UniMath ## Dataset Summary **B2-UniMath** is a large multilingual mathematics dataset of question-solution-answer pairs extracted from mathematical PDFs. The dataset is designed for training and studying natural-language mathematical reasoning, with a strong emphasis on university-level content. - **Sample count**: approximately 4 million Q&A pairs - **Main focus**: university-level mathematics - **Format**: problem, optional choices, solution, final answer, and auxiliary annotations - **Metadata**: We provide rich metadata for each sample collected from various stages of our data generation pipeline - **Translation**: Each sample contains a translation from the base language into English using Qwen3-32B as the translator The data itself is collected from PDF data contained in the [CommonCrawl archives](https://commoncrawl.org/), which we refetch, OCR, and then pass through several filtering and processing stages. The full pipeline is described and visualized in our linked paper. ## Dataset Statistics ### Math Levels The dataset is predominantly rated highest as university-level: - `score_university`: 80.2% - `score_university_competition`: about 19% - `score_highschool_competition`: less than 1% This is intended, as our goal was to collect data only from high school competition to the university-level math. However, each datapoint is scored on all difficulty levels (see Section 'Features' below), with only the highest of those scores determining the class. ### Language Distribution Our dataset covers a **wide range of languages**, all of which offer a machine translation for the QSA pairs. This enables the use of our dataset for downstream tasks, such as distilling the translation capability. multi-language math-reasoning or development of language-specific models. The language distribution is roughly (top 10): - `en`: 60.5% - `fr`: 8.7% - `de`: 4.9% - `ru`: 3.3% - `es`: 3.3% - `ca`: 2.6% - `it`: 2.4% - `pt`: 2.3% - `ro`: 1.5% - `cs`: 1.1% - `other`: 9.5% ## Dataset Structure Each row corresponds to one math QSA item. The dataset combines source text with machine-generated metadata, translations, and filtering annotations. For a full description of the various stages and processes that generate the metadata, refer to our linked paper. ## Features | Column | Description | |---|---| | `text` | Raw text record for the sample. | | `id` | Unique identifier for the Q&A pair. | | `doc_id` | Identifier of the source document from the CommonCrawl archives. | | `qa_nr_in_doc` | Index of the sample within the source document. | | `problem` | Problem statement. | | `options` | Multiple-choice options, if present. | | `solution` | Worked solution or explanation. | | `math_type` | Coarse mathematical topic label. | | `answer` | Final answer. | | `dataset` | Source subset or release name. | | `paper_score` | Source-document score from the PDF mining pipeline. | | `score_elementary` | Difficulty score for elementary level. | | `score_highschool` | Difficulty score for high-school level. | | `score_highschool_competition` | Difficulty score for high-school competition level. | | `score_university` | Difficulty score for university level. | | `score_university_competition` | Difficulty score for university competition level. | | `score_research` | Difficulty score for research level. | | `self_contained` | Whether the problem is self-contained. | | `qwen_translated_problem` | Machine-translated version of the problem. | | `qwen_translated_solution` | Machine-translated version of the solution. | | `qwen_translated_answer` | Machine-translated version of the final answer. | | `lang` | Detected language code of the original sample. | | `model_answer` | Auxiliary model-produced answer or extraction field. | | `question_type` | Coarse question format, such as `math-word-problem`, `proof`, or `MCQ`. | | `problem_is_valid` | Validity tag for the problem statement. | | `solution_is_valid` | Validity tag for the solution. | | `fine_math_domain` | Fine-grained [FineMath](https://arxiv.org/abs/2403.07747) domain annotation stored in numeric form. | Translation and verification steps are conducted using [Qwen3-32B](https://huggingface.co/Qwen/Qwen3-32B). ## Use B2-UniMath is intended for: - **continued pretraining** on mathematical text - supervised fine-tuning on math Q&A data - filtering, ablation, and multilingual analysis of math corpora - language-specific training tasks, such as translation of mathematical texts In our paper, we provide several ablation experiments for different use cases using B2-UniMath, showing significant gains on CPT for various pre-trained models, such as Qwen3-1.7B, LLama-3.2-1B,and 7B, etc. ## Acknowledgments We would like to thank the **HuggingFace data team**, **Benchmarks&Baselines** and **Project Numina** for providing the resources (both compute and time) that enabled us to create this dataset.