PolyUniMath

# PolyUniMath ## Dataset Summary **PolyUniMath** is a large multilingual mathematics dataset of question-solution-answer pairs extracted from mathematical PDFs. The dataset is designed for training and studying natural-language mathematical reasoning, with a strong emphasis on university-level content. - **Sample count**: approximately 4 million Q&A pairs - **Main focus**: university-level mathematics - **Format**: problem, optional choices, solution, final answer, and auxiliary annotations - **Metadata**: We provide rich metadata for each sample collected from various stages of our data generation pipeline - **Translation**: Each sample contains a translation from the base language into English using Qwen3-32B as the translator The data itself is collected from PDF data contained in the [CommonCrawl archives](https://commoncrawl.org/), which we refetch, OCR, and then pass through several filtering and processing stages. The full pipeline is described and visualized in our linked paper. ## Dataset Statistics ### Math Levels The dataset is predominantly rated highest as university-level: - `score_university`: 80.2% - `score_university_competition`: about 19% - `score_highschool_competition`: less than 1% This is intended, as our goal was to collect data only from high school competition to the university-level math. However, each datapoint is scored on all difficulty levels (see Section 'Features' below), with only the highest of those scores determining the class. ### Language Distribution Our dataset covers a **wide range of languages**, all of which offer a machine translation for the QSA pairs. This enables the use of our dataset for downstream tasks, such as distilling the translation capability. multi-language math-reasoning or development of language-specific models. The language distribution is roughly (top 10): - `en`: 60.5% - `fr`: 8.7% - `de`: 4.9% - `ru`: 3.3% - `es`: 3.3% - `ca`: 2.6% - `it`: 2.4% - `pt`: 2.3% - `ro`: 1.5% - `cs`: 1.1% - `other`: 9.5% ## Dataset Structure Each row corresponds to one math QSA item. The dataset combines source text with machine-generated metadata, translations, and filtering annotations. For a full description of the various stages and processes that generate the metadata, refer to our linked paper. ## Features | Column | Description | |---|---| | `text` | Raw text record for the sample. | | `id` | Unique identifier for the Q&A pair. | | `doc_id` | Identifier of the source document from the CommonCrawl archives. | | `qa_nr_in_doc` | Index of the sample within the source document. | | `problem` | Problem statement. | | `options` | Multiple-choice options, if present. | | `solution` | Worked solution or explanation. | | `math_type` | Coarse mathematical topic label. | | `answer` | Final answer. | | `dataset` | Source subset or release name. | | `paper_score` | Source-document score from the PDF mining pipeline. | | `score_elementary` | Difficulty score for elementary level. | | `score_highschool` | Difficulty score for high-school level. | | `score_highschool_competition` | Difficulty score for high-school competition level. | | `score_university` | Difficulty score for university level. | | `score_university_competition` | Difficulty score for university competition level. | | `score_research` | Difficulty score for research level. | | `self_contained` | Whether the problem is self-contained. | | `qwen_translated_problem` | Machine-translated version of the problem. | | `qwen_translated_solution` | Machine-translated version of the solution. | | `qwen_translated_answer` | Machine-translated version of the final answer. | | `lang` | Detected language code of the original sample. | | `model_answer` | Auxiliary model-produced answer or extraction field. | | `question_type` | Coarse question format, such as `math-word-problem`, `proof`, or `MCQ`. | | `problem_is_valid` | Validity tag for the problem statement. | | `solution_is_valid` | Validity tag for the solution. | | `fine_math_domain` | Fine-grained [FineMath](https://arxiv.org/abs/2403.07747) domain annotation stored in numeric form. | Translation and verification steps are conducted using [Qwen3-32B](https://huggingface.co/Qwen/Qwen3-32B). ## Use PolyUniMath is intended for: - **continued pretraining** on mathematical text - supervised fine-tuning on math Q&A data - filtering, ablation, and multilingual analysis of math corpora - language-specific training tasks, such as translation of mathematical texts In our paper, we provide several ablation experiments for different use cases using PolyUniMath, showing significant gains on CPT for various pre-trained models, such as Qwen3-1.7B, LLama-3.2-1B,and 7B, etc. ## Acknowledgments Thanks to **BenchXiv**, **HuggingFace data team**, and **Project Numina** for providing the resources (both compute and time) that enabled us to create this dataset.