reasoning-core/symbolic-pretraining-pile

---
dataset_info:
features:
- name: prompt
dtype: string
- name: answer
dtype: string
- name: metadata
dtype: string
- name: task
dtype: string
- name: level
dtype: int64
- name: mode
dtype: string
splits:
- name: train
num_bytes: 35043826484.797325
num_examples: 16009152
- name: test
num_bytes: 353978907.62921673
num_examples: 161709
download_size: 13412661648
dataset_size: 35397805392.426544
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: test
path: data/test-*
license: mit
task_categories:
- question-answering
language:
- en
tags:
- SFT
- reasoning
- logic
- procedural
- formal
- synthetic
- pretraining
- pre-training
- corpus
- formal-pretraining
size_categories:
- 10M<n<100M
---

# Reasoning-Core : Symbolic Pre-Training pile (SPT) ◉

SPT is designed for symbolic/formal pre-training, mid-training and SFT.
The data is procedurally generated on cpu and can be scaled to trillion tokens, and the difficulty is also adjustable with a single knob.

## Task Categories
📐 **Formal Reasoning**: planning • conjecture_entailment • proof_reconstruction
📜 **Formal Semantics, Logic**: logic_nli • evidence_retrieval
🔢 **Mathematical computation**: equation_system • arithmetics • symbolic_arithmetics • sequential_induction
💻 **Code & Execution**: code_execution • diff_prediction • diff_patching
🕸️ **Graph Theory**: graph_pathfinding • graph_node_centrality • graph_cycle_detection • graph_isomorphism
🎲 **Probabilistic**: bayesian_association • bayesian_intervention
📝 **Language Parsing, Syntax**: regex_following • regex_induction • parsability • parsing • continuation
📋 **Table Processing**: table_qa • table_conversion
🔎 **Set Operations, Retrieval**: set_intersection • set_missing_element • set_equality

## Task Modes
We provide three modes for most tasks, all in SFT/pretraining suitable format:
➡️ **Instruct mode**: Direct prompt/answer format
🧠 **Trace mode**: Most tasks include reasoning traces to bake-in chain-of-thought reasoning patterns
✅ **Verification mode**: Tasks framed as prompt/candidate: valid (yes/no)? 10% of the time, to strengthen reasoning self-verification capabilities

🧪 [Paper: Reasoning Core: A Scalable RL Environment for LLM Symbolic Reasoning](https://huggingface.co/papers/2509.18083)
📦 [Code: GitHub Repository](https://github.com/sileod/reasoning_core) *(An updated paper for pre-training results is coming.)*

## RLVR version
See [rc1](https://huggingface.co/datasets/reasoning-core/rc1/) for the post-training/RLVR version

## Abstract
We introduce Reasoning Core, a new scalable environment for Reinforcement
Learning with Verifiable Rewards (RLVR), designed to advance foundational
symbolic reasoning in Large Language Models (LLMs). Unlike existing benchmarks
that focus on games or isolated puzzles, Reasoning Core procedurally generates
problems across core formal domains, including PDDL planning, first-order
logic, context-free grammar parsing, causal reasoning, and system equation
solving. The environment is built on key design principles of high-generality
problem distributions, verification via external tools, and continuous
difficulty control, which together provide a virtually infinite supply of novel
training instances. Initial zero-shot evaluations with frontier LLMs confirm
the difficulty of Reasoning Core's tasks, positioning it as a promising
resource to improve the reasoning capabilities of future models.

## Usage
`ds = load_dataset("reasoning-core/symbolic-pretraining-pile")`

# Citation
```
@article{reasoningcore2026,
title={Reasoning Core: A Scalable Procedural Data Generation Suite for Symbolic Pre-training and Post-Training},
author={Lacombe, Valentin and Quesnel, Valentin and Sileo, Damien},
journal={arXiv preprint arXiv:2603.02208},
year={2026},
url={https://arxiv.org/abs/2603.02208}
}
```