logic-rl-24k

# Logic RL 24K Logic RL 24K is a 24,461-example English reasoning mixture prepared for reinforcement learning with verifiable rewards (RLVR). It combines procedurally generated, algorithmically verifiable tasks from NVIDIA's Nemotron RL Reasoning Gym release with logic and table-reasoning tasks selected from LLM360's Guru RL 92K collection. Every record contains one user message, a reference answer, and task-specific metadata that can be used to route the example to the appropriate reward function. The dataset does not contain model-generated responses or chain-of-thought traces. ## Dataset summary | Property | Value | |---|---:| | Examples | 24,461 | | Splits | `train` only | | File | `logic-rl-24k.jsonl` | | File size | 176,601,547 bytes (about 168 MiB) | | Format | UTF-8 JSON Lines; one object per line | | Prompt shape | One `user` message per example | | Token count (`metadata.num_tokens`) | median 240; mean 503.5; 95th percentile 1,907; range 41–3,967 | The statistics above were computed over the released file. All 24,461 lines parse as JSON and contain the top-level fields `prompt`, `label`, and `metadata`. ## Composition | Component | Selector | Examples | Share | |---|---|---:|---:| | Nemotron RL Reasoning Gym | `metadata.data_type == "nemotron_rl_reasoning_gym"` | 14,234 | 58.19% | | Guru RL | `metadata.data_type == "guru_rl"` | 10,227 | 41.81% | ### Nemotron RL Reasoning Gym subset The 14,234 Nemotron records cover 99 procedural task generators. Representative areas include arithmetic and algebra, geometry, symbolic manipulation, logic, graphs, matrices, calendars, string operations, constraint puzzles, and games. The generator name is stored in `metadata.source_dataset`; task-specific generation parameters and the reference answer are retained in `metadata`. All records in this component have string-valued labels and include a unique UUID. The source records identify the license as CC BY 4.0. Original source: [nvidia/Nemotron-RL-ReasoningGym-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-ReasoningGym-v1) ### Guru RL subset | `metadata.data_source` | Ability | Examples | `label` type | |---|---|---:|---| | `table__hitab` | Table reasoning | 4,278 | string | | `logic__ordering_puzzle_dataset` | Logical reasoning | 1,884 | list | | `table__multihier` | Table reasoning | 1,515 | string | | `logic__zebra_puzzle_dataset` | Logical reasoning | 1,308 | object | | `logic__graph_logical_dataset` | Logical reasoning | 1,242 | string | | **Total** | | **10,227** | | The Guru component contains 5,793 table-reasoning examples and 4,434 logical-reasoning examples. Its source is the MIT-licensed [LLM360/guru-RL-92k](https://huggingface.co/datasets/LLM360/guru-RL-92k) collection. ## Data structure Each JSON object has three top-level fields: ```json { "prompt": [ {"role": "user", "content": "..."} ], "label": "...", "metadata": { "data_type": "...", "ground_truth": "...", "num_tokens": 123 } } ``` ### Fields - `prompt`: A list containing exactly one chat message with `role: "user"`. The content includes the problem and its expected answer-format instruction. - `label`: The reference answer. This field is intentionally heterogeneous: 21,269 values are strings, 1,884 are lists, and 1,308 are objects. - `metadata`: Provenance, reward-routing information, ground truth, token count, and task-specific fields. Its nested schema varies by source task. Important metadata fields include: - `data_type`: Selects the top-level reward family (`nemotron_rl_reasoning_gym` or `guru_rl`). - `ground_truth`: The expected answer, generally matching `label` in value. - `num_tokens`: Precomputed token count supplied for every example. - `source_dataset`: Procedural environment name for Nemotron records. - `data_source`: Guru task family used for reward routing. - `ability`: `table` or `logical_reasoning` for Guru records. - `extra_info`: Source-specific auxiliary data for Guru records. - `uuid`, `question`, `answer`, `difficulty`, and task-specific generation fields: Present where applicable in Nemotron records. ## Loading the dataset Load the dataset directly with ModelScope: ```python from modelscope.msdatasets import MsDataset dataset = MsDataset.load( "wflying000/logic-rl-24k", subset_name="default", split="train", ) example = dataset[0] prompt = example["prompt"] label = example["label"] metadata = example["metadata"] ``` Frameworks that require a fixed column type can JSON-encode the heterogeneous fields before constructing their table or dataset: ```python import json example["label_json"] = json.dumps(example.pop("label"), ensure_ascii=False) example["metadata_json"] = json.dumps(example.pop("metadata"), ensure_ascii=False) ``` For RLVR training, route examples first by `metadata.data_type`, and route Guru examples again by `metadata.data_source`. Expected response formats differ across tasks, including `<answer>...</answer>` tags and LaTeX `\\boxed{...}` output. ## Data quality and validation - 24,461 of 24,461 lines are valid JSON. - Every record contains `prompt`, `label`, and `metadata`. - Every prompt contains exactly one user message. - All records contain `metadata.num_tokens`. - The 14,234 Nemotron records contain 14,234 distinct UUIDs; no duplicate UUIDs were found. - Reference answers are exact task outputs, not generated reasoning traces. These checks validate syntax and basic structural invariants; they do not constitute a manual correctness review of every problem or reference answer. ## Intended use This dataset is intended for research and development involving: - reinforcement learning with verifiable rewards; - multi-task logic and mathematical reasoning; - table question answering; - reward-function routing and evaluation; - controlled experiments on mixtures of procedural and curated reasoning tasks. It should not be treated as a standalone benchmark: there is no held-out validation or test split, and source overlap should be checked before evaluation. ## Limitations - The release contains only a training split. - Task families are not uniformly distributed. - Answer representations and required output formats vary by source. - `label` and `metadata` are heterogeneous JSON values, which can require normalization for columnar data tools. - Some prompts are long tables or structured puzzles; `metadata.num_tokens` reaches 3,967. - Source datasets may contain their own errors, ambiguities, biases, or duplicated concepts. - The compilation has not been independently audited for personal information, harmful content, or contamination against downstream benchmarks. Users are responsible for validating task suitability, reward functions, and train/evaluation separation for their application. ## Licensing and attribution This is a mixed-license compilation: - Records whose `metadata.data_type` is `nemotron_rl_reasoning_gym` identify their license as [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) and originate from [nvidia/Nemotron-RL-ReasoningGym-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-ReasoningGym-v1). - Records whose `metadata.data_type` is `guru_rl` are derived from the MIT-licensed [LLM360/guru-RL-92k](https://huggingface.co/datasets/LLM360/guru-RL-92k) collection. The repository therefore uses the Hub's `other` license tag rather than incorrectly assigning one license to every record. Consult and comply with the original source licenses and attribution requirements. This card is informational and is not legal advice. ## Citation For the Guru RL component, cite the source dataset and paper: ```bibtex @misc{cheng2025revisiting, title = {Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective}, author = {Zhoujun Cheng and Shibo Hao and Tianyang Liu and Fan Zhou and Yutao Xie and Feng Yao and Yuexin Bian and Yonghao Zhuang and Nilabjo Dey and Yuheng Zha and Yi Gu and Kun Zhou and Yuqi Wang and Yuan Li and Richard Fan and Jianshu She and Chengqian Gao and Abulhair Saparov and Haonan Li and Taylor W. Killian and Mikhail Yurochkin and Zhengzhong Liu and Eric P. Xing and Zhiting Hu}, journal = {arXiv preprint arXiv:2506.14965}, year = {2025}, doi = {10.48550/arXiv.2506.14965}, url = {https://arxiv.org/abs/2506.14965} } ``` For the procedural component, also cite and attribute [nvidia/Nemotron-RL-ReasoningGym-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-ReasoningGym-v1) and its upstream [Reasoning Gym](https://github.com/open-thought/reasoning-gym) project as requested by their respective releases.