instruction-following-rl-content-constrained-30k

# Instruction-Following RL Content-Constrained 30K ## Dataset summary Instruction-Following RL Content-Constrained 30K is a training dataset for precise instruction following and reinforcement learning from verifiable rewards (RLVR). The current cleaned revision contains 29,520 heterogeneous, single-turn user prompts. Each prompt combines a substantive task with one to five explicit output constraints, such as keyword inclusion or exclusion, response length, capitalization, punctuation, paragraph structure, copying, target language, and start/end requirements. The data is primarily derived from AllenAI's [IF_multi_constraints_upto5](https://huggingface.co/datasets/allenai/IF_multi_constraints_upto5). That upstream dataset contains IF-RLVR prompts with up to five constraints sampled from 25 IFEval constraint types and 29 IFBench-Train constraint types. This release contains the same 54 instruction IDs across 16 ID-prefix families. Every local prompt also includes a common content-quality instruction telling the model to satisfy both the substantive task and the formal constraints, avoid superficial reward hacking, and produce relevant, useful content. This shared appendix is not represented by a separate machine-checkable instruction ID. ## Files and integrity | File | Split | Rows | Bytes | SHA-256 | |---|---:|---:|---:|---| | `instruction-following-rl-content-constrained-30k.jsonl` | train | 29,520 | 65,247,519 (62.23 MiB) | `02f9c6155441d46b6cc8d8cd02358715444f133f681f54cc4be646ee30177a79` | The statistics in this card were recomputed with a complete streaming pass over the cleaned JSONL file on 2026-08-11. All 29,520 lines parsed as JSON and shared the same schema. ## Data structure Each JSONL row has the following shape: ```text prompt: list[{ role: "user", content: string }] label: string # empty in every row metadata: { data_type: "if_content_constrained", instruction_id_list: list[string], instruction_kwargs: list[dict | null] } ``` `instruction_id_list[i]` and `instruction_kwargs[i]` form an aligned verifier specification. A `null` or empty dictionary is expected for a parameter-free constraint. Across this file, the two lists have equal lengths in every row; all instruction IDs are strings. The records contain no reference response, scalar reward, pass rate, or model output. `label` is an empty string in all 29,520 rows. ## Provenance and processing The dataset owner identifies [allenai/IF_multi_constraints_upto5](https://huggingface.co/datasets/allenai/IF_multi_constraints_upto5) as the primary source. A content-level comparison also matched the first local record to the corresponding AllenAI record, including the substantive bill-summarization task, constraint text, instruction IDs, and arguments. The observed processing is: - Upstream user `messages` are stored as `prompt`. - The upstream verifier specification in `ground_truth` is decoded into `metadata.instruction_id_list` and `metadata.instruction_kwargs`. - `metadata.data_type` is set to `if_content_constrained`. - A common content-quality/anti-reward-hacking appendix is added to every prompt. - An empty `label` field is added. - Upstream fields such as `key`, `dataset`, `constraint_type`, and the plain-text `constraint` field are not retained. Because the upstream `key` was dropped, this release does not provide reliable row-level joins back to the source. The exact 30K selection rule and any additional source records are not encoded in the file. ## Dataset statistics ### Record-level validation | Statistic | Value | |---|---:| | Rows | 29,520 | | Unique prompts | 29,520 | | Unique complete rows | 29,520 | | Exact duplicate prompts or rows | 0 | | Rows with the common content-quality appendix | 29,520 | | Rows with mismatched instruction/kwargs lengths | 0 | | Total constraint instances | 92,192 | | Unique instruction IDs | 54 | | Unique ordered instruction-ID combinations | 21,695 | Prompt lengths range from 699 to 392,434 characters. The median is 1,335 characters, the mean is 1,687.29, and the 95th percentile is 2,705.05. A small number of very long source documents create substantial length outliers. ### Constraints per row | Constraints | Rows | Share | |---:|---:|---:| | 1 | 3,959 | 13.41% | | 2 | 5,573 | 18.88% | | 3 | 7,343 | 24.87% | | 4 | 8,167 | 27.67% | | 5 | 4,478 | 15.17% | Mean constraints per row: **3.1230**. ### Constraint-family distribution Counts below are constraint instances, not rows. A row may contribute to several families. | Instruction-ID prefix | Instances | Share of all constraints | |---|---:|---:| | `keywords` | 22,005 | 23.87% | | `detectable_format` | 12,780 | 13.86% | | `length_constraints` | 8,318 | 9.02% | | `count` | 7,592 | 8.23% | | `change_case` | 4,869 | 5.28% | | `last_word` | 4,588 | 4.98% | | `copy` | 4,516 | 4.90% | | `letters` | 4,482 | 4.86% | | `detectable_content` | 4,478 | 4.86% | | `first_word` | 4,426 | 4.80% | | `punctuation` | 4,057 | 4.40% | | `startend` | 3,563 | 3.86% | | `paragraphs` | 2,594 | 2.81% | | `language` | 1,913 | 2.08% | | `combination` | 1,229 | 1.33% | | `new` | 782 | 0.85% | The most frequent individual IDs are `last_word:last_word_answer` (2,335), `length_constraints:number_sentences` (2,267), `keywords:word_once` (2,267), `startend:end_checker` (2,263), and `last_word:last_word_sent` (2,253). `language:response_language` occurs 1,913 times and requests one of 30 response-language codes: `ar`, `bg`, `bn`, `de`, `en`, `es`, `fa`, `fi`, `fr`, `gu`, `he`, `hi`, `it`, `ja`, `kn`, `ko`, `ml`, `mr`, `ne`, `pa`, `pl`, `pt`, `ru`, `sw`, `ta`, `te`, `th`, `uk`, `ur`, or `vi`. The prompts themselves are primarily English; this list describes requested response languages, not a verified language label for every prompt. ### Heuristic task shapes The following distribution was produced by a rule-based content router over prompt text and constraint metadata. It is descriptive rather than ground-truth annotation. | Heuristic class | Rows | Share | |---|---:|---:| | Format-heavy | 7,926 | 26.85% | | Code | 5,803 | 19.66% | | Math | 5,124 | 17.36% | | Copy-only | 3,075 | 10.42% | | General generation | 2,455 | 8.32% | | Target language | 1,729 | 5.86% | | Repeat-then-answer | 832 | 2.82% | | Table/SQL | 778 | 2.64% | | Structured output | 742 | 2.51% | | Short answer | 624 | 2.11% | | Translation | 432 | 1.46% | ## Loading the dataset ```python from modelscope.msdatasets import MsDataset dataset = MsDataset.load( "wflying000/instruction-following-rl-content-constrained-30k", subset_name="default", split="train", ) row = dataset[0] for instruction_id, kwargs in zip( row["metadata"]["instruction_id_list"], row["metadata"]["instruction_kwargs"], ): print(instruction_id, kwargs) ``` Because of the 392K-character prompt outlier, users with strict memory or batching limits may prefer `streaming=True` and length-aware batching. ## Reward design The metadata is intended for an ID-based hard-constraint verifier: instantiate the checker named by each instruction ID with its paired kwargs, evaluate the candidate response, and require all hard constraints to pass. The shared content-quality appendix is not captured by these IDs, so a rule-only verifier cannot determine whether a formally valid response substantively answers the task. For content-constrained RL, a two-stage reward is recommended: 1. Require every machine-checkable constraint to pass. 2. Separately evaluate substantive relevance and non-degeneracy with a pinned semantic judge or carefully validated content rules. Verifier versions, tokenization, sentence splitting, Unicode normalization, whitespace handling, and judge prompts can materially change rewards. Pin and document them for reproducibility. This repository contains the training data only; it does not bundle a canonical verifier implementation or judge model. ## Intended uses - RLVR training for precise instruction following. - Multi-constraint curriculum and composition research. - Development of hard-constraint verifiers and hybrid rule/semantic rewards. - Analysis of format compliance versus substantive task quality. ## Limitations and risks - Only a `train` split is provided; there is no independent validation or test set. - All labels are empty, and no reference responses or precomputed rewards are included. - Multi-constraint compositions have not been exhaustively proven satisfiable under every verifier implementation. - The exact verifier implementation is not part of the dataset, so results can vary across implementations. - The common content-quality instruction requires semantic judgment and is not fully verifiable from the structured metadata. - Dropping upstream row keys reduces provenance, deduplication, contamination analysis, and correction traceability. - The substantive tasks are heterogeneous and may contain factual errors, outdated material, copyrighted text, personal information, political content, or unsafe requests inherited from source mixtures. No comprehensive content or privacy audit was performed for this repackaging. - A prompt can request output in a language for which the underlying task or verifier has not been linguistically validated. - Training on these templates may improve template-specific compliance without guaranteeing generalization to unseen constraints or real-world user intent. ## License, responsible use, and citation The primary upstream dataset, [allenai/IF_multi_constraints_upto5](https://huggingface.co/datasets/allenai/IF_multi_constraints_upto5), is released under ODC-BY-1.0, which is reflected in this repository's machine-readable metadata. Its card states that the data is intended for research and educational use under [Ai2's Responsible Use Guidelines](https://allenai.org/responsible-use), and that output data generated by third-party models may be subject to separate terms. Any records or underlying task content originating from additional sources remain subject to their original licenses and terms. Users should review the upstream card and source-mixture obligations before redistribution or commercial use. The constraint taxonomy builds on [Google IFEval](https://huggingface.co/datasets/google/IFEval) and IFBench-Train. For methodology and citation details, see [Generalizing Verifiable Instruction Following](https://arxiv.org/abs/2507.02833) by Pyatkin et al. (NeurIPS 2025). ```bibtex @inproceedings{pyatkin2025generalizing, title={Generalizing Verifiable Instruction Following}, author={Pyatkin, Valentina and Malik, Saumya and Graf, Victoria and Ivison, Hamish and Huang, Shengyi and Dasigi, Pradeep and Lambert, Nathan and Hajishirzi, Hannaneh}, booktitle={Advances in Neural Information Processing Systems}, year={2025}, url={https://arxiv.org/abs/2507.02833} } ``` ## Changes in this release - Published a cleaned 29,520-row, JSONL-formatted, content-constrained training subset. - Parsed verifier IDs and kwargs into compact nested metadata. - Added an empty `label` field and `if_content_constrained` data type for RL pipelines. - Added a shared instruction discouraging surface-only reward hacking. - Preserved the relative order and byte content of every surviving record. - Added full-file integrity, schema, constraint, task-shape, duplicate, and known-unsatisfiable-record statistics.