Nemotron-System-User-Instruction-Hierarchy-v1

## Dataset Description: Nemotron System-User Instruction Hierarchy v1 is a collection of model-generated evaluation rubrics for assessing whether language-model responses follow the instruction hierarchy between system and user messages. It contains three companion subsets: * **conflicting** pairs a source user instruction with a synthetic system instruction that deliberately conflicts with part of the user instruction. The rubric describes how the system instruction takes precedence while preserving compatible user requirements. * **supplementary** pairs a source user instruction with a synthetic, non-conflicting system instruction. The rubric describes how both instructions should be satisfied together. * **user_only** creates a rubric from the source user instruction alone. The `system_prompt` field contains the literal string `N/A`. Each record contains the source prompt context, an analysis of instruction conflicts where applicable, and self-contained evaluation criteria for judging a future model response. The records do not contain the responses being judged. The dataset supports instruction-hierarchy research, rubric-based reinforcement learning, and evaluation with an LLM-as-judge. The system prompts, conflict analyses, and evaluation rubrics were generated using `Qwen/Qwen3.5-27B`. The dataset was used in a proof of concept for GRPO post-training of `nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16` with a rubric-based reward and evaluation on IHEval. This dataset is ready for commercial or non-commercial uses. ## Dataset Owner(s): NVIDIA Corporation Dataset PIC: Liwei Jiang (liweij) ## Dataset Creation Date: Created on: 2026-06-06 Last Modified on: 2026-06-08 ## Versioning: Nemotron System-User Instruction-Hierarchy v1 <br> Previous Version(s): no previous version <br> Relationship to Previous Version(s): Not applicable — this is the initial release. <br> ## License/Terms of Use: This dataset is governed by the [Creative Commons Attribution 4.0 International License (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/). Each user is responsible for checking the content of the dataset and the applicable licenses and determining whether the dataset is suitable for the intended use. ### Source Dataset Attribution Contains prompts derived from [nvidia/Nemotron-RL-Instruction-Following-Adversarial-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Adversarial-v1), made available under the [Open Data Commons Attribution License v1.0 (ODC-By 1.0)](https://opendatacommons.org/licenses/by/1-0/). Includes prompts from [nvidia/Nemotron-SFT-Instruction-Following-Chat-v2](https://huggingface.co/datasets/nvidia/Nemotron-SFT-Instruction-Following-Chat-v2), © NVIDIA Corporation, licensed under the [Creative Commons Attribution 4.0 International License (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/). Includes prompts from [nvidia/Nemotron-Instruction-Following-Chat-v1](https://huggingface.co/datasets/nvidia/Nemotron-Instruction-Following-Chat-v1), © NVIDIA Corporation, licensed under the [Creative Commons Attribution 4.0 International License (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/). The system instructions, conflict analyses, and evaluation rubrics were generated using [Qwen/Qwen3.5-27B](https://huggingface.co/Qwen/Qwen3.5-27B), which is made available under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). ## Intended Usage: This dataset is intended for researchers and developers who train or evaluate language models for instruction-hierarchy adherence. Potential uses include: * training a policy with rubric-based rewards for system/user instruction following; * evaluating whether a response obeys higher-priority system instructions while retaining compatible user requirements; * constructing LLM-as-judge prompts from self-contained evaluation criteria; and * studying differences between conflicting, supplementary, and user-only instruction settings. The rubrics are model-generated, silver-quality supervision. They should not be treated as human ground truth or used as the sole basis for high-impact decisions. Users should validate criteria against the associated instructions and their application requirements. ## Dataset Composition The dataset contains three configurations in the final Hugging Face release. Each configuration uses one JSONL file and one `train` split. | Subset | Definition | Final dataset partition | |--------|------------|--------------------| | conflicting | Rubrics for deliberately conflicting system/user prompt pairs | [`conflicting`](https://huggingface.co/datasets/nvidia/System-User-Instruction-Hierarchy/viewer/conflicting/train) | | supplementary | Rubrics for compatible, supplementary system/user prompt pairs | [`supplementary`](https://huggingface.co/datasets/nvidia/System-User-Instruction-Hierarchy/viewer/supplementary/train) | | user_only | Rubrics for user instructions without a system instruction | [`user_only`](https://huggingface.co/datasets/nvidia/System-User-Instruction-Hierarchy/viewer/user_only/train) | **Data sample definition:** One JSONL record represents one generated instruction-hierarchy evaluation rubric for a source user instruction and, depending on the subset, a conflicting system instruction, a supplementary system instruction, or no system instruction. ## Data Preparation User instructions were sampled from these NVIDIA Nemotron instruction-following datasets: * [nvidia/Nemotron-SFT-Instruction-Following-Chat-v2](https://huggingface.co/datasets/nvidia/Nemotron-SFT-Instruction-Following-Chat-v2) * [nvidia/Nemotron-Instruction-Following-Chat-v1](https://huggingface.co/datasets/nvidia/Nemotron-Instruction-Following-Chat-v1) * [nvidia/Nemotron-RL-Instruction-Following-Adversarial-v1](https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Adversarial-v1) For each selected user instruction, `Qwen/Qwen3.5-27B` generated a conflicting system instruction and a supplementary system instruction. The conflicting and supplementary prompt-pair collections each contain 15,558 candidate pairs and are row-aligned by `id`. A user-only variant was also prepared with no system instruction. `Qwen/Qwen3.5-27B` then generated conflict analyses and evaluation criteria using the `hierarchy_constitution_extraction_with_safety` v6 rubric template and its user-only variant. Generated outputs were parsed as JSON and compiled. Records whose generated output could not be parsed as valid JSON were excluded. The resulting cleanly parsed subsets contain 14,607 conflicting records, 13,782 supplementary records, and 15,414 user-only records. The parsing requirement verifies machine readability but does not establish semantic correctness. Differences between candidate and published counts may introduce selection bias and must be considered when using the data. For the combined Hugging Face release, the dynamically keyed `conflict_analysis` and `evaluation_criteria` maps are normalized into fixed-schema arrays. Map keys are retained in each entry's `id` field. Common fields remain directly accessible, additional fields are serialized in `extra_json`, and non-object values are preserved in `raw_value_json`. This packaging transformation preserves the generated content while making all three configurations loadable through the standard Hugging Face `datasets` library. ## Dataset Characterization **Data Collection Method** * Hybrid: Human, Synthetic. Source user instructions come from existing NVIDIA Nemotron datasets; system instructions, conflict analyses, and rubrics are synthetic model generations. **Labeling Method** * Synthetic. `Qwen/Qwen3.5-27B` generated the conflict analyses and rubric criteria. ## Dataset Format Modality: Text<br> Format: JSONL<br> Structure: Text + Metadata<br> Configuration: One configuration and one split per companion artifact Each JSONL record contains: * `id`: unique rubric-record identifier; * `prompt_pair_id`: join key to the generated prompt-pair data; * `source_dataset` and `source_split`: provenance for the user instruction; * `user_query`: source user instruction; * `system_prompt`: generated system instruction, or `N/A` for `user_only`; * `conflict_analysis`: array of conflict entries containing `id`, `conflict_point`, `resolution`, `extra_json`, and `raw_value_json`; * `evaluation_criteria`: array of rubric entries containing `id`, `source`, `rationale`, `criterion`, `extra_json`, and `raw_value_json`; * `rubric_generator_model`: model used to generate the rubric; and * `rubric_template_path`: rubric-generation template reference. `extra_json` preserves generated fields outside the common schema. `raw_value_json` preserves map values that were not JSON objects. Empty strings indicate that no extra or raw value was present. ## Dataset Quantification Feature count: 10 top-level features<br> Total Data Storage: 251.77 MB | Subset | Samples | Features | Data storage | Rubric criteria | Criteria per sample | |--------|--------:|---------:|-------------:|----------------:|---------------------| | conflicting | 14,607 | 10 | 95.87 MB | 148,786 | median 10; mean 10.19; range 4-28 | | supplementary | 13,782 | 10 | 85.67 MB | 141,931 | median 10; mean 10.30; range 4-29 | | user_only | 15,414 | 10 | 70.23 MB | 102,450 | median 6; mean 6.65; range 2-31 | | **Total** | **43,803** | **10** | **251.77 MB** | **393,167** | — | Storage values report the published `rubrics.jsonl` payloads in decimal megabytes (1 MB = 1,000,000 bytes) and exclude repository metadata and dataset-card files. Source-user-instruction distribution: | Source dataset | conflicting | supplementary | user_only | Total | |----------------|------------:|--------------:|----------:|------:| | Nemotron-SFT-Instruction-Following-Chat-v2 | 8,522 | 8,186 | 9,036 | 25,744 | | Nemotron-Instruction-Following-Chat-v1 | 5,099 | 4,726 | 5,412 | 15,237 | | Nemotron-RL-Instruction-Following-Adversarial-v1 | 986 | 870 | 966 | 2,822 | | **Total** | **14,607** | **13,782** | **15,414** | **43,803** | ## Validation and Quality Controls The published artifacts contain only records whose generated rubric output was parsed successfully as JSON. Record identifiers, prompt-pair identifiers, source metadata, and nested rubric fields are included to support provenance and structural checks. Before final approval, the intake issue should record the results of: * schema and full-file validation; * uniqueness checks for `id` and referential checks for `prompt_pair_id`; * duplicate, null, and malformed nested-object checks; * a stratified human review across all subsets and source datasets; * language-distribution validation; * PII, secret, unsafe-content, and sensitive-source-content review; and * reconciliation of the source, candidate, compiled, and published sample counts. The current release metadata lists English only; however, sample inspection found non-English content, including Russian. The Data PIC should correct the managed language metadata based on full-distribution validation. ## Reference(s): * Generation model: https://huggingface.co/Qwen/Qwen3.5-27B ## Ethical Considerations: The source user instructions may contain unsafe, adversarial, multilingual, copyrighted, personal, or otherwise sensitive content inherited from the source datasets. Synthetic system instructions may add conflicting or restrictive behavior, and generated analyses or criteria may reproduce or amplify errors and biases in the source prompts or generation model. Users should inspect applicable records and source-dataset terms before training, evaluation, or redistribution. The rubrics are generated rather than human-authored. A criterion can misinterpret an instruction, omit a requirement, incorrectly identify a conflict, or encode an unsafe or undesirable resolution. The successful JSON parsing used during compilation does not measure these semantic failure modes. Filtering out unparsable generations may also change the distribution of retained examples. The dataset is designed to evaluate instruction following, including cases where system and user instructions conflict. It should not be used to infer that every system instruction is safe, lawful, or appropriate. Applications remain responsible for independent safety policies and for reviewing the content and provenance of the records they use. NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal developer teams to ensure this dataset meets requirements for the relevant industry and use case and addresses unforeseen product misuse. Please report quality, risk, security vulnerabilities, or NVIDIA AI concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).