Nemotron-Pretraining-Legal-v1

# Nemotron-Pretraining-Legal-v1 ## Dataset Description: The [Nemotron-Pretraining-Legal-v1](https://huggingface.co/datasets/nvidia/Nemotron-Pretraining-Legal-v1) dataset is part of the [Nemotron Pretraining Data](https://huggingface.co/collections/nvidia/nemotron-pre-training-datasets) collection of pretraining datasets. Designed for the [NVIDIA Nemotron 3](https://huggingface.co/collections/nvidia/nvidia-nemotron-v3) family of LLMs, this dataset contains a collection of synthetic datasets intended to improve the legal capabilities of LLMs. In one ablation, adding these datasets to Nemotron 3 Nano pretraining boosted a proxy LegalBench average accuracy from 64.6 to 74.7. This dataset is ready for commercial use. ## Dataset Details: For more details, please see the [NVIDIA Nemotron 3 Ultra tech report](https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Ultra-Technical-Report.pdf). ### Subsets The [Nemotron-Pretraining-Legal-v1](https://huggingface.co/datasets/nvidia/Nemotron-Pretraining-Legal-v1) dataset has the following subsets. **Please note that three of the datasets require some special processing to re-create: Nemotron-Pretraining-Legal-Contract-NLI, Nemotron-Pretraining-Legal-ToS-Clause-Understanding, and Nemotron-Pretraining-Legal-ToSDR-QA.** **Datasets extracted from HTML files** - **Nemotron-Pretraining-Legal-California-Code-Of-Regulations**: [California Code of Regulations](https://govt.westlaw.com/calregs/Index), excluding Title 6 and Title 24. - **Nemotron-Pretraining-Legal-NYCourts-Judicial-Ethics-Opinions**: [New York Court Judicial Ethical Opinions](https://www.nycourts.gov/ipjudicialethicsopinions). - **Nemotron-Pretraining-Legal-eCFR**: [Code of Federal Regulations](https://www.ecfr.gov/). **LLM-cleaned datasets** - **Nemotron-Pretraining-Legal-Case-Law-Summary**: 5.4M summaries generated from a filtered version of [Caselaw](https://huggingface.co/datasets/common-pile/caselaw_access_project_filtered) using Qwen3-235B-A22B-Instruct-2507. **Reformatted datasets** - **Nemotron-Pretraining-Legal-CaseHOLD**: We transformed the [CaseHOLD](https://huggingface.co/datasets/casehold/casehold) dataset into a multiple-choice format. - **Nemotron-Pretraining-Legal-Contract-NLI**: For each non-disclosure agreement in the [ContractNLI](https://stanfordnlp.github.io/contract-nli/) dataset, we extracted the annotated hypotheses, answers, and evidence statements and appended them to the source document. **To re-create this dataset, please see the `convert_contract_nli.py` script.** **Synthetic datasets** - **Nemotron-Pretraining-Legal-Definition-Classification**: From Caselaw, we extracted passages containing defining language as positive examples using Qwen3-235B-A22B-Instruct-2507 and randomly selected passages not containing defining language as negative examples. We use these passages to construct questions that classify whether a text from a judicial opinion defines a term. - **Nemotron-Pretraining-Legal-Diversity-Jurisdiction**: This dataset contains questions that ask whether complete diversity exists between plaintiffs and defendants. The questions are generated from templates using random person names sampled from Nemotron Persona, along with states and causes of action sampled from two predefined lists. We also rephrase questions using Qwen3-235B-A22B-Instruct-2507 to increase diversity. - **Nemotron-Pretraining-Legal-Function-Of-Decision**: We randomly sampled paragraphs from Caselaw documents and prompted Qwen3-235B-A22B-Instruct-2507 to classify its function into 7 pre-defined categories (facts, procedural history, issue, rule, analysis, conclusion, decree). We further balanced the number of examples for each category. - **Nemotron-Pretraining-Legal-GlobalCit**: This dataset contains questions related to global nationality laws, converted from the [GLOBALCIT](https://globalcit.eu/databases/globalcit-citizenship-law-dataset) dataset based on its codebook. We rephrased each question into three different version using Qwen3-235B-A22B-Instruct-2507. - **Nemotron-Pretraining-Legal-LegalBench-CUAD-v2**: This dataset contains questions that ask whether a clause is a specific type of clause in a contract defined in the [Contract Understanding Atticus Dataset](https://www.atticusprojectai.org/cuad/). Using Qwen3-235B-A22B-Instruct-2507, we cleaned raw CUAD contracts shorter than 8k tokens, extracted the first qualifying clause of each type from each contract, and generated a negative example from each extracted clause. For some categories (affiliate_license_licensor, affiliate_license_licensee, post_termination_services, exclusivity, effective _date, non_disparagement, unlimited_all_you_can_eat_license) where the identification accuracy is low, we composed longer prompts with detailed instructions based on the labeling handbook to extract qualifying clauses. - **Nemotron-Pretraining-Legal-ToS-Clause-Understanding**: This dataset contains terms of service clause understanding questions. We generated a relevant legal question about each clause from the [TOS Dataset](https://huggingface.co/datasets/CodeHima/TOS_Dataset) using Qwen3-235B-A22B-Instruct-2507. **The clause used to generate each question is replaced with a `<CLAUSE>` placeholder and can be retrieved using the `split` and `index` provided in `metadata`.** - **Nemotron-Pretraining-Legal-ToSDR-QA**: This dataset contains Yes/No questions that address different sections or issues covered by each contract in the [ToSDR Terms of Service Corpus](https://www.kaggle.com/datasets/sonu1607/tosdr-terms-of-service-corpus) using Qwen3-235B-A22B-Instruct-2507. **The source contract is replaced with a `<DOCUMENT>` placeholder and can be retrieved using the `filename` provided in `metadata`.** - **Nemotron-Pretraining-Legal-eCFR-QA**: This dataset contains DiverseQA-like data generated from the Code of Federal Regulations. We generated a variety of question from CFR excerpts and evaluate the correctness of each answer using Qwen3-235B-A22B-Instruct-2507. ### Overview The table below shows the number of tokens and the model used to generate these subsets: | Subset | Tokens (M) | Models | License | Special Instructions | | --- | --- | --- | --- | --- | | **Nemotron-Pretraining-Legal-California-Code-Of-Regulations** | 34.9 | | cc-by-4.0 | | **Nemotron-Pretraining-Legal-Case-Law-Summary** | 4026.9 | Qwen3-235B-A22B-Instruct-2507 | cc-by-4.0 | | **Nemotron-Pretraining-Legal-CaseHOLD** | 29.3 | | cc-by-4.0 | | **Nemotron-Pretraining-Legal-Contract-NLI** | 1.7 | | | Run included script `convert_contract_nli.py` to re-create the dataset | **Nemotron-Pretraining-Legal-Definition-Classification** | 1.4 | Qwen3-235B-A22B-Instruct-2507 | cc-by-4.0 | | **Nemotron-Pretraining-Legal-Diversity-Jurisdiction** | 0.9 | Qwen3-235B-A22B-Instruct-2507 | cc-by-4.0 | | **Nemotron-Pretraining-Legal-eCFR** | 131.5 | | cc-by-4.0 | | **Nemotron-Pretraining-Legal-eCFR-QA** | 593.2 | Qwen3-235B-A22B-Instruct-2507 | cc-by-4.0 | | **Nemotron-Pretraining-Legal-Function-Of-Decision** | 23.8 | Qwen3-235B-A22B-Instruct-2507 | cc-by-4.0 | | **Nemotron-Pretraining-Legal-GlobalCit** | 7.5 | Qwen3-235B-A22B-Instruct-2507 | cc-by-4.0 | | **Nemotron-Pretraining-Legal-LegalBench-CUAD-v2** | 53.4 | Qwen3-235B-A22B-Instruct-2507 | cc-by-4.0 | | **Nemotron-Pretraining-Legal-NYCourts-Judicial-Ethics-Opinions** | 4.6 | | cc-by-4.0 | | **Nemotron-Pretraining-Legal-ToS-Clause-Understanding** | 0.5 | Qwen3-235B-A22B-Instruct-2507 | cc-by-4.0 | Need to replace the `<CLAUSE>` placeholder. See instructions above. | **Nemotron-Pretraining-Legal-ToSDR-QA** | 24.6 | Qwen3-235B-A22B-Instruct-2507 | cc-by-4.0 | Need to replace the `<DOCUMENT>` placeholder. See instructions above. ### Columns The columns are as follows: - **text**: The **primary data field,** containing the content to be used for pretraining. - **license**: The license(s) governing the sample (e.g., ‘cc-by-4.0’). - **metadata**: A dictionary detailing the following: - **category**: Data type (e.g., 'Nemotron-Pretraining-Legal-Contract-NLI', 'Nemotron-Pretraining-Legal-GlobalCit', ...). - **models_used**: Models used to generate the data (e.g., 'Qwen3-235B-A22B-Instruct-2507'). - For Nemotron-Pretraining-Legal-ToS-Clause-Understanding only: - **split** and **index**: Use to select the appropriate clause from [TOS Dataset](https://huggingface.co/datasets/CodeHima/TOS_Dataset) to replace the `<CLAUSE>` placeholder. - For Nemotron-Pretraining-Legal-ToSDR-QA only: - **filename:** Use to select the appropriate file from [ToSDR Terms of Service Corpus](https://www.kaggle.com/datasets/sonu1607/tosdr-terms-of-service-corpus) to replace the `<DOCUMENT>` placeholder. - **uuid**: The unique identifier for this dataset entry. ## Dataset Owner(s): NVIDIA Corporation ## Dataset Creation Date: 05/18/2026 ## Version: [Nemotron-Pretraining-Legal-v1](https://huggingface.co/datasets/nvidia/Nemotron-Pretraining-Legal-v1) Previous Version(s): N/A Relationship to Previous Version(s): N/A ## License/Terms of Use: The **Nemotron-Pretraining-Legal-v1** dataset is licensed under the [Creative Commons Attribution 4.0 International License](https://creativecommons.org/licenses/by/4.0/) (CC-BY-4.0). This dataset contains synthetic data created using the following model: Qwen3-235B-A22B-Instruct-2507 ## Intended Usage: The Nemotron-Pretraining-Legal-v1 Dataset is intended to be used by the community to continue to improve open models. ## Dataset Characterization **Data Collection Method** - Synthetic: Synthetic generation using a large language model (Qwen3-235B-A22B-Instruct-2507). **Labeling Method** - Not Applicable ## Dataset Format Modality: Text Format: Parquet ## Dataset Quantification Record Count: 9.6M samples Measurement of Total Data Storage: 7.0 GB ## Reference(s): If you use our dataset in your research, please cite our [NVIDIA Nemotron 3 Ultra tech report](https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Ultra-Technical-Report.pdf). ## Ethical Considerations: NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal developer teams to ensure this dataset meets requirements for the relevant industry and use case and addresses unforeseen product misuse. Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).