us-patents

The us-patents dataset is a collection of ~ 8M US patent grants and applications from 1976-2025, cleaned, filtered, and formatted for pre-training of language models. ### Document Format - `corpus_id`: Unique integer key with no semantic value. - `filing_date`: The filing date of the grant or application. In case of duplicates, earliest filing date from the duplicate cluster. - `patent_type`: The type of patent. - `text`: The text content of the concatenated title, abstract, and specification. ### Processing The following pre-processing steps were applied with the goal of preserving science and technology information. 1. Deduplication The dataset includes a single record for a (grant, application) pair, and include applications which are not yet issued or abandoned. We further deduplicate based on document contents at the document level on 5-gram 90% Jaccard similarity. 2. Legal Process Content Removal Some common content relating to legal processes is removed. This includes standard language about embodiments not being exhaustively listed and other common legal language not directly related to the technology at issue. 3. English Content with a very low probability of being English prose is removed. 4. Unlikely Prose Paragraphs with a very low average word log probability is removed. All sections that have a average log word probability of less than `-20` are removed. To calculate the average log word probability, we use word frequencies extracted from the [1T Web Ngram corpus](https://catalog.ldc.upenn.edu/LDC2006T13); specifically, we use the list available [created by Rachel Tatman](https://www.kaggle.com/datasets/rtatman/english-word-frequency). A copy is hosted [here](https://ai2-s2-research-public.s3-us-west-2.amazonaws.com/lucas/google-1T-unigram/unigram_freq.csv). 5. Low Information Text Paragraphs that are extremely compressible, such as long lists of enumerated trivial embodiment combinations are removed. 6. Copyright Assertions Patent specifications that include variants of the USPTO standard assertion of copyrights relating to portions of the specification are omitted. 7. Design Patents Removed Grants and applications with a design patent type are omitted. ## Stats | Split | Documents | Date Range | |-------|-----------|--------------------------| | Train | 7,820,247 | 1976-01-02 to 2025-01-13 | | Valid | 79,836 | 2025-01-14 to 2025-09-19 | ## License This dataset is licensed under ODC-BY. It is intended for research and educational use in accordance with [Ai2's Responsible Use Guidelines](https://allenai.org/responsible-use). Attribution: PatentsView (www.patentsview.org)