sonset-schemaset-1k

# Schemaset-1K by Sonset 1,000 preview rows available here. Full dataset on [Gumroad](https://creayo.gumroad.com/l/schemaset). Build datasets like this at https://sonset.dev --- JSON Schema datasets on HuggingFace almost all cover the same ground. Types, required fields, maybe some nested objects. Models trained on them fall apart when they see a `$ref` chain, stacked `if/then` conditionals, or a recursive schema. They start inventing keywords. 68.8% of Schemaset-100K is L4 or L5. That's the stuff that breaks things. --- ## Dataset Three tasks: **create_schema** (75%): natural language in, complete JSON Schema out. Seven vagueness levels distributed roughly evenly. The easy end is a detailed spec with field names spelled out. The hard end is someone who barely knows what they want. **fill_schema** (17%): unstructured text and a schema in, valid JSON instance out. **fill_schema_blind** (8%): same, no schema provided. The model has to figure out the structure. Schemas were sourced from [SchemaStore](https://www.schemastore.org) and the [Passau JSON Schema Corpus](https://github.com/sdbs-uni-p/json-schema-corpus): real CI configs, OpenAPI specs, package manifests, plus synthetic generation seeded from the same sources. Prompts were seeded from Wikipedia titles across 300+ languages. --- ## Difficulty | Level | Coverage | % | |---|---|---| | L1 | Basic types, simple properties | 5.1% | | L2 | Required fields, enums, formats | 2.4% | | L3 | Nested objects, arrays, pattern validation | 23.8% | | L4 | `$ref`, `definitions`, complex constraints | 21.7% | | L5 | Recursive schemas, cross-schema composition, `if/then/else` | 47.1% | --- ## Splits | Split | Rows | Size | |---|---|---| | train | 89,012 | 290 MB | | val | 4,945 | 16 MB | | test | 4,945 | 17 MB | Test split is L3+ only (L3=1,292 / L4=1,157 / L5=2,496). --- ## Variants The dataset is available in both Parquet and in JSONL. Parquet has a _string variant. You may use this for training, it has simply had all of it's objects serialized into strings to make it compatible with the dataset viewer and is intended to be used. The non-suffixed variants do work, just with some extra post-processing. JSON loads differently, and does not need this optimization. It is reccomended to use the _string variant when training in Parquet. --- ## Prompt Robustness Each schema in SchemaSet appears across multiple prompt styles — from terse one-liners ("schema for audio layers") to fully specified requirements with exact field names, constraints, and nesting rules. This is intentional. Training on a single prompt per schema teaches a model what to generate, but training on several phrasings of the same request teaches it to handle how real users actually ask. The full dataset includes seven prompt vagueness levels (vibes_only, casual, idea, moderate, detailed, spec_level, precise) so the model learns to produce correct schemas whether the user is brainstorming or writing a spec. If you need a higher-signal, lower-redundancy version, the paid release on Gumroad includes a deduplicated cut (32,547 rows, one prompt per unique schema) alongside the full 98,902-row multi-prompt set. --- ## Load ```python from datasets import load_dataset ds = load_dataset("sonset/schemaset-1k") ``` --- ## Benchmark We ran a small LoRA fine-tune on Qwen 2.5 3B Instruct, early checkpoint, well short of full training. Compared against the base model on 15 vague prompts. | | Base | LoRA | |---|---|---| | Used `$ref` or `additionalProperties` | 0/15 | 8/15 | | Avg fields per schema | 36 | 62 | | Avg quality score | 6.3/10 | 7.6/10 | The base model produced zero advanced features across all 15 prompts. A few comparisons: **Multiplayer leaderboard.** Base: 17 fields. LoRA: 89 fields, `$ref` Player and Achievement definitions, pattern-validated IDs. **SaaS billing.** Base: 23 flat fields. LoRA: 60 fields, `additionalProperties: false` throughout, UUID patterns, `exclusiveMinimum`. **CMS article.** Base: invalid JSON, invented a keyword. LoRA: 82 fields, `$ref` Author and Comment definitions. **Complex prompt (oneshot):** multi-tenant event sourcing, 8 `$defs`, 4 stacked `allOf/if/then` blocks, recursive self-reference. Base model output had 6 structural bugs: ```json "definitions": { "$defs": { "TenantId": {} } } ``` `$defs` inside `definitions` is invalid. It belongs at root. ```json "if": { "propertyPath": "eventType", "equals": "deleted" } ``` `propertyPath` and `equals` don't exist in JSON Schema. The schema also had four `if` keys at the same level. JSON drops all but the last. Three conditional blocks silently gone. A `$ref` with a typo (`AgregateType`) pointing to nothing. No `allOf`. LoRA output: ```json "$defs": { "TenantId": { "type": "string", "format": "uuid" } }, "allOf": [ { "if": { "properties": { "eventType": { "const": "deleted" } } }, "then": { "required": ["deletedReason"] } }, { "if": { "properties": { "eventType": { "const": "transferred" } } }, "then": { "required": ["fromTenantId", "toTenantId"] } } ] ``` Correct syntax, all four conditionals present, passes validation. Full outputs in [`/benchmark`](https://huggingface.co/datasets/sonset/schemaset-1k/tree/main/benchmark). --- ## License See license for full details. Four tiers. | Tier | Price | Notes | |---|---|---| | Preview (You Are Here) | Free | 1k rows, non-commercial, copyleft | | Research | $99 | Full dataset, non-commercial, open weights | | Indie | $1,499 | Commercial (single product), attribution required | | Commercial | $4,999 | Unlimited commercial, no attribution | Version numbers map roughly to row count: v1.0 = 100k rows, v1.50 = ~150k rows. sdl-v1 covers all releases up to v2.0 which is ~200,000 rows. --- ## Citation ```bibtex @dataset{schemaset100k_2026, title = {Schemaset-100K}, author = {Creayo}, year = {2026}, url = {https://huggingface.co/datasets/sonset/schemaset-1k}, license = {sdl-v1} } ```