rpDungeon/rp-synth-deslopped

--- license: apache-2.0 language: - en tags: - roleplay - creative-writing - synthetic size_categories: - n<1K --- # A Note on the License So the data is synthed from two models that do not have open licenses (Mistral Large 2411 under MRL and Command-A under Cohere). Claude wrote this readme + I havent checked if outputs from these models fall under the Apache license or not so use commercially at your own risk (partly bc the data is probably Not That Good) # RP Synth Deslopped Synthetic roleplay training data with clichéd phrases ("slop") removed. Rather than using a fixed list, I ran n-gram analysis on the original set of synthed data to look for patterns that were common in that dataset specifically to "update". Character cards come from one of the Charcards datasets (probably Opus?), I extracted them from one of the sets a while ago but forget which. User characters were generated by `labs-mistral-small-creative`. Aside from generating the actual turns, I used different Mistral API models to judge/select/rewrite/etc in the pipeline (thanks MistralAI for having such a generous free tier). ## Dataset Details - **Samples**: 295 multi-turn roleplay conversations - **Format**: ShareGPT/conversations format - **Processing**: De-slopped using targeted sentence rewriting (V4 pipeline) - **Slop Reduction**: 88.2% (847 → 100 occurrences) ## Processing Pipeline The V4 de-slop pipeline: 1. Identifies specific sentences containing clichéd phrases 2. Generates rewrite candidates using Mistral Large 3. Selects the best candidate that removes slop without introducing new patterns 4. Only processes assistant turns to preserve user voice ### Patterns Removed Common RP clichés like: - "weight of" (metaphorical) - "deliberate step" - "voice dropping" - "edge of" (metaphorical) - "force of" - And 60+ other patterns ## Usage ```python from datasets import load_dataset ds = load_dataset("rpDungeon/rp-synth-deslopped") ``` ## License Apache 2.0