SPB-2603

# Dataset Card for Synthetic Persona Bank (Mar 2026) ## Dataset Summary This dataset contains synthetically generated, fictional character personas formatted as structured narrative character cards, designed for LLM role-playing and conversational AI applications. Each persona is a richly detailed XML-structured profile covering physical appearance, personality, emotional patterns, relationships, values, communication style, and example dialogue: these were modelled after the character card guidelines used in platforms such as [SillyTavern](https://github.com/SillyTavern/SillyTavern). The dataset was created programmatically using Large Language Models ([Kimi-K2.5](https://huggingface.co/moonshotai/Kimi-K2.5), [GLM-5](https://huggingface.co/zai-org/GLM-5), and [Qwen3.5-35B-A3B](https://huggingface.co/unsloth/Qwen3.5-35B-A3B) as a lighter local model) guided by a detailed, component-based prompting strategy emphasising psychological realism, show-don't-tell narrative writing, and demographic diversity. This is the March 2026 iteration of the Synthetic Persona Bank series (see [SPB-2602](https://huggingface.co/datasets/marcodsn/SPB-2602) for the previous release). The primary motivation for this iteration was to produce richer, more internally consistent characters suitable as the persona layer in a synthetic conversation dataset. Age ranges are intentionally weighted toward younger adults to match that downstream use case; broader age coverage is planned for future releases. ## Dataset Structure The dataset consists of a single JSONL file (`data.jsonl`). Each line is a JSON object with four top-level fields: `persona_text`, `meta`, `eval_score`, and `eval_results`. ### Data Instances ```json { "persona_text": "haracter>\n**Basic Information**\n**Name:** Jackson\n**Age:** 21\n**Location:** Astoria, Queens, New York\n**Pronouns:** He/Him\n\n**Physical & Lifestyle**\nJackson has an average build and keeps his hair cut short out of convenience rather than style...\n\n**Example Messages**\n<START>\n...\n</character>", "meta": { "model": "zai-org/GLM-5:fireworks-ai", "region": "en_US", "subregion": "New York", "name": "Jackson", "age": 21, "archetype": "grounded_pragmatist", "regulation_style": "stable", "input_tokens": 4517, "output_tokens": 2160, "source_timestamp": "2026-03-08T18:35:52", "id": "e5290b10aa33b13e98dafc7c7007ba82" }, "eval_score": 4.88, "eval_results": { "scores": { "internal_consistency": 5, "archetype_fidelity": 5, "show_dont_tell": 5, "psychological_realism": 5, "roleplay_utility": 5, "distinctiveness": 4, "format_compliance": 5 }, "reasoning": "The persona maintains a coherent narrative where Jackson's habits (resoled sneakers, printed lists) consistently reflect his grounded, pragmatic nature without contradiction...", "judge_model": "unsloth/Qwen3.5-35B-A3B-UD-Q4_K_XL" } } ``` ### Data Fields **`persona_text`** (string): The full character card in XML format (`<character>...</character>`), containing the following narrative sections written in markdown: - **Basic Information** — name, age, location, pronouns - **Physical & Lifestyle** — appearance, daily routines, hobbies, living situation (2–3 paragraphs) - **Personality Overview** — core personality, self-perception, contradictions (2–3 paragraphs) - **Core Traits** — defining characteristics, explained narratively (1–2 paragraphs) - **Emotional Profile** — emotional patterns, regulation strategies, anxiety/joy manifestations (2–3 paragraphs) - **Relationships** — family, friends, romantic, professional dynamics with named people (2–4 paragraphs) - **Values, Motivations & Fears** — core values, drivers, fears (2–3 paragraphs) - **Behavioral Patterns** — adaptive and maladaptive coping strategies (1–2 paragraphs) - **Communication Style** — speech and texting patterns, vocabulary, formality, humor (2 paragraphs) - **Example Messages** — 2–3 short exchanges showing the persona's actual voice, preceded by `<START>` - **Summary** — one-paragraph synthesis of their current life stage **`meta`** (object): - **model** (string): The model used to generate the persona - **region** (string): Locale code used to seed name and demographic sampling (e.g. `en_US`) - **subregion** (string): More specific location within the region - **name** (string): The persona's full name, sampled from locale-specific name lists - **age** (int): The persona's age, sampled with a mean of 25 (σ=5) - **archetype** (string): High-level personality archetype assigned during generation - **regulation_style** (string): Emotional regulation style assigned during generation - **input_tokens** (int): Number of input tokens used for the generation call - **output_tokens** (int): Number of output tokens produced by the model - **source_timestamp** (string): ISO 8601 timestamp extracted from the source generation file name, indicating when that generation batch was run - **id** (string): MD5 hash of the `persona_text` field, used as a unique identifier **`eval_score`** (float): Overall quality rating (1.0–5.0 scale) computed as a weighted mean across 7 evaluation criteria using LLM-as-a-judge assessment. **`eval_results`** (object): Detailed evaluation breakdown containing: - **scores** (object): Per-criterion integer scores (1–5) for `internal_consistency`, `archetype_fidelity`, `show_dont_tell`, `psychological_realism`, `roleplay_utility`, `distinctiveness`, and `format_compliance` - **reasoning** (string): Judge's textual analysis explaining the scores with specific citations from the persona text - **judge_model** (string): Model identifier of the LLM judge used for evaluation ### Data Splits The dataset is provided as a single file constituting the `train` split. Users are encouraged to define their own validation and test splits as needed. ## Dataset Creation ### Curation Rationale The goal of this iteration was to move beyond keyword-label personas toward psychologically grounded character profiles with genuine narrative depth. The resulting format is compatible with character card conventions used in LLM role-playing tools and is intended as the persona layer for an upcoming synthetic conversation dataset. ### Source Data This is a fully synthetic dataset, not derived from any existing corpus of human-written text. #### Generation Process 1. **Demographic Seeding**: Each generation call samples a world region (weighted by population), a locale-appropriate name, and an age (mean 25, σ=5). 2. **Few-Shot Prompting**: Shots are selected from a curated seed set of character cards for the first 10 iterations. Subsequently, a 50% probability is used to draw from generated personas instead, promoting diversity while avoiding drift. The generated pool is periodically reset to prevent echo-chamber effects. 3. **Prompt Rendering**: A Jinja2 template renders a detailed system prompt encoding psychological realism guidelines, show-don't-tell narrative requirements, positive framing rules, and cross-dimensional diversity targets (occupation, personality, life circumstances, mental health, socioeconomic status, life stage). 4. **LLM Generation**: The prompt is sent to the LLM endpoint asynchronously in configurable batch sizes. 5. **Output Parsing**: The `<character>...</character>` XML block is extracted from the model output; records with empty or malformed outputs are discarded. 6. **Collection**: Valid personas are appended to timestamped JSONL files and later merged with source timestamps and unique IDs. 7. **Quality Assessment**: Each persona is evaluated by an LLM judge ([Qwen3.5-35B-A3B](https://huggingface.co/unsloth/Qwen3.5-35B-A3B)) on 7 criteria: internal consistency, archetype fidelity, show-don't-tell execution, psychological realism, roleplay utility, distinctiveness, and format compliance. Scores are combined into a weighted mean (1.0–5.0 scale) with psychological realism and narrative quality weighted most heavily. > [!NOTE] > Region and name sampling is weighted by real-world population distributions, and name > lists are locale-specific (covering 40+ locales), ensuring demographic diversity in the > generated personas. > [!NOTE] > The age distribution is intentionally weighted toward younger adults (mean 25, σ=5) to > support the primary downstream use case of synthetic online conversation generation. This > will be broadened in future releases. ## Quality Metrics The dataset includes automated quality assessment via LLM-as-a-judge evaluation across all 1,073 personas, using [Qwen3.5-35B-A3B](https://huggingface.co/unsloth/Qwen3.5-35B-A3B) as the judge model. Each persona receives scores (1–5) across 7 dimensions: - **Internal Consistency** (weight 1.0): Coherence across traits, backstory, and behavior - **Archetype Fidelity** (weight 1.5): Authentic embodiment of assigned psychological direction - **Show Don't Tell** (weight 1.5): Use of concrete scenes vs. adjective labels - **Psychological Realism** (weight 2.0): Believability as a real person - **Roleplay Utility** (weight 1.0): Usability for conversational AI applications - **Distinctiveness** (weight 1.0): Difference from a generic template persona - **Format Compliance** (weight 0.5): Adherence to required XML structure ### Overall Score Distribution Across all 1,073 evaluated personas, the `eval_score` has a **mean of 4.87** (median 5.0, StdDev 0.23). The distribution is strongly right-skewed: 58.1% of personas receive a perfect 5.0, with only a small tail of lower-quality outputs below 4.0 (< 1%). ### Per-Criterion Scores | Criterion | Mean | Median | StdDev | |---|---|---|---| | `internal_consistency` | 4.94 | 5.0 | 0.33 | | `archetype_fidelity` | 4.98 | 5.0 | 0.18 | | `show_dont_tell` | 4.65 | 5.0 | 0.50 | | `psychological_realism` | 4.91 | 5.0 | 0.32 | | `roleplay_utility` | 4.99 | 5.0 | 0.16 | | `distinctiveness` | 4.74 | 5.0 | 0.47 | | `format_compliance` | 4.95 | 5.0 | 0.30 | The weakest criterion is `show_dont_tell` (mean 4.65), consistent with the known tendency of LLMs to lean on adjective labels rather than concrete scenes. The strongest is `roleplay_utility` (mean 4.99), reflecting how well the format serves its intended application, at least according to our evaluator model. ### Scores by Generation Model The dataset was generated with three models. GLM-5 and Kimi-K2.5 are SOTA open-weight frontier models; Qwen3.5-35B-A3B (35B total / 3B active, MoE) was used as a lighter local alternative. Quality differences between models are reflected in the per-criterion breakdown below. | Model | n | Mean | `show_dont_tell` | `psychological_realism` | `distinctiveness` | |---|---|---|---|---|---| | `zai-org/GLM-5:fireworks-ai` | 505 | 4.97 | 4.91 | 5.00 | 4.95 | | `moonshotai/Kimi-K2.5:fireworks-ai` | 108 | 4.95 | 4.96 | 4.96 | 4.95 | | `unsloth/Qwen3.5-35B-A3B-UD-Q4_K_XL` | 460 | 4.74 | 4.30 | 4.80 | 4.45 | GLM-5 and Kimi-K2.5 produce broadly comparable output quality. Qwen3.5-35B-A3B scores notably lower on `show_dont_tell` and `distinctiveness`, consistent with its reduced capacity relative to the frontier models. ## Known Limitations - **Age Distribution**: Heavily skewed toward 18–35; not suitable as-is for applications requiring broad age coverage. - **Generation Patterns**: Despite few-shot diversity mechanisms, the model may drift toward certain personality archetypes or narrative tropes across many iterations (specifically, we noticed many personas share the same coping mechanisms). - **Profile Length Variance**: Target length is 2,000–3,000 characters, but actual output length varies by model and sampling conditions. - **No Factual Grounding**: Specific places, institutions, and people mentioned in personas may be fictional or confabulated and may be inconsistent with real-world geography or culture. - **Evaluation Subjectivity**: Quality scores reflect the judgment of a single LLM evaluator (Qwen3.5-35B-A3B) and may not align with human preferences across all dimensions. ## Additional Information ### Code and Seed Data The generation scripts, seed personas, and prompt templates can be found on ~~[GitHub](https://github.com/marcodsn/SOC/tree/2603)~~ (TBD, check the [dev branch](https://github.com/marcodsn/SOC/tree/dev) for the time being). ### Licensing Information This dataset is licensed under the [CC BY 4.0 License](https://creativecommons.org/licenses/by/4.0/). The code used to generate the dataset is available under the Apache 2.0 License. ### Citation Information ```bibtex @misc{marcodsn_SPB, title = {Synthetic Persona Bank}, author = {Marco De Santis}, year = {2026}, version = {2603}, url = {https://huggingface.co/datasets/marcodsn/SPB-2603}, } ```