moss-local-voice-acting-v4-emorant-64x200

# MOSS-Local Voice Acting v4 "emorant" — maximally emotional deliveries, 64 takes per group **510,024 synthetic voice-acting clips** (48 kHz mono FLAC): **40 emotions × 200 groups × 64 takes**, generated with the merged 4.55 B local-transformer MOSS-TTS voice-acting model ([`laion/moss-tts-local-transformer-4.55b-voice-acting`](https://huggingface.co/laion/moss-tts-local-transformer-4.55b-voice-acting)) — raw model output, no enhancement, temperature 1.0, **no reference audio**: the model invents each group's voice from a persona description of an ordinary human speaker. 🎧 **Listen:** [random-take preview grid](https://projects.laion.ai/laion-moss-local-1.5-voice-acting-4.55b/v4_preview_grid.html) — honest random samples (not cherry-picked) from the first finished emotions, with prompts and scores. ## What v4 is for: intensity v2/v3 asked for emotions politely; v4 *pushes*. The instructions were rewritten from a three-round evolutionary prompting study on character/emotion archetypes ([round 2 — emotion-aware rewards](https://projects.laion.ai/laion-moss-local-1.5-voice-acting-4.55b/archetypes_round2.html)), whose main lessons are baked into every cell: - **delivery classes** — each emotion maps to a class with its own voice-mechanics language: - *RANT* (anger, contempt, malice, jealousy, bitterness, impatience...): "mid-explosion in an unhinged furious rant... voice shredding and cracking with seething rage, spitting venom, escalating louder and more vicious with every line" - *DISTRESS* (fear, pain, helplessness...): trembling, hyperventilating, breaking - *GRIEF* (sadness, disappointment, longing, shame...): sobbing, hollow, catching - *JOY* (elation, amusement, triumph, relief...): whooping, breathless, bursting - *TENDER* (affection, thankfulness, contentment...): hushed, warm, trembling with feeling - *WONDER* (awe, astonishment, interest, contemplation...): hushed astonishment, gasping - *SPECIAL* for intoxication / teasing / ecstasy - **visceral emotion naming** — feelings are named directly and physically, never abstractly; - **escalation arcs** — "more X with every line" phrasing, which measurably amplifies emotion scores; - **parenthetical vocal-burst cues** inside the script (gasps, sobs, cackles, moans...); - **intelligibility guard** — every instruction ends with a variant of "...but every word still lands", keeping WER usable as a filter (check `wer` per take). Personas are **ordinary humans** (no fantasy creatures), scripts are fresh single prompts in English never used in v2/v3, everything lowercase. ## Repo layout ``` data/<Emotion>.tar 40 tars, one per emotion (~14 GB each): <Emotion>/<Emotion>_<group>_v<take>.flac 64 takes x 200 groups, 48 kHz mono FLAC <Emotion>/scores.parquet per-emotion slice of the score table <Emotion>/cells.jsonl per-emotion generation cells scores.parquet all 510,024 rows in one file (recommended entry point) cells.jsonl all 8,000 cells: gid, text, instruction, delivery class, vocal burst ``` ## Score table (`scores.parquet`, one row per take) | column | description | |---|---| | `gid`, `emotion`, `ei_head`, `prompt_idx`, `seed` | group id, target emotion, Empathic-Insight head used as `target`, prompt index, take index (matches `_v<take>` in the filename) | | `vocal_burst` | the non-verbal burst cue named in the script | | `wer`, `inv_wer`, `hyp` | word error rate vs. the script (NVIDIA Parakeet-TDT-0.6b-v3 ASR), `1/(1+WER)`, and the ASR transcript | | `blend`, `genu` | VoiceCLAP-commercial emotion-blend and genuineness MLP scores | | `prompt_sim` | VoiceCLAP audio–text cosine between the take and its style instruction | | `ei_*` (42 columns) | Empathic-Insight-Voice-Plus scores: 40 emotion dimensions plus `ei_Arousal`, `ei_Valence` (and `ei_Authenticity`) | | `target` | the EI-Plus score of the emotion this group was prompted for | | `rms_db`, `peak_db`, `dur` | loudness (dBFS) and duration in seconds | | `finished` | `True` if generation ended with EOS (recommended filter) | ## Best-of-N selection recipe ```python import pandas as pd df = pd.read_parquet("scores.parquet") n = lambda s: (s - s.min()) / (s.max() - s.min()) df["reward"] = (n(df["blend"]) + n(df["genu"]) + 1.25 * n(df["target"])) * df["inv_wer"] best3 = df[df.finished].sort_values("reward", ascending=False).groupby("gid").head(3) ``` The `1.25 × target` term rewards hitting the *prompted* emotion, which matters more in v4 than in v2/v3 because deliveries are pushed so hard that off-target intensity is a real failure mode. ## How it was generated - **Model**: merged (base + rank-256 LoRA) 4.55 B `moss_tts_local` architecture, 12-codebook RVQ, [`MOSS-Audio-Tokenizer-v2`](https://huggingface.co/OpenMOSS-Team/MOSS-Audio-Tokenizer-v2) codec, native 48 kHz. - **Sampling**: temperature 1.0, one batched `generate()` per group (batch = 64), bf16 + SDPA attention. - **Pacing**: token budget = `words × 6.0` codec frames at 12.5 Hz; `max_new_tokens = 2.2 × budget + 300`. - **Scoring**: fused into generation — Parakeet ASR (bf16, batch 32), VoiceCLAP blend/genuineness/ prompt-sim, Empathic-Insight-Voice-Plus 42-dim vector, RMS/peak loudness. - **Compute**: 8 × A100-80GB, ≈ 2.0 s per clip end-to-end (generation + full scoring), ~36 h wall clock. Inference code, fast-inference guide and SGLang serving notes: [LAION-AI/laion-moss-local-1.5-voice-acting-4.55b](https://github.com/LAION-AI/laion-moss-local-1.5-voice-acting-4.55b). ## Sibling releases | corpus | what differs | where | |---|---|---| | **v2** (255,746 clips) | single-prompt scripts, moderate emotional prompting | [`laion/moss-local-voice-acting-64x100`](https://huggingface.co/datasets/laion/moss-local-voice-acting-64x100) | | **v3** (255,272 clips) | unique voice-clone reference per group (4,000 speakers, shipped in-repo), double-length paired scripts | [`laion/moss-local-voice-acting-v3-refs-64x100`](https://huggingface.co/datasets/laion/moss-local-voice-acting-v3-refs-64x100) | | **v4 (this repo)** (510,024 clips) | maximum-intensity delivery-class prompting, 200 groups/emotion | here | | **DramaBox reinterpretations** (188,125 clips) | 2,953 full human performances re-performed ×64, cloned from the original | [`laion/moss-local-dramabox-full-reinterpretations-64`](https://huggingface.co/datasets/laion/moss-local-dramabox-full-reinterpretations-64) | ## Known limitations - Extreme deliveries (screaming, sobbing, slurring) intentionally trade some articulation for intensity — use `wer`/`finished` to filter to your tolerance. - Scripts are theatrical monologues, not conversational speech. - All scores are model-based — rankable signals, not human ground truth. ## License **Apache-2.0.** The audio is fully synthetic; no real speaker's voice is reproduced.