narrative-bench

# Narrative Identity Effect on LLM Reasoning ## Description {'model': Value('string'), 'biography_level': Value('int64'), 'persona_type': Value('string'), 'task_type': Value('string'), 'accuracy': Value('bool'), 'prompt': Value('string'), 'completion': Value('string'), 'completion_len': Value('int64'), 'lexical_diversity': Value('float64'), 'mattr': Value('float64'), 'sentiment_valence': Value('float64'), 'sentiment_arousal': Value('float64'), 'self_reference_rate': Value('float64'), 'hedging_density': Value('float64'), 'reasoning_steps': Value('int64'), 'hallucination_indicators': Value('int64'), 'emotion_positive': Value('float64'), 'emotion_negative': Value('float64'), 'emotion_neutral': Value('float64'), 'emotion_joy': Value('float64'), 'emotion_sadness': Value('float64'), 'emotion_anger': Value('float64'), 'emotion_fear': Value('float64'), 'emotion_disgust': Value('float64'), 'emotion_surprise': Value('float64'), 'emotion_trust': Value('float64')} *(metadata above)* A controlled **3 × 4 × 3 factorial experiment** investigating whether injecting narrative biography, persona type, and emotional framing causally changes large language model outputs. - **Models tested**: DeepSeek v4 Flash, Mimo v2.5, MiniMax M2.5 (all via OpenRouter paid inference) - **Conditions**: 3 biography levels (neutral / factual / narrative) × 4 persona types (expert / novice / emotional / control) × 3 task types (GSM8K math, HELM-creative, moral dilemmas) - **N**: 1,528 successful runs (15 per cell, 45 cells) - **Statistical approach**: cluster-robust GLM + Benjamini-Hochberg FDR correction across 10 primary tests ## Key Findings | Claim | Stat | FDR | |-------|------|-----| | Bio richness → self-ref rate (T.1) | β=0.00652, p=0.0072 | p_adj=0.0145 * | | Bio richness → self-ref rate (T.2) | β=0.01254, p<1e-11 | p_adj<0.0001 *** | | Only self-reference survives correction | 10 tests | — | Full results and paper: https://github.com/plntrprotocol/narrative-bench --- *N rows: 1528 | N columns: 26 | Authors: Anduril (Ousia Research)*