anonymous-submissions/delibsim-bench

delibsim-bench是一个多智能体大语言模型审议模拟的基准测试数据集。它测量了结构化小组审议前后的审议理性指数(DRI)，以及每轮对话的AQuA话语质量评分。数据集涵盖：11种模型设置（单模型和混合模型，包含/不包含推理能力）：GPT-5.1、Gemini-3-Pro-Preview、DeepSeek-V3.2-Exp、Kimi-K2-Thinking、Claude Opus 4.5；12个政策话题（acp, auscj, bep, biobanking_wa, ccps, energy_futures, fnqcj, forestera, fremantle, swiss_health, uppsala_speaks, zukunft）；3种提示条件（control_1, control_2, treatment）；主要网格中每个单元格进行5次迭代（1,980次主要运行）；5项次要研究，涉及小组规模、角色使用、提示设计、轮次数和采样温度的变化（430次额外运行）。总计包含2,410次审议运行、2,410份转录文本、13,168个每轮AQuA评分、1,094,850个个体DRI评分。该数据集为NeurIPS 2026双盲评审进行了匿名化处理，采用CC BY 4.0许可证。A benchmark of multi-agent LLM deliberation simulations. Measures the Deliberative Reason Index (DRI) before and after a structured group deliberation, plus per-turn AQuA discourse-quality scores, across: 11 model setups (single-model and mixed-model, with/without reasoning): GPT-5.1, Gemini-3-Pro-Preview, DeepSeek-V3.2-Exp, Kimi-K2-Thinking, Claude Opus 4.5; 12 policy topics (acp, auscj, bep, biobanking_wa, ccps, energy_futures, fnqcj, forestera, fremantle, swiss_health, uppsala_speaks, zukunft); 3 prompt conditions (control_1, control_2, treatment); 5 iterations per cell in the main grid (1,980 main runs); 5 secondary studies varying group size, persona use, prompt design, number of rounds, and sampling temperature (430 additional runs). → 2,410 deliberation runs | 2,410 transcripts | 13,168 per-turn AQuA scores | 1,094,850 individual DRI ratings. Anonymized for double-blind NeurIPS 2026 review. License: CC BY 4.0.