遇见数据集

Normative Conflict LLM Scaling Dataset (v1)

收藏
Zenodo2025-12-07 更新2026-05-26 收录
官方服务:

资源简介:

This dataset accompanies the manuscript “The Affective Thermodynamic Relationship: An Information-Theoretic Scaling Law for Normative-Conflict Collapse in Large Language Models” (and the earlier preprint “A Scaling Law for Normative-Conflict-Induced Failure in Large Language Models”) and provides the full corpus and analysis artifacts used in the normative-conflict scaling experiments. The dataset contains raw JSONL logs of large language model (LLM) responses to synthetic normative-dilemma prompts across five graded levels of normative conflict (C = 1–5). For each scenario, we release multiple sampled generations across model families and decoding configurations (temperature, nucleus p, top-k), together with human annotation labels for (i) normative conflict level, (ii) interpretative collapse, and (iii) ordinal affective severity (Affective Degradation Index, ADI). Derived features used in the thermodynamic and information-theoretic scaling analyses are also included. The repository is organised into four main directories: raw/ Unedited model interaction logs grouped by conflict level and model family. Each file contains the full prompt, system/user messages, and model outputs in JSONL form. analysis/ Analysis-ready CSV and JSON files, including scenario-level consensus labels, the full panel of annotator labels, variance-mapping sweeps, AIC model-comparison tables, and feature matrices used in GLMM and regression analyses. This directory also contains the file analysis/irr_summary_norm_conflict_v1.json, which reports inter-rater reliability statistics for the norm-conflict annotations: pairwise Cohen’s κ and mean κ for collapse labels, raw agreement across three raters, and Krippendorff’s α for the ordinal ADI severity ratings. These values correspond directly to the inter-rater reliability results reported in the main manuscript and Supplementary Materials. code/ A Colab-ready notebook and supporting scripts that reproduce the key figures and tables from the article, including the Kramers-like scaling plots, entropy-based collapse curves, and GLMM outputs. All code is designed to run on the released data without external dependencies on proprietary logs. meta/ Documentation of the analysis pipeline, file manifests, integrity checksums, and licensing information. This folder specifies the exact preprocessing and aggregation steps used to move from raw logs to the analysis-ready tables. All prompts are synthetic and all outputs are model-generated; no human-identifiable or real-world personal data are included. Model families are encoded at the abstract family level (e.g., open-weight vs. frontier models) to enable reproducibility of the statistical findings while respecting upstream deployment and confidentiality constraints. This version (v1.0.1) reflects the complete Phase 1 dataset used for the normative-conflict scaling analyses in the manuscript and its Supplementary Materials, and adds the explicit inter-rater reliability summary (irr_summary_norm_conflict_v1.json) computed from the released annotation panel.

提供机构:
Zenodo
创建时间:
2025-12-07
二维码
社区交流群
二维码
科研交流群
商业服务