PaulR11/niah-realism

该数据集是ICLR 2026论文Training Alignment Auditors via Reinforcement Learning的数据发布部分，主要用于针在干草堆(NIAH)现实主义评估。它比较了审计模型生成的合成审计记录与真实WildChat对话的成对对比，通过Opus 4.6法官判断哪个更真实。数据集包含多个子目录和文件，如结果、日志和中间转换输出等。每个运行的JSON模式包含配置、胜率、准确率等信息。数据集仅供研究使用，不包含API密钥或PII。

This dataset is part of the data release for the ICLR 2026 paper Training Alignment Auditors via Reinforcement Learning. It contains full outputs of the Needle-in-a-Haystack realism evaluation (Cell G of the 4-bucket suite). The methodology compares a synthetic audit transcript pairwise against a real WildChat conversation, with an Opus 4.6 judge determining which is more realistic. The dataset includes various subdirectories and files such as results, logs, and intermediate conversions. Each runs JSON schema contains configuration, win rate, accuracy, and other metrics. The dataset is for research use only and contains no API keys or PII.