bethgelab/lm-similarity

这是一个用于研究论文《Great Models Think Alike and this Undermines AI Oversight》的数据集。包含三个数据文件：judge_scores_mmlu_pro_free_filtered记录了九位评委在没有参考答案的情况下对经过筛选的开放式MMLU-Pro数据集的评分；judge_w_gt_mmlu_pro_free_filtered记录了五位评委在可以访问参考选项和真实信息的情况下对经过筛选的OSQ MMLU-Pro的集体评分；filter_mmlu_pro记录了在不需要访问参考选项的情况下可以回答的MMLU-Pro样本的每个样本的决策（粗略和精细）和索引。This is the dataset for the research paper Great Models Think Alike and this Undermines AI Oversight. It includes three data files: judge_scores_mmlu_pro_free_filtered contains the scores of nine judges without access to reference answers on the filtered open-style MMLU-Pro dataset; judge_w_gt_mmlu_pro_free_filtered contains the ensemble scores of five judges with access to reference options and ground-truth information on the filtered OSQ MMLU-Pro; filter_mmlu_pro contains per-sample decisions (coarse and fine) and indices of samples in MMLU-Pro that can be answered without access to reference options.