open-llm-leaderboard-old/details_adept__persimmon-8b-base

该数据集是在模型 adept/persimmon-8b-base 在 Open LLM Leaderboard 上的评估运行期间自动创建的。它由 61 个配置组成，每个配置对应一个被评估的任务。数据集包含 1 次运行的结果，每次运行在每个配置中表示为特定的分割，分割名称使用运行的时间戳。train 分割始终指向最新的结果。一个额外的配置 results 存储了所有运行的聚合结果，用于计算和显示 Open LLM Leaderboard 上的聚合指标。可以使用 Hugging Face 的 datasets 库加载该数据集，README 中提供了示例。

原始信息汇总

数据集概述

数据集简介

该数据集是在对模型 adept/persimmon-8b-base 进行评估运行期间自动创建的，用于 Open LLM Leaderboard。

数据集组成

数据集包含 61 个配置，每个配置对应一个评估任务。数据集从 1 次运行中创建，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。"train" 分割始终指向最新的结果。

结果配置

一个额外的配置 "results" 存储所有运行的聚合结果，用于计算并在 Open LLM Leaderboard 上显示聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_adept__persimmon-8b-base", "harness_truthfulqa_mc_0", split="train")

最新结果

以下是 2023-10-11T16:30:00.730198 运行的最新结果：

python { "all": { "acc": 0.4373382174928584, "acc_stderr": 0.03537473296886481, "acc_norm": 0.440779620602171, "acc_norm_stderr": 0.03536781150443019, "mc1": 0.22888616891064872, "mc1_stderr": 0.014706994909055027, "mc2": 0.378505315070287, "mc2_stderr": 0.013586954257578736 }, "harness|arc:challenge|25": { "acc": 0.41552901023890787, "acc_stderr": 0.014401366641216384, "acc_norm": 0.4274744027303754, "acc_norm_stderr": 0.014456862944650652 }, "harness|hellaswag|10": { "acc": 0.5203146783509262, "acc_stderr": 0.004985661282998582, "acc_norm": 0.7114120693089027, "acc_norm_stderr": 0.004521798577922143 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.27, "acc_stderr": 0.04461960433384739, "acc_norm": 0.27, "acc_norm_stderr": 0.04461960433384739 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.45925925925925926, "acc_stderr": 0.04304979692464242, "acc_norm": 0.45925925925925926, "acc_norm_stderr": 0.04304979692464242 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.4276315789473684, "acc_stderr": 0.04026097083296559, "acc_norm": 0.4276315789473684, "acc_norm_stderr": 0.04026097083296559 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.5, "acc_stderr": 0.050251890762960605, "acc_norm": 0.5, "acc_norm_stderr": 0.050251890762960605 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.4377358490566038, "acc_stderr": 0.030533338430467512, "acc_norm": 0.4377358490566038, "acc_norm_stderr": 0.030533338430467512 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.5208333333333334, "acc_stderr": 0.041775789507399935, "acc_norm": 0.5208333333333334, "acc_norm_stderr": 0.041775789507399935 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.28, "acc_stderr": 0.04512608598542127, "acc_norm": 0.28, "acc_norm_stderr": 0.04512608598542127 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.35, "acc_stderr": 0.0479372485441102, "acc_norm": 0.35, "acc_norm_stderr": 0.0479372485441102 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.3, "acc_stderr": 0.046056618647183814, "acc_norm": 0.3, "acc_norm_stderr": 0.046056618647183814 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.3988439306358382, "acc_stderr": 0.03733626655383509, "acc_norm": 0.3988439306358382, "acc_norm_stderr": 0.03733626655383509 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.17647058823529413, "acc_stderr": 0.03793281185307809, "acc_norm": 0.17647058823529413, "acc_norm_stderr": 0.03793281185307809 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.57, "acc_stderr": 0.04975698519562428, "acc_norm": 0.57, "acc_norm_stderr": 0.04975698519562428 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.3617021276595745, "acc_stderr": 0.03141082197596241, "acc_norm": 0.3617021276595745, "acc_norm_stderr": 0.03141082197596241 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.34210526315789475, "acc_stderr": 0.04462917535336936, "acc_norm": 0.34210526315789475, "acc_norm_stderr": 0.04462917535336936 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.47586206896551725, "acc_stderr": 0.041618085035015295, "acc_norm": 0.47586206896551725, "acc_norm_stderr": 0.041618085035015295 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.2804232804232804, "acc_stderr": 0.023135287974325642, "acc_norm": 0.2804232804232804, "acc_norm_stderr": 0.023135287974325642 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.40476190476190477, "acc_stderr": 0.04390259265377563, "acc_norm": 0.40476190476190477, "acc_norm_stderr": 0.04390259265377563 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.33, "acc_stderr": 0.047258156262526045, "acc_norm": 0.33, "acc_norm_stderr": 0.047258156262526045 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.4838709677419355, "acc_stderr": 0.028429203176724555, "acc_norm": 0.4838709677419355, "acc_norm_stderr": 0.028429203176724555 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.33004926108374383, "acc_stderr": 0.033085304262282574, "acc_norm": 0.33004926108374383, "acc_norm_stderr": 0.033085304262282574 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.45, "acc_stderr": 0.05, "acc_norm": 0.45, "acc_norm_stderr": 0.05 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.5696969696969697, "acc_stderr": 0.03866225962879077, "acc_norm": 0.5696969696969697, "acc_norm_stderr": 0.03866225962879077 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.5050505050505051, "acc_stderr": 0.035621707606254015, "acc_norm": 0.5050505050505051, "acc_norm_stderr": 0.035621707606254015 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.5181347150259067, "acc_stderr": 0.036060650018329185, "acc_norm": 0.5181347150259067, "acc_norm_stderr": 0.036060650018329185 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.39487179487179486, "acc_stderr": 0.02478431694215638, "acc_norm": 0.39487179487179486, "acc_norm_stderr": 0.02478431694215638 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.2851851851851852, "acc_stderr": 0.027528599210340492, "acc_norm": 0.2851851851851852, "acc_norm_stderr": 0.027528599210340492 }, "