open-llm-leaderboard-old/details_argilla__notus-8x7b-experiment

该数据集是在模型 argilla/notus-8x7b-experiment 在 Open LLM Leaderboard 上的评估运行期间自动创建的。数据集由 63 个配置组成，每个配置对应一个被评估的任务。它包含 1 次运行的结果，每次运行都作为一个特定的分割表示，分割名称由运行的时间戳命名。train 分割始终指向最新的结果。一个名为 results 的额外配置存储了所有运行的聚合结果，这些结果用于计算和显示 Open LLM Leaderboard 上的聚合指标。README 还提供了如何使用 Python 中的 datasets 库加载运行中的详细信息的示例。

原始信息汇总

数据集概述

该数据集是在评估模型 argilla/notus-8x7b-experiment 的过程中自动创建的，用于 Open LLM Leaderboard。

数据集组成

• 数据集包含 63 个配置，每个配置对应一个评估任务。
• 数据集从 1 次运行中创建，每个运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
• "train" 分割始终指向最新的结果。
• 一个额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_argilla__notus-8x7b-experiment", "harness_winogrande_5", split="train")

最新结果

以下是 2023-12-24T21:16:18.856195 运行的最新结果：

python { "all": { "acc": 0.7124178625456173, "acc_stderr": 0.030199955715548343, "acc_norm": 0.7160635607907738, "acc_norm_stderr": 0.0307822236181654, "mc1": 0.5079559363525091, "mc1_stderr": 0.01750128507455182, "mc2": 0.6579117349463197, "mc2_stderr": 0.015011154188590699 }, "harness|arc:challenge|25": { "acc": 0.6757679180887372, "acc_stderr": 0.013678810399518822, "acc_norm": 0.7098976109215017, "acc_norm_stderr": 0.013261573677520762 }, "harness|hellaswag|10": { "acc": 0.688707428799044, "acc_stderr": 0.004620758579628659, "acc_norm": 0.8773152758414658, "acc_norm_stderr": 0.0032740447231806207 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.43, "acc_stderr": 0.04975698519562429, "acc_norm": 0.43, "acc_norm_stderr": 0.04975698519562429 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.6814814814814815, "acc_stderr": 0.04024778401977108, "acc_norm": 0.6814814814814815, "acc_norm_stderr": 0.04024778401977108 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.7894736842105263, "acc_stderr": 0.03317672787533157, "acc_norm": 0.7894736842105263, "acc_norm_stderr": 0.03317672787533157 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.74, "acc_stderr": 0.0440844002276808, "acc_norm": 0.74, "acc_norm_stderr": 0.0440844002276808 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.7886792452830189, "acc_stderr": 0.025125766484827845, "acc_norm": 0.7886792452830189, "acc_norm_stderr": 0.025125766484827845 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.8194444444444444, "acc_stderr": 0.032166008088022675, "acc_norm": 0.8194444444444444, "acc_norm_stderr": 0.032166008088022675 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.5, "acc_stderr": 0.050251890762960605, "acc_norm": 0.5, "acc_norm_stderr": 0.050251890762960605 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.64, "acc_stderr": 0.04824181513244218, "acc_norm": 0.64, "acc_norm_stderr": 0.04824181513244218 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.45, "acc_stderr": 0.05, "acc_norm": 0.45, "acc_norm_stderr": 0.05 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.7572254335260116, "acc_stderr": 0.0326926380614177, "acc_norm": 0.7572254335260116, "acc_norm_stderr": 0.0326926380614177 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.4117647058823529, "acc_stderr": 0.048971049527263666, "acc_norm": 0.4117647058823529, "acc_norm_stderr": 0.048971049527263666 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.81, "acc_stderr": 0.039427724440366234, "acc_norm": 0.81, "acc_norm_stderr": 0.039427724440366234 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.676595744680851, "acc_stderr": 0.030579442773610334, "acc_norm": 0.676595744680851, "acc_norm_stderr": 0.030579442773610334 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.5964912280701754, "acc_stderr": 0.04615186962583707, "acc_norm": 0.5964912280701754, "acc_norm_stderr": 0.04615186962583707 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.6482758620689655, "acc_stderr": 0.0397923663749741, "acc_norm": 0.6482758620689655, "acc_norm_stderr": 0.0397923663749741 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.48148148148148145, "acc_stderr": 0.025733641991838987, "acc_norm": 0.48148148148148145, "acc_norm_stderr": 0.025733641991838987 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.5079365079365079, "acc_stderr": 0.044715725362943486, "acc_norm": 0.5079365079365079, "acc_norm_stderr": 0.044715725362943486 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.4, "acc_stderr": 0.04923659639173309, "acc_norm": 0.4, "acc_norm_stderr": 0.04923659639173309 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.8516129032258064, "acc_stderr": 0.020222737554330378, "acc_norm": 0.8516129032258064, "acc_norm_stderr": 0.020222737554330378 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.6157635467980296, "acc_stderr": 0.03422398565657551, "acc_norm": 0.6157635467980296, "acc_norm_stderr": 0.03422398565657551 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.79, "acc_stderr": 0.040936018074033256, "acc_norm": 0.79, "acc_norm_stderr": 0.040936018074033256 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.793939393939394, "acc_stderr": 0.03158415324047709, "acc_norm": 0.793939393939394, "acc_norm_stderr": 0.03158415324047709 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.8585858585858586, "acc_stderr": 0.024825909793343336, "acc_norm": 0.8585858585858586, "acc_norm_stderr": 0.024825909793343336 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.9637305699481865, "acc_stderr": 0.013492659751295159, "acc_norm": 0.9637305699481865, "acc_norm_stderr": 0.013492659751295159 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.6923076923076923, "acc_stderr": 0.02340092891831049, "acc_norm": 0.6923076923076923, "acc_norm_stderr": 0.02340092891831049 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.3888888888888889, "acc_stderr": 0.029723278961476664, "acc_norm": 0.3888888888888889, "acc_norm_stderr": 0.029723278961476664 }, "harness|hendrycksTest-high_school_microeconomics|5": {