open-llm-leaderboard-old/details_bartowski__internlm2-chat-7b-sft-llama

该数据集是在模型 bartowski/internlm2-chat-7b-sft-llama 在 Open LLM Leaderboard 上的评估运行期间自动创建的。数据集由 63 个配置组成，每个配置对应一个评估任务。它包括 1 次运行的结果，每次运行在每个配置中表示为特定的拆分。train 拆分始终指向最新结果。此外，还有一个 results 配置，存储了所有运行的聚合结果，用于计算和显示 Open LLM Leaderboard 上的聚合指标。README 还提供了如何使用 `datasets` 库中的 `load_dataset` 函数加载运行中的详细信息的示例。还包括了 2024-04-23 运行的最新结果，显示了不同任务的各种准确性指标。

原始信息汇总

数据集概述

该数据集是在对模型 bartowski/internlm2-chat-7b-sft-llama 进行评估运行期间自动创建的。数据集包含 63 个配置，每个配置对应一个评估任务。数据集由 1 次运行创建，每个运行的详细信息可以在每个配置中找到，使用运行的时间戳作为分割名称。"train" 分割始终指向最新的结果。

数据集结构

配置

数据集包含以下配置：

• harness_arc_challenge_25
• harness_gsm8k_5
• harness_hellaswag_10
• harness_hendrycksTest_5

每个配置包含多个数据文件，这些文件分布在不同的分割中，包括特定运行的时间戳分割和最新的分割。

数据文件路径示例

• harness_arc_challenge_25 配置的数据文件路径示例：

  • **/details_harness|arc:challenge|25_2024-04-23T10-18-01.055392.parquet

• harness_gsm8k_5 配置的数据文件路径示例：

  • **/details_harness|gsm8k|5_2024-04-23T10-18-01.055392.parquet

• harness_hellaswag_10 配置的数据文件路径示例：

  • **/details_harness|hellaswag|10_2024-04-23T10-18-01.055392.parquet

• harness_hendrycksTest_5 配置的数据文件路径示例：

  • **/details_harness|hendrycksTest-abstract_algebra|5_2024-04-23T10-18-01.055392.parquet
  • **/details_harness|hendrycksTest-anatomy|5_2024-04-23T10-18-01.055392.parquet
  • ...（其他相关文件路径）

最新结果

以下是来自 2024-04-23T10:18:01.055392 运行的最新结果：

python { "all": { "acc": 0.6379935488333144, "acc_stderr": 0.032435853217767985, "acc_norm": 0.6407670286827761, "acc_norm_stderr": 0.03308905592919796, "mc1": 0.35495716034271724, "mc1_stderr": 0.016750862381375905, "mc2": 0.5094697488583244, "mc2_stderr": 0.015131458674991922 }, "harness|arc:challenge|25": { "acc": 0.5443686006825939, "acc_stderr": 0.014553749939306863, "acc_norm": 0.5716723549488054, "acc_norm_stderr": 0.014460496367599013 }, "harness|hellaswag|10": { "acc": 0.6027683728340968, "acc_stderr": 0.004883246579496663, "acc_norm": 0.8016331408086038, "acc_norm_stderr": 0.003979549459374298 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.31, "acc_stderr": 0.04648231987117316, "acc_norm": 0.31, "acc_norm_stderr": 0.04648231987117316 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.6074074074074074, "acc_stderr": 0.04218506215368879, "acc_norm": 0.6074074074074074, "acc_norm_stderr": 0.04218506215368879 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.6973684210526315, "acc_stderr": 0.037385206761196686, "acc_norm": 0.6973684210526315, "acc_norm_stderr": 0.037385206761196686 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.67, "acc_stderr": 0.047258156262526094, "acc_norm": 0.67, "acc_norm_stderr": 0.047258156262526094 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.7056603773584905, "acc_stderr": 0.028049186315695255, "acc_norm": 0.7056603773584905, "acc_norm_stderr": 0.028049186315695255 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.7361111111111112, "acc_stderr": 0.03685651095897532, "acc_norm": 0.7361111111111112, "acc_norm_stderr": 0.03685651095897532 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.44, "acc_stderr": 0.049888765156985884, "acc_norm": 0.44, "acc_norm_stderr": 0.049888765156985884 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.54, "acc_stderr": 0.05009082659620333, "acc_norm": 0.54, "acc_norm_stderr": 0.05009082659620333 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.29, "acc_stderr": 0.04560480215720684, "acc_norm": 0.29, "acc_norm_stderr": 0.04560480215720684 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.6820809248554913, "acc_stderr": 0.0355068398916558, "acc_norm": 0.6820809248554913, "acc_norm_stderr": 0.0355068398916558 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.47058823529411764, "acc_stderr": 0.049665709039785295, "acc_norm": 0.47058823529411764, "acc_norm_stderr": 0.049665709039785295 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.76, "acc_stderr": 0.04292346959909282, "acc_norm": 0.76, "acc_norm_stderr": 0.04292346959909282 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.5234042553191489, "acc_stderr": 0.03265019475033581, "acc_norm": 0.5234042553191489, "acc_norm_stderr": 0.03265019475033581 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.5, "acc_stderr": 0.047036043419179864, "acc_norm": 0.5, "acc_norm_stderr": 0.047036043419179864 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.6689655172413793, "acc_stderr": 0.03921545312467122, "acc_norm": 0.6689655172413793, "acc_norm_stderr": 0.03921545312467122 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.41798941798941797, "acc_stderr": 0.02540255550326091, "acc_norm": 0.41798941798941797, "acc_norm_stderr": 0.02540255550326091 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.5158730158730159, "acc_stderr": 0.044698818540726076, "acc_norm": 0.5158730158730159, "acc_norm_stderr": 0.044698818540726076 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.34, "acc_stderr": 0.04760952285695235, "acc_norm": 0.34, "acc_norm_stderr": 0.04760952285695235 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.7580645161290323, "acc_stderr": 0.024362599693031083, "acc_norm": 0.7580645161290323, "acc_norm_stderr": 0.024362599693031083 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.46798029556650245, "acc_stderr": 0.035107665979592154, "acc_norm": 0.46798029556650245, "acc_norm_stderr": 0.035107665979592154 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.71, "acc_stderr": 0.045604802157206845, "acc_norm": 0.71, "acc_norm_stderr": 0.045604802157206845 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.7636363636363637, "acc_stderr": 0.03317505930009181, "acc_norm": 0.7636363636363637, "acc_norm_stderr": 0.03317505930009181 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.7828282828282829, "acc_stderr": 0.02937661648494563, "acc_norm": 0.7828282828282829, "acc_norm_stderr": 0.02937661648494563 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.9015544041450777, "acc_stderr": 0.021500249576033453, "acc_norm": 0.9015544041450777, "acc_norm_stderr": 0.021500249576033453 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.6307692307692307, "acc_stderr": 0.024468615241478926, "acc_norm": 0.6307692307692307, "acc_norm_stderr": 0.024468615241478926 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.3111111111111111, "acc_stderr": 0.028226446749683515, "acc_norm": 0.3111111111111111, "acc_norm_stderr": 0.028226446749683515 }, "harness|hendrycksTest-high_school_microeconomics|5": { "acc": 0.6848739495798319, "acc_stderr": 0.03017680828897434, "acc_norm": 0.6848739495798319, "acc_norm_stderr": 0.03017680828897434 }, "harness|hendrycksTest-high_school_physics|5": { "acc": 0.33112582781456956, "acc_stderr": 0.038425817186598696, "acc_norm": 0.33112582781456956, "acc_norm_stderr": 0.038425817186598696 }, "harness|hendrycksTest-high_school_psychology|5": { "acc": 0.8440366972477065, "acc_stderr": 0.015555802713590172, "acc_norm": 0.8440366972477065, "acc_norm_stderr": 0.015555802713590172 }, "harness|hendrycksTest-high_school_statistics|5": { "acc": 0.5462962962962963, "acc_stderr": 0.033953227263757976, "acc_norm": 0.5462962962962963, "acc_norm_stderr": 0.033953227263757976 }, "harness|hendrycksTest-high_school_us_history|5": { "acc": 0.8186274509803921, "acc_stderr": 0.027044621719474082, "acc_norm": 0.8186274509803921, "acc_norm_stderr": 0.027044621719474082 }, "harness|hendrycksTest-high_school_world_history|5": { "acc": 0.810126582278481, "acc_stderr": 0.02553010046023349, "acc_norm": 0.810126582278481, "acc_norm_stderr": 0.02553010046023349 }, "harness|hendrycksTest-human_aging|5": { "acc": 0.7040358744394619, "acc_stderr": 0.030636591348699796, "acc_norm": 0.7040358744394619, "acc_norm_stderr": 0.030636591348699796 }, "harness|hendrycksTest-human_sexuality|5": { "acc": 0.6946564885496184, "acc_stderr": 0.04039314978724561, "acc_norm": 0.6946564885496184, "acc_norm_stderr": 0.04039314978724561 }, "harness|hendrycksTest-international_law|5": { "acc": 0.7355371900826446, "acc_stderr": 0.04026187527591205, "acc_norm": 0.7355371900826446, "acc_norm_stderr": 0.04026187527591205 }, "harness|hendrycksTest-jurisprudence|5": { "acc": 0.7777777777777778, "acc_stderr": 0.0401910747255735, "acc_norm": 0.7777777777777778, "acc_norm_stderr": 0.0401910747255735 }, "harness|hendrycksTest-logical_fallacies|5": { "acc": 0.754601226993865, "acc_stderr": 0.03380939813943354, "acc_norm": 0.754601226993865, "acc_norm_stderr": 0.03380939813943354 }, "harness|hendrycksTest-machine_learning|5": { "acc": 0.5803571428571429, "acc_stderr": 0.04684099321077106, "acc_norm": 0.5803571428571429, "acc_norm_stderr": 0.04684099321077106 }, "harness|hendrycksTest-management|5": { "acc": 0.7572815533980582, "acc_stderr": 0.04245022486384495, "acc_norm": 0.7572815533980582, "acc_norm_stderr": 0.04245022486384495 }, "harness|hendrycksTest-marketing|5": { "acc": 0.8589743589743589, "acc_stderr": 0.022801382534597528, "acc_norm": 0.8589743589743589, "acc_norm_stderr": 0.022801382534597528 }, "harness|hendrycksTest-medical_genetics|5": { "acc": 0.66, "acc_stderr": 0.04760952285695237, "acc_norm": 0.66, "acc_norm_stderr": 0.04760952285695237 }, "harness|hendrycksTest-miscellaneous|5": { "acc": 0.8135376756066411, "acc_stderr": 0.013927751372001505, "acc_norm": 0.8135376756066411, "acc_norm_stderr": 0.013927751372001505 }, "harness|hendrycksTest-moral_disputes|5": { "acc": 0.6907514450867052, "acc_stderr": 0.02488314057007176, "acc_norm": 0.6907514450867052, "acc_norm_stderr": 0.02488314057007176 }, "harness|hendrycksTest-moral_scenarios|5": { "acc": 0.40893854748603353, "acc_stderr": 0.01644283065471554, "acc_norm": 0.40893854748603353, "acc_norm_stderr": 0.01644283065471554 }, "harness|hendrycksTest-nutrition|5": { "acc": 0.7189542483660131, "acc_stderr": 0.025738854797818733, "acc_norm": 0.7189542483660131, "acc_norm_stderr": 0.025738854797818733 }, "harness|hendrycksTest-philosophy|5": { "acc": 0.7041800643086816, "acc_stderr": 0.02592237178881877, "acc_norm": 0.7041800643086816, "acc_norm_stderr": 0.02592237178881877 }, "harness|hendrycksTest-prehistory|5": { "acc": 0.7037037037037037, "acc_stderr": 0.025407197798890165, "acc_norm": 0.7037037037037037, "acc_norm_stderr": 0.025407197798890165 }, "harness|hendrycksTest-professional_accounting|5": { "acc": 0.4787234042553192, "acc_stderr": 0.029800481645628693, "acc_norm": 0.4787234042553192, "acc_norm_stderr": 0.029800481645628693 }, "harness|hendrycksTest-professional_law|5": { "acc": 0.4765319426336376, "acc_stderr": 0.012756161942523372, "acc_norm": 0.4765319426336376, "acc_norm_stderr": 0.012756161942523372 }, "harness|hendrycksTest-professional_medicine|5": { "acc": 0.6507352941176471, "acc_stderr": 0.028959755196824873, "acc_norm": 0.6507352941176471, "acc_norm_stderr": 0.028959755196824873 }, "harness|hendrycksTest-professional_psychology|5": { "acc": 0.6405228758169934, "acc_stderr": 0.01941253924203216, "acc_norm": 0.6405228758169934, "acc_norm_stderr": 0.01941253924203216 }, "harness|hendrycksTest-public_relations|5": { "acc": 0.6545454545454545, "acc_stderr": 0.04554619617541054, "acc_norm": 0.6545454545454545, "acc_norm_stderr": 0.04554619617541054 }, "harness|hendrycksTest-security_studies|5": { "acc": 0.7183673469387755, "acc_stderr": 0.02879518557429129, "acc_norm": 0.7183673469387755, "acc_norm_stderr": 0.02879518557429129 }, "harness|hendrycksTest-sociology|5": { "acc": 0.8059701492537313, "acc_stderr": 0.027962677604768924, "acc_norm": 0.8059701492537313, "acc_norm_stderr": 0.027962677604768924 }, "harness|hendrycksTest-us_foreign_policy|5": { "acc": 0.87, "acc_stderr": 0.03379976689896309, "acc_norm": 0.87, "acc_norm_stderr": 0.03379976689896309 }, "harness|hendrycksTest-virology|5": { "acc": 0.5180722891566265, "acc_stderr": 0.03889951252827216, "acc_norm": 0.5180722891566265, "acc_norm_stderr": 0.03889951252827216 }, "harness|hendrycksTest-world_religions|5": { "acc": 0.8187134502923976, "acc_stderr": 0.029547741687640038, "acc_norm": 0.8187134502923976, "acc_norm_stderr": 0.029547741687640038 }, "harness|truthfulqa:mc|0": { "mc1": 0.35495716034271724, "mc1_stderr": 0.016750862381375905, "mc2": 0.5094697488583244, "mc2_stderr": 0.015131458674991922 }, "harness|winogrande|5": { "acc": 0.7790055248618785, "acc_stderr": 0.01166122363764341 }, "harness|gsm8k|5": { "acc": 0.5595147839272175, "acc_stderr": 0.01367457213169389 } }

数据集加载示例

要加载特定运行的详细信息，可以使用以下代码：

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_bartowski__internlm2-chat-7b-sft-llama", "harness_winogrande_5", split="train")