open-llm-leaderboard-old/details_grimjim__Mistral-Starling-merge-trial1-7B

该数据集是在模型 grimjim/Mistral-Starling-merge-trial1-7B 在 Open LLM Leaderboard 上进行评估运行时自动创建的。数据集由 63 个配置组成，每个配置对应一个被评估的任务。数据集是从 1 次运行中创建的，每次运行在每个配置中作为一个特定的分割找到。train 分割始终指向最新的结果。一个额外的配置 results 存储了所有运行的聚合结果。README 还提供了一个如何使用 datasets 库加载运行细节的示例。

原始信息汇总

数据集概述

该数据集是在对模型 grimjim/Mistral-Starling-merge-trial1-7B 进行评估运行期间自动创建的。数据集包含 63 个配置，每个配置对应一个评估任务。数据集从 1 次运行中创建，每次运行的详细信息可以在每个配置中找到，使用运行的时间戳作为分割名称。"train" 分割始终指向最新的结果。

数据集结构

数据集包含以下配置：

• harness_arc_challenge_25

  • 分割：2024_03_29T18_33_08.521257 和 latest
  • 文件路径：**/details_harness|arc:challenge|25_2024-03-29T18-33-08.521257.parquet

• harness_gsm8k_5

  • 分割：2024_03_29T18_33_08.521257 和 latest
  • 文件路径：**/details_harness|gsm8k|5_2024-03-29T18-33-08.521257.parquet

• harness_hellaswag_10

  • 分割：2024_03_29T18_33_08.521257 和 latest
  • 文件路径：**/details_harness|hellaswag|10_2024-03-29T18-33-08.521257.parquet

• harness_hendrycksTest_5

  • 分割：2024_03_29T18_33_08.521257 和 latest
  • 文件路径：多个文件，包括但不限于：
    • **/details_harness|hendrycksTest-abstract_algebra|5_2024-03-29T18-33-08.521257.parquet
    • **/details_harness|hendrycksTest-anatomy|5_2024-03-29T18-33-08.521257.parquet
    • **/details_harness|hendrycksTest-astronomy|5_2024-03-29T18-33-08.521257.parquet
    • 等等。

最新结果

以下是来自 2024-03-29T18:33:08.521257 运行的最新结果：

python { "all": { "acc": 0.6422078400145433, "acc_stderr": 0.03224453953291506, "acc_norm": 0.6450704597100282, "acc_norm_stderr": 0.032889854692922056, "mc1": 0.3671970624235006, "mc1_stderr": 0.01687480500145318, "mc2": 0.5317839659773655, "mc2_stderr": 0.015260448918448506 }, "harness|arc:challenge|25": { "acc": 0.6092150170648464, "acc_stderr": 0.01425856388051378, "acc_norm": 0.6612627986348123, "acc_norm_stderr": 0.01383056892797433 }, "harness|hellaswag|10": { "acc": 0.6459868552081258, "acc_stderr": 0.004772358395130456, "acc_norm": 0.8467436765584545, "acc_norm_stderr": 0.003594981823319896 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.28, "acc_stderr": 0.04512608598542129, "acc_norm": 0.28, "acc_norm_stderr": 0.04512608598542129 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.6148148148148148, "acc_stderr": 0.04203921040156279, "acc_norm": 0.6148148148148148, "acc_norm_stderr": 0.04203921040156279 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.7236842105263158, "acc_stderr": 0.03639057569952929, "acc_norm": 0.7236842105263158, "acc_norm_stderr": 0.03639057569952929 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.6, "acc_stderr": 0.049236596391733084, "acc_norm": 0.6, "acc_norm_stderr": 0.049236596391733084 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.7056603773584905, "acc_stderr": 0.02804918631569525, "acc_norm": 0.7056603773584905, "acc_norm_stderr": 0.02804918631569525 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.7708333333333334, "acc_stderr": 0.03514697467862388, "acc_norm": 0.7708333333333334, "acc_norm_stderr": 0.03514697467862388 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.47, "acc_stderr": 0.050161355804659205, "acc_norm": 0.47, "acc_norm_stderr": 0.050161355804659205 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.54, "acc_stderr": 0.05009082659620332, "acc_norm": 0.54, "acc_norm_stderr": 0.05009082659620332 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.35, "acc_stderr": 0.047937248544110196, "acc_norm": 0.35, "acc_norm_stderr": 0.047937248544110196 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.6184971098265896, "acc_stderr": 0.03703851193099521, "acc_norm": 0.6184971098265896, "acc_norm_stderr": 0.03703851193099521 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.38235294117647056, "acc_stderr": 0.04835503696107223, "acc_norm": 0.38235294117647056, "acc_norm_stderr": 0.04835503696107223 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.75, "acc_stderr": 0.04351941398892446, "acc_norm": 0.75, "acc_norm_stderr": 0.04351941398892446 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.5446808510638298, "acc_stderr": 0.032555253593403555, "acc_norm": 0.5446808510638298, "acc_norm_stderr": 0.032555253593403555 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.43859649122807015, "acc_stderr": 0.04668000738510455, "acc_norm": 0.43859649122807015, "acc_norm_stderr": 0.04668000738510455 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5724137931034483, "acc_stderr": 0.041227371113703316, "acc_norm": 0.5724137931034483, "acc_norm_stderr": 0.041227371113703316 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.4576719576719577, "acc_stderr": 0.025658868862058325, "acc_norm": 0.4576719576719577, "acc_norm_stderr": 0.025658868862058325 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.46825396825396826, "acc_stderr": 0.04463112720677172, "acc_norm": 0.46825396825396826, "acc_norm_stderr": 0.04463112720677172 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.37, "acc_stderr": 0.04852365870939098, "acc_norm": 0.37, "acc_norm_stderr": 0.04852365870939098 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.7451612903225806, "acc_stderr": 0.024790118459332208, "acc_norm": 0.7451612903225806, "acc_norm_stderr": 0.024790118459332208 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.49261083743842365, "acc_stderr": 0.035176035403610084, "acc_norm": 0.49261083743842365, "acc_norm_stderr": 0.035176035403610084 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.66, "acc_stderr": 0.04760952285695237, "acc_norm": 0.66, "acc_norm_stderr": 0.04760952285695237 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.8121212121212121, "acc_stderr": 0.03050193405942914, "acc_norm": 0.8121212121212121, "acc_norm_stderr": 0.03050193405942914 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.7777777777777778, "acc_stderr": 0.02962022787479049, "acc_norm": 0.7777777777777778, "acc_norm_stderr": 0.02962022787479049 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.8808290155440415, "acc_stderr": 0.02338193534812144, "acc_norm": 0.8808290155