open-llm-leaderboard-old/details_maldv__dragonwar-7b-s1

该数据集是在评估maldv/dragonwar-7b-s1模型期间自动创建的，用于Open LLM排行榜。数据集由63个配置组成，每个配置对应一个评估任务。数据集来源于2次运行，每次运行作为一个特定分割，以运行的时间戳命名。train分割始终指向最新结果。额外的results配置存储了运行的聚合结果，用于计算和显示排行榜上的聚合指标。文件还包括了加载数据集详细信息的Python代码，并展示了特定运行的最新结果。

原始信息汇总

数据集概述

该数据集是在评估模型 maldv/dragonwar-7b-s1 的过程中自动创建的，用于 Open LLM Leaderboard 的评估。数据集包含 63 个配置，每个配置对应一个评估任务。

数据集结构

数据集由 2 次运行结果组成，每个运行结果作为一个特定的分片存储在每个配置中，分片名称使用运行的时间戳。"train" 分片始终指向最新的结果。

配置详情

数据集包含以下配置：

• harness_arc_challenge_25

  • 分片：2024_04_03T03_53_11.022110
  • 分片：2024_04_03T03_58_55.962837
  • 分片：latest

• harness_gsm8k_5

  • 分片：2024_04_03T03_53_11.022110
  • 分片：2024_04_03T03_58_55.962837
  • 分片：latest

• harness_hellaswag_10

  • 分片：2024_04_03T03_53_11.022110
  • 分片：2024_04_03T03_58_55.962837
  • 分片：latest

• harness_hendrycksTest_5

  • 分片：2024_04_03T03_53_11.022110
  • 分片：2024_04_03T03_58_55.962837
  • 分片：latest

最新结果

以下是 2024-04-03T03:58:55.962837 运行的最新结果：

python { "all": { "acc": 0.601896582580257, "acc_stderr": 0.03328316753189195, "acc_norm": 0.6085994883559023, "acc_norm_stderr": 0.033974207815405236, "mc1": 0.2692778457772338, "mc1_stderr": 0.015528566637087291, "mc2": 0.4353277421658519, "mc2_stderr": 0.014498475220585973 }, "harness|arc:challenge|25": { "acc": 0.5614334470989761, "acc_stderr": 0.014500682618212865, "acc_norm": 0.5938566552901023, "acc_norm_stderr": 0.014351656690097862 }, "harness|hellaswag|10": { "acc": 0.6408086038637721, "acc_stderr": 0.004787829168255655, "acc_norm": 0.8349930292770364, "acc_norm_stderr": 0.0037042823907817166 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.35, "acc_stderr": 0.0479372485441102, "acc_norm": 0.35, "acc_norm_stderr": 0.0479372485441102 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.5407407407407407, "acc_stderr": 0.04304979692464241, "acc_norm": 0.5407407407407407, "acc_norm_stderr": 0.04304979692464241 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.6578947368421053, "acc_stderr": 0.038607315993160904, "acc_norm": 0.6578947368421053, "acc_norm_stderr": 0.038607315993160904 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.52, "acc_stderr": 0.050211673156867795, "acc_norm": 0.52, "acc_norm_stderr": 0.050211673156867795 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.5886792452830188, "acc_stderr": 0.030285009259009787, "acc_norm": 0.5886792452830188, "acc_norm_stderr": 0.030285009259009787 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.7013888888888888, "acc_stderr": 0.03827052357950756, "acc_norm": 0.7013888888888888, "acc_norm_stderr": 0.03827052357950756 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.48, "acc_stderr": 0.050211673156867795, "acc_norm": 0.48, "acc_norm_stderr": 0.050211673156867795 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.48, "acc_stderr": 0.050211673156867795, "acc_norm": 0.48, "acc_norm_stderr": 0.050211673156867795 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.4, "acc_stderr": 0.049236596391733084, "acc_norm": 0.4, "acc_norm_stderr": 0.049236596391733084 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5895953757225434, "acc_stderr": 0.03750757044895537, "acc_norm": 0.5895953757225434, "acc_norm_stderr": 0.03750757044895537 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.4411764705882353, "acc_stderr": 0.04940635630605659, "acc_norm": 0.4411764705882353, "acc_norm_stderr": 0.04940635630605659 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.75, "acc_stderr": 0.04351941398892446, "acc_norm": 0.75, "acc_norm_stderr": 0.04351941398892446 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.48936170212765956, "acc_stderr": 0.03267862331014063, "acc_norm": 0.48936170212765956, "acc_norm_stderr": 0.03267862331014063 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.4473684210526316, "acc_stderr": 0.04677473004491199, "acc_norm": 0.4473684210526316, "acc_norm_stderr": 0.04677473004491199 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5793103448275863, "acc_stderr": 0.0411391498118926, "acc_norm": 0.5793103448275863, "acc_norm_stderr": 0.0411391498118926 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.3888888888888889, "acc_stderr": 0.025107425481137282, "acc_norm": 0.3888888888888889, "acc_norm_stderr": 0.025107425481137282 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.3968253968253968, "acc_stderr": 0.043758884927270605, "acc_norm": 0.3968253968253968, "acc_norm_stderr": 0.043758884927270605 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.36, "acc_stderr": 0.04824181513244218, "acc_norm": 0.36, "acc_norm_stderr": 0.04824181513244218 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.6741935483870968, "acc_stderr": 0.026662010578567104, "acc_norm": 0.6741935483870968, "acc_norm_stderr": 0.026662010578567104 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.5073891625615764, "acc_stderr": 0.035176035403610105, "acc_norm": 0.5073891625615764, "acc_norm_stderr": 0.035176035403610105 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.69, "acc_stderr": 0.04648231987117316, "acc_norm": 0.69, "acc_norm_stderr": 0.04648231987117316 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.7757575757575758, "acc_stderr": 0.03256866661681102, "acc_norm": 0.7757575757575758, "acc_norm_stderr": 0.03256866661681102 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.7323232323232324, "acc_stderr": 0.03154449888270286, "acc_norm": 0.7323232323232324, "acc_norm_stderr": 0.03154449888270286 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.8186528497409327, "acc_stderr": 0.02780703236068609, "acc_norm": 0.8186528497409327, "acc_norm_stderr": 0.02780703236068609 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.5948717948717949, "acc_stderr": 0.024890471769938145, "acc_norm": 0.5948717948717949, "acc_norm_stderr": 0.024890471769938145