open-llm-leaderboard-old/details_aboros98__form1

该数据集是在Open LLM Leaderboard上对模型aboros98/form1进行评估时自动创建的。数据集由63个配置组成，每个配置对应一个评估任务。数据集是从1次运行中创建的，每次运行在每个配置中表示为特定的分割，分割名称使用运行的时间戳。train分割始终指向最新的结果。一个名为results的额外配置存储了运行的所有聚合结果，用于计算和显示Open LLM Leaderboard上的聚合指标。README还包含了如何加载运行中的详细信息的说明，并提供了特定运行的最新结果。

原始信息汇总

数据集概述

该数据集是在对模型 aboros98/form1 进行评估运行期间自动创建的，用于 Open LLM Leaderboard。数据集包含 63 个配置，每个配置对应一个评估任务。数据集从 1 次运行中创建，每次运行的详细结果存储在特定的分割中，分割名称使用运行的时间戳。"train" 分割始终指向最新的结果。

数据集结构

配置

• harness_arc_challenge_25

  • 分割: 2024_03_24T15_18_06.796680, latest
  • 路径: **/details_harness|arc:challenge|25_2024-03-24T15-18-06.796680.parquet

• harness_gsm8k_5

  • 分割: 2024_03_24T15_18_06.796680, latest
  • 路径: **/details_harness|gsm8k|5_2024-03-24T15-18-06.796680.parquet

• harness_hellaswag_10

  • 分割: 2024_03_24T15_18_06.796680, latest
  • 路径: **/details_harness|hellaswag|10_2024-03-24T15-18-06.796680.parquet

• harness_hendrycksTest_5

  • 分割: 2024_03_24T15_18_06.796680, latest
  • 路径:
    • **/details_harness|hendrycksTest-abstract_algebra|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-anatomy|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-astronomy|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-business_ethics|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-clinical_knowledge|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-college_biology|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-college_chemistry|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-college_computer_science|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-college_mathematics|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-college_medicine|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-college_physics|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-computer_security|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-conceptual_physics|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-econometrics|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-electrical_engineering|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-elementary_mathematics|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-formal_logic|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-global_facts|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-high_school_biology|5_2024-03-24T15-18-06.796680.parquet
    • **/details_harness|hendrycksTest-high_school_chemistry|5_2024-03-24T15-18-06.796680.parquet

最新结果

以下是 2024-03-24T15:18:06.796680 运行 的最新结果：

python { "all": { "acc": 0.5702336183155093, "acc_stderr": 0.03387894122336087, "acc_norm": 0.5717312036488028, "acc_norm_stderr": 0.03457555945885344, "mc1": 0.3219094247246022, "mc1_stderr": 0.016355567611960404, "mc2": 0.4585327362411011, "mc2_stderr": 0.015280976563441748 }, "harness|arc:challenge|25": { "acc": 0.5733788395904437, "acc_stderr": 0.014453185592920293, "acc_norm": 0.5878839590443686, "acc_norm_stderr": 0.014383915302225405 }, "harness|hellaswag|10": { "acc": 0.5727942640908186, "acc_stderr": 0.004936616428922639, "acc_norm": 0.7525393347938658, "acc_norm_stderr": 0.004306547156331402 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.26, "acc_stderr": 0.044084400227680794, "acc_norm": 0.26, "acc_norm_stderr": 0.044084400227680794 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.4074074074074074, "acc_stderr": 0.04244633238353228, "acc_norm": 0.4074074074074074, "acc_norm_stderr": 0.04244633238353228 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.5394736842105263, "acc_stderr": 0.04056242252249035, "acc_norm": 0.5394736842105263, "acc_norm_stderr": 0.04056242252249035 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.54, "acc_stderr": 0.05009082659620332, "acc_norm": 0.54, "acc_norm_stderr": 0.05009082659620332 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.6113207547169811, "acc_stderr": 0.030000485448675986, "acc_norm": 0.6113207547169811, "acc_norm_stderr": 0.030000485448675986 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.6041666666666666, "acc_stderr": 0.04089465449325582, "acc_norm": 0.6041666666666666, "acc_norm_stderr": 0.04089465449325582 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.46, "acc_stderr": 0.05009082659620333, "acc_norm": 0.46, "acc_norm_stderr": 0.05009082659620333 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.39, "acc_stderr": 0.04902071300001975, "acc_norm": 0.39, "acc_norm_stderr": 0.04902071300001975 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.38, "acc_stderr": 0.04878317312145632, "acc_norm": 0.38, "acc_norm_stderr": 0.04878317312145632 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5664739884393064, "acc_stderr": 0.03778621079092056, "acc_norm": 0.5664739884393064, "acc_norm_stderr": 0.03778621079092056 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.3333333333333333, "acc_stderr": 0.04690650298201942, "acc_norm": 0.3333333333333333, "acc_norm_stderr": 0.04690650298201942 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.68, "acc_stderr": 0.04688261722621505, "acc_norm": 0.68, "acc_norm_stderr": 0.04688261722621505 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.502127659574468, "acc_stderr": 0.03268572658667492, "acc_norm": 0.502127659574468, "acc_norm_stderr": 0.03268572658667492 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.35964912280701755, "acc_stderr": 0.045144961328736334, "acc_norm": 0.35964912280701755, "acc_norm_stderr": 0.045144961328736334 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5448275862068965, "acc_stderr": 0.04149886942192118, "acc_norm": 0.5448275862068965, "acc_norm_stderr": 0.04149886942192118 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.4126984126984127, "acc_stderr": 0.025355741263055273, "acc_norm": 0.4126984126984127, "acc_norm_stderr": 0.025355741263055273 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.373015873015873,