open-llm-leaderboard-old/details_jsfs11__WildMBXMarconi-SLERP-7B

该数据集是在模型 jsfs11/WildMBXMarconi-SLERP-7B 在 Open LLM Leaderboard 上的评估运行期间自动创建的。数据集由 63 个配置组成，每个配置对应一个评估任务。它包含 1 次运行的结果，每次运行都作为每个配置中的一个特定分割存储。train 分割始终指向最新的结果。一个名为 results 的额外配置存储了所有运行的聚合结果，这些结果用于计算和显示 Open LLM Leaderboard 上的聚合指标。README 还提供了如何使用 `datasets` 库中的 `load_dataset` 函数加载数据集的示例。

原始信息汇总

数据集概述

该数据集是在对模型 jsfs11/WildMBXMarconi-SLERP-7B 进行评估运行期间自动创建的，用于 Open LLM Leaderboard。数据集包含 63 个配置，每个配置对应一个评估任务。数据集从 1 次运行中创建，每个运行的详细信息可以在每个配置中找到，使用运行的时间戳作为分割名称。"train" 分割始终指向最新的结果。

数据集结构

配置列表

• harness_arc_challenge_25

  • 分割: 2024_01_24T03_57_13.465418
  • 路径: **/details_harness|arc:challenge|25_2024-01-24T03-57-13.465418.parquet
  • 分割: latest
  • 路径: **/details_harness|arc:challenge|25_2024-01-24T03-57-13.465418.parquet

• harness_gsm8k_5

  • 分割: 2024_01_24T03_57_13.465418
  • 路径: **/details_harness|gsm8k|5_2024-01-24T03-57-13.465418.parquet
  • 分割: latest
  • 路径: **/details_harness|gsm8k|5_2024-01-24T03-57-13.465418.parquet

• harness_hellaswag_10

  • 分割: 2024_01_24T03_57_13.465418
  • 路径: **/details_harness|hellaswag|10_2024-01-24T03-57-13.465418.parquet
  • 分割: latest
  • 路径: **/details_harness|hellaswag|10_2024-01-24T03-57-13.465418.parquet

• harness_hendrycksTest_5

  • 分割: 2024_01_24T03_57_13.465418
  • 路径:
    • **/details_harness|hendrycksTest-abstract_algebra|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-anatomy|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-astronomy|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-business_ethics|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-clinical_knowledge|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-college_biology|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-college_chemistry|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-college_computer_science|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-college_mathematics|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-college_medicine|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-college_physics|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-computer_security|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-conceptual_physics|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-econometrics|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-electrical_engineering|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-elementary_mathematics|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-formal_logic|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-global_facts|5_2024-01-24T03-57-13.465418.parquet
    • **/details_harness|hendrycksTest-high_school_biology|5_2024-01-24T03-57-13.465418.parquet
    • ...

最新结果

以下是 2024-01-24T03:57:13.465418 运行 的最新结果：

python { "all": { "acc": 0.6550573661941113, "acc_stderr": 0.03204593323100834, "acc_norm": 0.6543869571104515, "acc_norm_stderr": 0.032715471402145695, "mc1": 0.5569155446756426, "mc1_stderr": 0.017389730346877116, "mc2": 0.6897987229376631, "mc2_stderr": 0.015167779378758222 }, "harness|arc:challenge|25": { "acc": 0.6996587030716723, "acc_stderr": 0.013395909309957002, "acc_norm": 0.7329351535836177, "acc_norm_stderr": 0.012928933196496364 }, "harness|hellaswag|10": { "acc": 0.7191794463254332, "acc_stderr": 0.00448481564706465, "acc_norm": 0.884883489344752, "acc_norm_stderr": 0.0031851021916879108 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.35, "acc_stderr": 0.0479372485441102, "acc_norm": 0.35, "acc_norm_stderr": 0.0479372485441102 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.6518518518518519, "acc_stderr": 0.041153246103369526, "acc_norm": 0.6518518518518519, "acc_norm_stderr": 0.041153246103369526 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.6973684210526315, "acc_stderr": 0.037385206761196686, "acc_norm": 0.6973684210526315, "acc_norm_stderr": 0.037385206761196686 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.65, "acc_stderr": 0.0479372485441102, "acc_norm": 0.65, "acc_norm_stderr": 0.0479372485441102 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.7169811320754716, "acc_stderr": 0.027724236492700914, "acc_norm": 0.7169811320754716, "acc_norm_stderr": 0.027724236492700914 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.7847222222222222, "acc_stderr": 0.03437079344106135, "acc_norm": 0.7847222222222222, "acc_norm_stderr": 0.03437079344106135 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.5, "acc_stderr": 0.050251890762960605, "acc_norm": 0.5, "acc_norm_stderr": 0.050251890762960605 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.54, "acc_stderr": 0.05009082659620333, "acc_norm": 0.54, "acc_norm_stderr": 0.05009082659620333 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.32, "acc_stderr": 0.046882617226215034, "acc_norm": 0.32, "acc_norm_stderr": 0.046882617226215034 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.6820809248554913, "acc_stderr": 0.0355068398916558, "acc_norm": 0.6820809248554913, "acc_norm_stderr": 0.0355068398916558 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.4215686274509804, "acc_stderr": 0.04913595201274498, "acc_norm": 0.4215686274509804, "acc_norm_stderr": 0.04913595201274498 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.75, "acc_stderr": 0.04351941398892446, "acc_norm": 0.75, "acc_norm_stderr": 0.04351941398892446 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.5872340425531914, "acc_stderr": 0.03218471141400351, "acc_norm": 0.5872340425531914, "acc_norm_stderr": 0.03218471141400351 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.5, "acc_stderr": 0.047036043419179864, "acc_norm": 0.5, "acc_norm_stderr": 0.047036043419179864 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5724137931034483, "acc_stderr": 0.04122737111370333, "acc_norm": 0.5724137931034483, "acc_norm_stderr": 0.04122737111370333 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.42328042328042326, "acc_stderr": 0.025446365634406