open-llm-leaderboard-old/details_AA051611__O0201

该数据集是在Open LLM Leaderboard上对模型AA051611/O0201进行评估运行期间自动创建的。数据集由63个配置组成，每个配置对应一个评估任务。数据集包含一次或多次运行的结果，每次运行都作为每个配置中的一个特定分割存储。train分割始终指向最新的结果。此外，还有一个results配置，用于存储运行的所有聚合结果，并用于计算和显示Open LLM Leaderboard上的聚合指标。README还提供了如何使用`datasets`库中的`load_dataset`函数加载数据集的示例。

原始信息汇总

数据集概述

该数据集是在模型AA051611/O0201在Open LLM Leaderboard上的评估运行期间自动创建的。数据集包含63个配置，每个配置对应一个评估任务。

数据集结构

• 配置数量：63个配置
• 数据来源：从1次运行中创建，每个运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
• 最新结果："train"分割始终指向最新结果。
• 结果汇总：一个额外的配置"results"存储所有运行的汇总结果，用于计算和显示在Open LLM Leaderboard上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_AA051611__O0201", "harness_winogrande_5", split="train")

最新结果

以下是2024-02-02T03:15:57.040098运行的最新结果：

python { "all": { "acc": 0.8785205668188928, "acc_stderr": 0.02108960224655999, "acc_norm": 0.8890480079512375, "acc_norm_stderr": 0.021357047298961734, "mc1": 0.40024479804161567, "mc1_stderr": 0.017151605555749138, "mc2": 0.5863202491791143, "mc2_stderr": 0.015280659551121102 }, "harness|arc:challenge|25": { "acc": 0.6476109215017065, "acc_stderr": 0.013960142600598685, "acc_norm": 0.6783276450511946, "acc_norm_stderr": 0.013650488084494162 }, "harness|hellaswag|10": { "acc": 0.6456881099382593, "acc_stderr": 0.004773267510112743, "acc_norm": 0.844851623182633, "acc_norm_stderr": 0.0036130615166899793 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.72, "acc_stderr": 0.045126085985421276, "acc_norm": 0.72, "acc_norm_stderr": 0.045126085985421276 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.8666666666666667, "acc_stderr": 0.029365879728106854, "acc_norm": 0.8666666666666667, "acc_norm_stderr": 0.029365879728106854 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.9276315789473685, "acc_stderr": 0.021085011261884105, "acc_norm": 0.9276315789473685, "acc_norm_stderr": 0.021085011261884105 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.84, "acc_stderr": 0.03684529491774711, "acc_norm": 0.84, "acc_norm_stderr": 0.03684529491774711 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.9283018867924528, "acc_stderr": 0.015878026288737926, "acc_norm": 0.9283018867924528, "acc_norm_stderr": 0.015878026288737926 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.9513888888888888, "acc_stderr": 0.017983689383153575, "acc_norm": 0.9513888888888888, "acc_norm_stderr": 0.017983689383153575 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.78, "acc_stderr": 0.04163331998932261, "acc_norm": 0.78, "acc_norm_stderr": 0.04163331998932261 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.84, "acc_stderr": 0.03684529491774711, "acc_norm": 0.84, "acc_norm_stderr": 0.03684529491774711 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.76, "acc_stderr": 0.042923469599092816, "acc_norm": 0.76, "acc_norm_stderr": 0.042923469599092816 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.8670520231213873, "acc_stderr": 0.025888042979662292, "acc_norm": 0.8670520231213873, "acc_norm_stderr": 0.025888042979662292 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.7549019607843137, "acc_stderr": 0.04280105837364395, "acc_norm": 0.7549019607843137, "acc_norm_stderr": 0.04280105837364395 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.91, "acc_stderr": 0.028762349126466108, "acc_norm": 0.91, "acc_norm_stderr": 0.028762349126466108 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.9234042553191489, "acc_stderr": 0.017385625826369294, "acc_norm": 0.9234042553191489, "acc_norm_stderr": 0.017385625826369294 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.8421052631578947, "acc_stderr": 0.03430265978485699, "acc_norm": 0.8421052631578947, "acc_norm_stderr": 0.03430265978485699 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.8827586206896552, "acc_stderr": 0.026808974229173797, "acc_norm": 0.8827586206896552, "acc_norm_stderr": 0.026808974229173797 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.8677248677248677, "acc_stderr": 0.01744855429068043, "acc_norm": 0.8677248677248677, "acc_norm_stderr": 0.01744855429068043 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.6904761904761905, "acc_stderr": 0.04134913018303318, "acc_norm": 0.6904761904761905, "acc_norm_stderr": 0.04134913018303318 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.74, "acc_stderr": 0.04408440022768078, "acc_norm": 0.74, "acc_norm_stderr": 0.04408440022768078 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.9709677419354839, "acc_stderr": 0.00955132381346253, "acc_norm": 0.9709677419354839, "acc_norm_stderr": 0.00955132381346253 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.8374384236453202, "acc_stderr": 0.025960300064605587, "acc_norm": 0.8374384236453202, "acc_norm_stderr": 0.025960300064605587 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.94, "acc_stderr": 0.02386832565759418, "acc_norm": 0.94, "acc_norm_stderr": 0.02386832565759418 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.9515151515151515, "acc_stderr": 0.016772158250856272, "acc_norm": 0.9515151515151515, "acc_norm_stderr": 0.016772158250856272 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.98989898989899, "acc_stderr": 0.0071243415212508135, "acc_norm": 0.98989898989899, "acc_norm_stderr": 0.0071243415212508135 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.9844559585492227, "acc_stderr": 0.008927492715084346, "acc_norm": 0.9844559585492227, "acc_norm_stderr": 0.008927492715084346 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.9230769230769231, "acc_stderr": 0.013510532610273879, "acc_norm": 0.9230769230769231, "acc_norm_stderr": 0.013510532610273879 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.8074074074074075, "acc_stderr": 0.02404307518194519, "acc_norm": 0.8074074074