open-llm-leaderboard-old/details_itsliupeng__llama_9b_long

该数据集是在模型itsliupeng/llama_9b_long的评估过程中自动创建的，用于Open LLM Leaderboard的评估。数据集由63个配置组成，每个配置对应一个评估任务。数据集由2次运行生成，每次运行的结果作为特定配置中的一个分割，分割名称使用运行的时间戳。train分割始终指向最新的结果。此外，还有一个名为results的配置存储了所有运行的聚合结果，用于计算和展示Open LLM Leaderboard上的聚合指标。

原始信息汇总

数据集概述

数据集组成

• 该数据集包含63个配置，每个配置对应一个评估任务。
• 数据集由2次运行结果组成，每次运行的结果可以在每个配置中找到，以运行的时间戳命名的特定分片形式存储。
• "train"分片始终指向最新的结果。

额外配置

• 一个额外的配置"results"存储所有运行结果的聚合，用于计算和显示在Open LLM Leaderboard上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_itsliupeng__llama_9b_long", "harness_winogrande_5", split="train")

最新结果

• 最新结果来自2024-03-13T04:19:31.526545的运行，包含多个任务的评估结果，具体结果如下：

python { "all": { "acc": 0.6983981584518038, "acc_stderr": 0.03079975479452849, "acc_norm": 0.7048843492764434, "acc_norm_stderr": 0.03139212193469086, "mc1": 0.31334149326805383, "mc1_stderr": 0.016238065069059608, "mc2": 0.4595165177815787, "mc2_stderr": 0.015014100990687205 }, "harness|arc:challenge|25": { "acc": 0.5665529010238908, "acc_stderr": 0.014481376224558902, "acc_norm": 0.6006825938566553, "acc_norm_stderr": 0.014312094557946698 }, "harness|hellaswag|10": { "acc": 0.5871340370444135, "acc_stderr": 0.0049134290105590644, "acc_norm": 0.7866958773152758, "acc_norm_stderr": 0.004088034745195348 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.38, "acc_stderr": 0.048783173121456316, "acc_norm": 0.38, "acc_norm_stderr": 0.048783173121456316 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.674074074074074, "acc_stderr": 0.040491220417025055, "acc_norm": 0.674074074074074, "acc_norm_stderr": 0.040491220417025055 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.7828947368421053, "acc_stderr": 0.03355045304882924, "acc_norm": 0.7828947368421053, "acc_norm_stderr": 0.03355045304882924 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.76, "acc_stderr": 0.04292346959909284, "acc_norm": 0.76, "acc_norm_stderr": 0.04292346959909284 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.7433962264150943, "acc_stderr": 0.026880647889051975, "acc_norm": 0.7433962264150943, "acc_norm_stderr": 0.026880647889051975 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.8194444444444444, "acc_stderr": 0.03216600808802267, "acc_norm": 0.8194444444444444, "acc_norm_stderr": 0.03216600808802267 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.52, "acc_stderr": 0.050211673156867795, "acc_norm": 0.52, "acc_norm_stderr": 0.050211673156867795 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.59, "acc_stderr": 0.04943110704237102, "acc_norm": 0.59, "acc_norm_stderr": 0.04943110704237102 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.39, "acc_stderr": 0.04902071300001974, "acc_norm": 0.39, "acc_norm_stderr": 0.04902071300001974 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.7572254335260116, "acc_stderr": 0.0326926380614177, "acc_norm": 0.7572254335260116, "acc_norm_stderr": 0.0326926380614177 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.5, "acc_stderr": 0.04975185951049946, "acc_norm": 0.5, "acc_norm_stderr": 0.04975185951049946 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.78, "acc_stderr": 0.04163331998932261, "acc_norm": 0.78, "acc_norm_stderr": 0.04163331998932261 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.7063829787234043, "acc_stderr": 0.02977164271249123, "acc_norm": 0.7063829787234043, "acc_norm_stderr": 0.02977164271249123 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.543859649122807, "acc_stderr": 0.046854730419077895, "acc_norm": 0.543859649122807, "acc_norm_stderr": 0.046854730419077895 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.6827586206896552, "acc_stderr": 0.03878352372138622, "acc_norm": 0.6827586206896552, "acc_norm_stderr": 0.03878352372138622 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.6137566137566137, "acc_stderr": 0.025075981767601688, "acc_norm": 0.6137566137566137, "acc_norm_stderr": 0.025075981767601688 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.5793650793650794, "acc_stderr": 0.04415438226743745, "acc_norm": 0.5793650793650794, "acc_norm_stderr": 0.04415438226743745 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.45, "acc_stderr": 0.05, "acc_norm": 0.45, "acc_norm_stderr": 0.05 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.8709677419354839, "acc_stderr": 0.01907088925479276, "acc_norm": 0.8709677419354839, "acc_norm_stderr": 0.01907088925479276 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.625615763546798, "acc_stderr": 0.03405155380561952, "acc_norm": 0.625615763546798, "acc_norm_stderr": 0.03405155380561952 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.8, "acc_stderr": 0.04020151261036846, "acc_norm": 0.8, "acc_norm_stderr": 0.04020151261036846 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.7818181818181819, "acc_stderr": 0.032250781083062896, "acc_norm": 0.7818181818181819, "acc_norm_stderr": 0.032250781083062896 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.8484848484848485, "acc_stderr": 0.025545650426603627, "acc_norm": 0.8484848484848485, "acc_norm_stderr": 0.025545650426603627 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.8756476683937824, "acc_stderr": 0.023814477086593556, "acc_norm": 0.8756476683937824, "acc_norm_stderr": 0.023814477086593556 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.7461538461538462, "acc_stderr": 0.022066054378726257, "acc_norm": 0.7461538461538462, "acc_norm_stderr": 0.022066054378726257 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.48148148148148145, "acc_stderr": 0.030464621718895322, "acc_norm": 0.48148148148148145, "acc_norm_stderr": 0.030464621718895322 }, "harness|hendrycksTest-high_school_microeconomics|5": { "acc": 0.819327731092437, "acc_stderr": 0.02499196496660077, "acc_norm": 0.819327731092437, "acc_norm_stderr": 0.02499196496660077 }, "harness|hendrycksTest-high_school_physics|5": { "acc": 0.41721854304635764, "acc_stderr":