open-llm-leaderboard-old/details_0x7194633__fialka-13B-v3

该数据集是在评估模型0x7194633/fialka-13B-v3时自动创建的，主要用于在Open LLM Leaderboard上进行评估。数据集包含63个配置，每个配置对应一个评估任务。数据集由1次运行生成，每次运行的结果作为特定配置中的一个分割，分割名称使用运行的时间戳。train分割始终指向最新的结果。此外，还有一个名为results的配置，存储了所有运行的聚合结果，并用于计算和显示Open LLM Leaderboard上的聚合指标。

原始信息汇总

数据集概述

数据集简介

该数据集是在对模型 0x7194633/fialka-13B-v3 进行评估运行时自动创建的，用于 Open LLM Leaderboard。

数据集组成

• 数据集包含 63 个配置，每个配置对应一个评估任务。
• 数据集从 1 次运行中创建，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
• "train" 分割始终指向最新的结果。
• 一个额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示聚合指标在 Open LLM Leaderboard 上。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_0x7194633__fialka-13B-v3", "harness_winogrande_5", split="train")

最新结果

以下是 2023-12-30T05:28:30.247566 运行的最新结果：

python { "all": { "acc": 0.267282756151897, "acc_stderr": 0.031086803905318892, "acc_norm": 0.2681655321925398, "acc_norm_stderr": 0.031861580271542414, "mc1": 0.26193390452876375, "mc1_stderr": 0.01539211880501503, "mc2": 0.4058016031292479, "mc2_stderr": 0.014619680021653623 }, "harness|arc:challenge|25": { "acc": 0.28498293515358364, "acc_stderr": 0.013191348179838793, "acc_norm": 0.3097269624573379, "acc_norm_stderr": 0.013512058415238361 }, "harness|hellaswag|10": { "acc": 0.38836885082652856, "acc_stderr": 0.004863831364848084, "acc_norm": 0.48834893447520417, "acc_norm_stderr": 0.004988426528513012 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.27, "acc_stderr": 0.044619604333847394, "acc_norm": 0.27, "acc_norm_stderr": 0.044619604333847394 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.22962962962962963, "acc_stderr": 0.03633384414073464, "acc_norm": 0.22962962962962963, "acc_norm_stderr": 0.03633384414073464 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.19736842105263158, "acc_stderr": 0.03238981601699397, "acc_norm": 0.19736842105263158, "acc_norm_stderr": 0.03238981601699397 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.28, "acc_stderr": 0.04512608598542126, "acc_norm": 0.28, "acc_norm_stderr": 0.04512608598542126 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.24528301886792453, "acc_stderr": 0.02648035717989568, "acc_norm": 0.24528301886792453, "acc_norm_stderr": 0.02648035717989568 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.25, "acc_stderr": 0.03621034121889507, "acc_norm": 0.25, "acc_norm_stderr": 0.03621034121889507 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.28, "acc_stderr": 0.04512608598542128, "acc_norm": 0.28, "acc_norm_stderr": 0.04512608598542128 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.32, "acc_stderr": 0.04688261722621504, "acc_norm": 0.32, "acc_norm_stderr": 0.04688261722621504 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.19, "acc_stderr": 0.03942772444036623, "acc_norm": 0.19, "acc_norm_stderr": 0.03942772444036623 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.23121387283236994, "acc_stderr": 0.032147373020294696, "acc_norm": 0.23121387283236994, "acc_norm_stderr": 0.032147373020294696 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.27450980392156865, "acc_stderr": 0.04440521906179328, "acc_norm": 0.27450980392156865, "acc_norm_stderr": 0.04440521906179328 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.22, "acc_stderr": 0.041633319989322716, "acc_norm": 0.22, "acc_norm_stderr": 0.041633319989322716 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.24680851063829787, "acc_stderr": 0.028185441301234102, "acc_norm": 0.24680851063829787, "acc_norm_stderr": 0.028185441301234102 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.2543859649122807, "acc_stderr": 0.0409698513984367, "acc_norm": 0.2543859649122807, "acc_norm_stderr": 0.0409698513984367 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.2482758620689655, "acc_stderr": 0.03600105692727772, "acc_norm": 0.2482758620689655, "acc_norm_stderr": 0.03600105692727772 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.21957671957671956, "acc_stderr": 0.021320018599770348, "acc_norm": 0.21957671957671956, "acc_norm_stderr": 0.021320018599770348 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.16666666666666666, "acc_stderr": 0.03333333333333337, "acc_norm": 0.16666666666666666, "acc_norm_stderr": 0.03333333333333337 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.17, "acc_stderr": 0.0377525168068637, "acc_norm": 0.17, "acc_norm_stderr": 0.0377525168068637 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.25806451612903225, "acc_stderr": 0.024892469172462836, "acc_norm": 0.25806451612903225, "acc_norm_stderr": 0.024892469172462836 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.2315270935960591, "acc_stderr": 0.02967833314144444, "acc_norm": 0.2315270935960591, "acc_norm_stderr": 0.02967833314144444 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.21, "acc_stderr": 0.04093601807403326, "acc_norm": 0.21, "acc_norm_stderr": 0.04093601807403326 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.22424242424242424, "acc_stderr": 0.032568666616811015, "acc_norm": 0.22424242424242424, "acc_norm_stderr": 0.032568666616811015 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.3484848484848485, "acc_stderr": 0.033948539651564025, "acc_norm": 0.3484848484848485, "acc_norm_stderr": 0.033948539651564025 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.29015544041450775, "acc_stderr": 0.03275264467791515, "acc_norm": 0.29015544041450775, "acc_norm_stderr": 0.03275264467791515 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.3282051282051282, "acc_stderr": 0.02380763319865726, "acc_norm": 0.3282051282051282, "acc_norm_stderr": 0.02380763319865726 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.27037037037037037, "acc_stderr": 0.027080372815145665, "acc_norm": 0.27037