open-llm-leaderboard-old/details_timpal0l__Mistral-7B-v0.1-flashback-v2

该数据集是在Open LLM Leaderboard上对模型timpal0l/Mistral-7B-v0.1-flashback-v2进行评估时自动创建的。数据集由63个配置组成，每个配置对应一个评估任务。数据集是从1次运行中创建的，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。train分割始终指向最新的结果。此外，还有一个名为results的配置，存储了所有运行的聚合结果，并用于在Open LLM Leaderboard上计算和显示聚合指标。

原始信息汇总

数据集概述

数据集简介

该数据集是在对模型 timpal0l/Mistral-7B-v0.1-flashback-v2 进行评估运行期间自动创建的，用于 Open LLM Leaderboard。

数据集结构

• 配置数量：63个配置，每个配置对应一个评估任务。
• 数据来源：数据集来自1次运行，每个运行在每个配置中都有一个特定的分割，分割名称使用运行的时间戳。
• 最新结果："train" 分割总是指向最新的结果。
• 结果汇总：一个额外的配置 "results" 存储所有运行的汇总结果，用于计算和显示 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_timpal0l__Mistral-7B-v0.1-flashback-v2", "harness_winogrande_5", split="train")

最新结果

以下是 2024-01-13T15:10:27.393635 运行的最新结果：

python { "all": { "acc": 0.5963604853429504, "acc_stderr": 0.03312318488664919, "acc_norm": 0.6028320780728574, "acc_norm_stderr": 0.03381496123659357, "mc1": 0.2766217870257038, "mc1_stderr": 0.015659605755326916, "mc2": 0.40658215292594935, "mc2_stderr": 0.014101721545122618 }, "harness|arc:challenge|25": { "acc": 0.523037542662116, "acc_stderr": 0.014595873205358273, "acc_norm": 0.5716723549488054, "acc_norm_stderr": 0.014460496367599017 }, "harness|hellaswag|10": { "acc": 0.6008763194582752, "acc_stderr": 0.004887174080003034, "acc_norm": 0.8074088826926907, "acc_norm_stderr": 0.003935286940315854 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.32, "acc_stderr": 0.04688261722621503, "acc_norm": 0.32, "acc_norm_stderr": 0.04688261722621503 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.5407407407407407, "acc_stderr": 0.04304979692464241, "acc_norm": 0.5407407407407407, "acc_norm_stderr": 0.04304979692464241 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.6118421052631579, "acc_stderr": 0.03965842097512744, "acc_norm": 0.6118421052631579, "acc_norm_stderr": 0.03965842097512744 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.55, "acc_stderr": 0.05, "acc_norm": 0.55, "acc_norm_stderr": 0.05 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.6415094339622641, "acc_stderr": 0.029514703583981765, "acc_norm": 0.6415094339622641, "acc_norm_stderr": 0.029514703583981765 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.6180555555555556, "acc_stderr": 0.040629907841466674, "acc_norm": 0.6180555555555556, "acc_norm_stderr": 0.040629907841466674 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.41, "acc_stderr": 0.049431107042371025, "acc_norm": 0.41, "acc_norm_stderr": 0.049431107042371025 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.46, "acc_stderr": 0.05009082659620333, "acc_norm": 0.46, "acc_norm_stderr": 0.05009082659620333 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.31, "acc_stderr": 0.04648231987117316, "acc_norm": 0.31, "acc_norm_stderr": 0.04648231987117316 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5780346820809249, "acc_stderr": 0.0376574669386515, "acc_norm": 0.5780346820809249, "acc_norm_stderr": 0.0376574669386515 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.3431372549019608, "acc_stderr": 0.047240073523838876, "acc_norm": 0.3431372549019608, "acc_norm_stderr": 0.047240073523838876 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.72, "acc_stderr": 0.045126085985421276, "acc_norm": 0.72, "acc_norm_stderr": 0.045126085985421276 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.5276595744680851, "acc_stderr": 0.03263597118409769, "acc_norm": 0.5276595744680851, "acc_norm_stderr": 0.03263597118409769 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.45614035087719296, "acc_stderr": 0.04685473041907789, "acc_norm": 0.45614035087719296, "acc_norm_stderr": 0.04685473041907789 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5655172413793104, "acc_stderr": 0.041307408795554966, "acc_norm": 0.5655172413793104, "acc_norm_stderr": 0.041307408795554966 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.37566137566137564, "acc_stderr": 0.024942368931159784, "acc_norm": 0.37566137566137564, "acc_norm_stderr": 0.024942368931159784 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.373015873015873, "acc_stderr": 0.04325506042017086, "acc_norm": 0.373015873015873, "acc_norm_stderr": 0.04325506042017086 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.37, "acc_stderr": 0.04852365870939099, "acc_norm": 0.37, "acc_norm_stderr": 0.04852365870939099 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.7161290322580646, "acc_stderr": 0.025649381063029268, "acc_norm": 0.7161290322580646, "acc_norm_stderr": 0.025649381063029268 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.4876847290640394, "acc_stderr": 0.035169204442208966, "acc_norm": 0.4876847290640394, "acc_norm_stderr": 0.035169204442208966 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.62, "acc_stderr": 0.04878317312145632, "acc_norm": 0.62, "acc_norm_stderr": 0.04878317312145632 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.7151515151515152, "acc_stderr": 0.03524390844511781, "acc_norm": 0.7151515151515152, "acc_norm_stderr": 0.03524390844511781 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.7222222222222222, "acc_stderr": 0.03191178226713549, "acc_norm": 0.7222222222222222, "acc_norm_stderr": 0.03191178226713549 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.8290155440414507, "acc_stderr": 0.027171213683164542, "acc_norm": 0.8290155440414507, "acc_norm_stderr": 0.027171213683164542 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.5897435897435898, "acc_stderr": 0.024939313906940798, "acc_norm": 0.5897435897435898, "acc_norm_stderr": 0.024939313906940798 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.32222222222222224, "acc_stderr": 0.028493465091028593, "acc_norm": 0.32222222222222224, "acc_norm_stderr": 0.0284934650910