open-llm-leaderboard-old/details_smelborp__MixtralOrochi8x7B-Alt

该数据集是在Open LLM Leaderboard上对模型smelborp/MixtralOrochi8x7B-Alt进行评估时自动生成的。数据集由63个配置组成，每个配置对应一个被评估的任务。它包含一次运行的数据，每次运行在每个配置中表示为特定的分割，分割名称由运行的时间戳命名。train分割始终指向最新结果。此外，results配置存储了所有运行的聚合结果，用于计算和显示Open LLM Leaderboard上的聚合指标。README还提供了如何使用Hugging Face datasets库加载数据集的示例，并包含了特定运行的最新结果。

原始信息汇总

数据集概述

数据集简介

该数据集是在模型smelborp/MixtralOrochi8x7B-Alt在Open LLM Leaderboard上的评估运行期间自动创建的。

数据集结构

• 配置数量：63个配置，每个配置对应一个评估任务。
• 创建来源：从1次运行中创建。每个运行可以在每个配置中作为一个特定的分割找到，分割名称使用运行的时间戳。
• 最新结果："train"分割始终指向最新的结果。
• 聚合结果：一个额外的配置"results"存储所有运行的聚合结果，用于计算和显示在Open LLM Leaderboard上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_smelborp__MixtralOrochi8x7B-Alt", "harness_winogrande_5", split="train")

最新结果

以下是2023-12-29T16:36:26.301610运行的最新结果：

python { "all": { "acc": 0.6897102628842678, "acc_stderr": 0.03039342739788087, "acc_norm": 0.7029575662168503, "acc_norm_stderr": 0.03120937495626396, "mc1": 0.45532435740514077, "mc1_stderr": 0.017433490102538772, "mc2": 0.6403236854599645, "mc2_stderr": 0.01510362269809065 }, "harness|arc:challenge|25": { "acc": 0.6621160409556314, "acc_stderr": 0.013822047922283505, "acc_norm": 0.6791808873720137, "acc_norm_stderr": 0.013640943091946535 }, "harness|hellaswag|10": { "acc": 0.6770563632742481, "acc_stderr": 0.004666457279979415, "acc_norm": 0.86247759410476, "acc_norm_stderr": 0.00343694164178278 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.43, "acc_stderr": 0.049756985195624284, "acc_norm": 0.43, "acc_norm_stderr": 0.049756985195624284 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.6592592592592592, "acc_stderr": 0.04094376269996793, "acc_norm": 0.6592592592592592, "acc_norm_stderr": 0.04094376269996793 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.7828947368421053, "acc_stderr": 0.03355045304882924, "acc_norm": 0.7828947368421053, "acc_norm_stderr": 0.03355045304882924 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.77, "acc_stderr": 0.04229525846816506, "acc_norm": 0.77, "acc_norm_stderr": 0.04229525846816506 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.7886792452830189, "acc_stderr": 0.025125766484827845, "acc_norm": 0.7886792452830189, "acc_norm_stderr": 0.025125766484827845 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.8333333333333334, "acc_stderr": 0.031164899666948614, "acc_norm": 0.8333333333333334, "acc_norm_stderr": 0.031164899666948614 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.51, "acc_stderr": 0.05024183937956912, "acc_norm": 0.51, "acc_norm_stderr": 0.05024183937956912 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.59, "acc_stderr": 0.04943110704237102, "acc_norm": 0.59, "acc_norm_stderr": 0.04943110704237102 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.41, "acc_stderr": 0.049431107042371025, "acc_norm": 0.41, "acc_norm_stderr": 0.049431107042371025 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.7109826589595376, "acc_stderr": 0.03456425745086998, "acc_norm": 0.7109826589595376, "acc_norm_stderr": 0.03456425745086998 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.4411764705882353, "acc_stderr": 0.04940635630605659, "acc_norm": 0.4411764705882353, "acc_norm_stderr": 0.04940635630605659 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.83, "acc_stderr": 0.03775251680686371, "acc_norm": 0.83, "acc_norm_stderr": 0.03775251680686371 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.6978723404255319, "acc_stderr": 0.030017554471880557, "acc_norm": 0.6978723404255319, "acc_norm_stderr": 0.030017554471880557 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.5614035087719298, "acc_stderr": 0.04668000738510455, "acc_norm": 0.5614035087719298, "acc_norm_stderr": 0.04668000738510455 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.6137931034482759, "acc_stderr": 0.04057324734419035, "acc_norm": 0.6137931034482759, "acc_norm_stderr": 0.04057324734419035 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.5026455026455027, "acc_stderr": 0.02575094967813038, "acc_norm": 0.5026455026455027, "acc_norm_stderr": 0.02575094967813038 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.5, "acc_stderr": 0.04472135954999579, "acc_norm": 0.5, "acc_norm_stderr": 0.04472135954999579 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.43, "acc_stderr": 0.049756985195624284, "acc_norm": 0.43, "acc_norm_stderr": 0.049756985195624284 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.8129032258064516, "acc_stderr": 0.022185710092252255, "acc_norm": 0.8129032258064516, "acc_norm_stderr": 0.022185710092252255 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.5911330049261084, "acc_stderr": 0.034590588158832314, "acc_norm": 0.5911330049261084, "acc_norm_stderr": 0.034590588158832314 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.74, "acc_stderr": 0.0440844002276808, "acc_norm": 0.74, "acc_norm_stderr": 0.0440844002276808 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.8363636363636363, "acc_stderr": 0.02888787239548795, "acc_norm": 0.8363636363636363, "acc_norm_stderr": 0.02888787239548795 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.8686868686868687, "acc_stderr": 0.024063156416822516, "acc_norm": 0.8686868686868687, "acc_norm_stderr": 0.024063156416822516 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.9378238341968912, "acc_stderr": 0.017426974154240524, "acc_norm": 0.9378238341968912, "acc_norm_stderr": 0.017426974154240524 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.676923076923077, "acc_stderr": 0.02371088850197057, "acc_norm": 0.676923076923077, "acc_norm_stderr": 0.02371088850197057 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.34074074074074073, "acc_stderr": 0.028897748741131137, "acc_norm": 0.34074074074074073, "acc_norm_stderr": 0.02889774874