open-llm-leaderboard-old/details_bigscience__bloomz

该数据集是在Open LLM Leaderboard上对模型进行自动评估时生成的。数据集由61个配置组成，每个配置对应一个评估任务。数据集从1次运行中创建，每次运行都可以在特定配置中找到，分割名称使用运行的时间戳。train分割始终指向最新结果。此外，results配置存储了所有运行的聚合结果，并用于计算和显示Open LLM Leaderboard上的聚合指标。

原始信息汇总

数据集概述

该数据集是在模型None在Open LLM Leaderboard上的评估运行期间自动创建的。数据集由61个配置组成，每个配置对应一个评估任务。

数据集结构

• 数据集从1次运行中创建。每个运行可以在每个配置中作为一个特定的分片找到，分片名称使用运行的时间戳。
• "train"分片始终指向最新的结果。
• 额外的配置"results"存储所有运行的聚合结果，用于计算和显示Open LLM Leaderboard上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_bigscience__bloomz", "harness_truthfulqa_mc_0", split="train")

最新结果

以下是2023-08-29T12:14:13.875692运行的最新结果：

python { "all": { "acc": 0.47917755403252543, "acc_stderr": 0.03572101484290109, "acc_norm": 0.48335485520551164, "acc_norm_stderr": 0.0357085480998606, "mc1": 0.2607099143206854, "mc1_stderr": 0.015368841620766372, "mc2": 0.4393940961026447, "mc2_stderr": 0.015292532701908591 }, "harness|arc:challenge|25": { "acc": 0.5042662116040956, "acc_stderr": 0.014610858923956955, "acc_norm": 0.5537542662116041, "acc_norm_stderr": 0.014526705548539982 }, "harness|hellaswag|10": { "acc": 0.5553674566819359, "acc_stderr": 0.004959094146471527, "acc_norm": 0.7523401712806214, "acc_norm_stderr": 0.004307709682499536 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.28, "acc_stderr": 0.04512608598542129, "acc_norm": 0.28, "acc_norm_stderr": 0.04512608598542129 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.45185185185185184, "acc_stderr": 0.04299268905480863, "acc_norm": 0.45185185185185184, "acc_norm_stderr": 0.04299268905480863 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.5, "acc_stderr": 0.04068942293855797, "acc_norm": 0.5, "acc_norm_stderr": 0.04068942293855797 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.55, "acc_stderr": 0.05, "acc_norm": 0.55, "acc_norm_stderr": 0.05 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.5735849056603773, "acc_stderr": 0.030437794342983052, "acc_norm": 0.5735849056603773, "acc_norm_stderr": 0.030437794342983052 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.5277777777777778, "acc_stderr": 0.04174752578923185, "acc_norm": 0.5277777777777778, "acc_norm_stderr": 0.04174752578923185 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.41, "acc_stderr": 0.049431107042371025, "acc_norm": 0.41, "acc_norm_stderr": 0.049431107042371025 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.45, "acc_stderr": 0.04999999999999999, "acc_norm": 0.45, "acc_norm_stderr": 0.04999999999999999 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.35, "acc_stderr": 0.0479372485441102, "acc_norm": 0.35, "acc_norm_stderr": 0.0479372485441102 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.43352601156069365, "acc_stderr": 0.03778621079092055, "acc_norm": 0.43352601156069365, "acc_norm_stderr": 0.03778621079092055 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.39215686274509803, "acc_stderr": 0.04858083574266344, "acc_norm": 0.39215686274509803, "acc_norm_stderr": 0.04858083574266344 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.47, "acc_stderr": 0.05016135580465919, "acc_norm": 0.47, "acc_norm_stderr": 0.05016135580465919 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.4425531914893617, "acc_stderr": 0.03246956919789958, "acc_norm": 0.4425531914893617, "acc_norm_stderr": 0.03246956919789958 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.32456140350877194, "acc_stderr": 0.04404556157374767, "acc_norm": 0.32456140350877194, "acc_norm_stderr": 0.04404556157374767 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.46206896551724136, "acc_stderr": 0.041546596717075474, "acc_norm": 0.46206896551724136, "acc_norm_stderr": 0.041546596717075474 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.32275132275132273, "acc_stderr": 0.024078943243597016, "acc_norm": 0.32275132275132273, "acc_norm_stderr": 0.024078943243597016 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.30952380952380953, "acc_stderr": 0.041349130183033156, "acc_norm": 0.30952380952380953, "acc_norm_stderr": 0.041349130183033156 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.28, "acc_stderr": 0.045126085985421276, "acc_norm": 0.28, "acc_norm_stderr": 0.045126085985421276 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.532258064516129, "acc_stderr": 0.028384747788813332, "acc_norm": 0.532258064516129, "acc_norm_stderr": 0.028384747788813332 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.39408866995073893, "acc_stderr": 0.034381579670365446, "acc_norm": 0.39408866995073893, "acc_norm_stderr": 0.034381579670365446 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.47, "acc_stderr": 0.05016135580465919, "acc_norm": 0.47, "acc_norm_stderr": 0.05016135580465919 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.49696969696969695, "acc_stderr": 0.03904272341431857, "acc_norm": 0.49696969696969695, "acc_norm_stderr": 0.03904272341431857 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.6616161616161617, "acc_stderr": 0.03371124142626303, "acc_norm": 0.6616161616161617, "acc_norm_stderr": 0.03371124142626303 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.616580310880829, "acc_stderr": 0.03508984236295342, "acc_norm": 0.616580310880829, "acc_norm_stderr": 0.03508984236295342 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.47435897435897434, "acc_stderr": 0.02531764972644865, "acc_norm": 0.47435897435897434, "acc_norm_stderr": 0.02531764972644865 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.29259259259259257, "acc_stderr": 0.02773896963217609, "acc_norm": 0.29259259259259257, "acc_norm_stderr": 0.02773896963217609 }, "harness|hendrycksTest-high_school_microeconomics|5": { "acc": 0.5672268907563025, "acc_stderr": 0.0321835810774