open-llm-leaderboard/details_frankenmerger__cosmo-3b-test-v0.2

该数据集是在评估模型frankenmerger/cosmo-3b-test-v0.2时自动生成的，包含63个配置，每个配置对应一个评估任务。数据集由1次运行生成，每次运行的结果作为特定的分割存储在配置中，分割名称使用运行的时间戳。train分割始终指向最新的结果。此外，还有一个名为results的配置存储了所有运行的聚合结果，用于在Open LLM Leaderboard上计算和显示聚合指标。

原始信息汇总

数据集概述

数据集摘要

该数据集是在模型frankenmerger/cosmo-3b-test-v0.2在Open LLM Leaderboard上的评估运行期间自动创建的。数据集由63个配置组成，每个配置对应一个评估任务。

数据集结构

• 配置数量：63个配置
• 创建来源：从1次运行中创建，每个运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
• 分割：每个配置包含多个分割，其中“train”分割始终指向最新结果。
• 结果配置：一个额外的配置“results”存储所有运行的聚合结果，用于计算和显示在Open LLM Leaderboard上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_frankenmerger__cosmo-3b-test-v0.2", "harness_winogrande_5", split="train")

最新结果

以下是2024-03-22T02:37:15.773693运行的最新结果：

python { "all": { "acc": 0.27603049873373764, "acc_stderr": 0.0315503771458836, "acc_norm": 0.2787704834342469, "acc_norm_stderr": 0.032334747829882354, "mc1": 0.2252141982864137, "mc1_stderr": 0.014623240768023505, "mc2": 0.3881867931183807, "mc2_stderr": 0.014568858333407474 }, "harness|arc:challenge|25": { "acc": 0.30887372013651876, "acc_stderr": 0.013501770929344004, "acc_norm": 0.3532423208191126, "acc_norm_stderr": 0.013967822714840055 }, "harness|hellaswag|10": { "acc": 0.40151364270065726, "acc_stderr": 0.004892026457294703, "acc_norm": 0.5170284803823939, "acc_norm_stderr": 0.004986886806565652 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.26, "acc_stderr": 0.0440844002276808, "acc_norm": 0.26, "acc_norm_stderr": 0.0440844002276808 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.2222222222222222, "acc_stderr": 0.035914440841969694, "acc_norm": 0.2222222222222222, "acc_norm_stderr": 0.035914440841969694 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.34868421052631576, "acc_stderr": 0.0387813988879761, "acc_norm": 0.34868421052631576, "acc_norm_stderr": 0.0387813988879761 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.22, "acc_stderr": 0.041633319989322695, "acc_norm": 0.22, "acc_norm_stderr": 0.041633319989322695 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.30566037735849055, "acc_stderr": 0.028353298073322666, "acc_norm": 0.30566037735849055, "acc_norm_stderr": 0.028353298073322666 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.25, "acc_stderr": 0.03621034121889507, "acc_norm": 0.25, "acc_norm_stderr": 0.03621034121889507 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.34, "acc_stderr": 0.04760952285695235, "acc_norm": 0.34, "acc_norm_stderr": 0.04760952285695235 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.31, "acc_stderr": 0.04648231987117316, "acc_norm": 0.31, "acc_norm_stderr": 0.04648231987117316 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.26, "acc_stderr": 0.0440844002276808, "acc_norm": 0.26, "acc_norm_stderr": 0.0440844002276808 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.3063583815028902, "acc_stderr": 0.03514942551267439, "acc_norm": 0.3063583815028902, "acc_norm_stderr": 0.03514942551267439 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.3627450980392157, "acc_stderr": 0.04784060704105655, "acc_norm": 0.3627450980392157, "acc_norm_stderr": 0.04784060704105655 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.23, "acc_stderr": 0.042295258468165044, "acc_norm": 0.23, "acc_norm_stderr": 0.042295258468165044 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.30638297872340425, "acc_stderr": 0.03013590647851756, "acc_norm": 0.30638297872340425, "acc_norm_stderr": 0.03013590647851756 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.22807017543859648, "acc_stderr": 0.03947152782669415, "acc_norm": 0.22807017543859648, "acc_norm_stderr": 0.03947152782669415 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.2413793103448276, "acc_stderr": 0.03565998174135302, "acc_norm": 0.2413793103448276, "acc_norm_stderr": 0.03565998174135302 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.29894179894179895, "acc_stderr": 0.02357760479165581, "acc_norm": 0.29894179894179895, "acc_norm_stderr": 0.02357760479165581 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.21428571428571427, "acc_stderr": 0.03670066451047182, "acc_norm": 0.21428571428571427, "acc_norm_stderr": 0.03670066451047182 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.18, "acc_stderr": 0.03861229196653694, "acc_norm": 0.18, "acc_norm_stderr": 0.03861229196653694 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.3225806451612903, "acc_stderr": 0.02659308451657228, "acc_norm": 0.3225806451612903, "acc_norm_stderr": 0.02659308451657228 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.3103448275862069, "acc_stderr": 0.03255086769970103, "acc_norm": 0.3103448275862069, "acc_norm_stderr": 0.03255086769970103 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.27, "acc_stderr": 0.044619604333847394, "acc_norm": 0.27, "acc_norm_stderr": 0.044619604333847394 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.2909090909090909, "acc_stderr": 0.03546563019624336, "acc_norm": 0.2909090909090909, "acc_norm_stderr": 0.03546563019624336 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.3383838383838384, "acc_stderr": 0.03371124142626303, "acc_norm": 0.3383838383838384, "acc_norm_stderr": 0.03371124142626303 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.22279792746113988, "acc_stderr": 0.030031147977641545, "acc_norm": 0.22279792746113988, "acc_norm_stderr": 0.030031147977641545 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.33589743589743587, "acc_stderr": 0.023946724741563976, "acc_norm": 0.33589743589743587, "acc_norm_stderr": 0.023946724741563976 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.24814814814814815, "acc_stderr": 0.0263357394040558, "acc_norm": 0.24814814814814