open-llm-leaderboard-old/details_adamo1139__yi-34b-200k-rawrr-dpo-1

该数据集是在对adamo1139/yi-34b-200k-rawrr-dpo-1模型进行评估运行期间自动创建的，包含63个配置，每个配置对应一个评估任务。数据集由1次运行创建，每次运行都可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。train分割始终指向最新的结果。此外，还有一个results配置，存储了运行（和用于计算和显示[Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)上的聚合指标）的所有聚合结果。

原始信息汇总

数据集概述

数据集简介

该数据集是在评估模型 adamo1139/yi-34b-200k-rawrr-dpo-1 在 Open LLM Leaderboard 上的自动创建的。数据集包含63个配置，每个配置对应一个评估任务。

数据集结构

数据集由1次运行创建，每个运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。"train" 分割始终指向最新的结果。

额外配置

一个额外的配置 "results" 存储了所有运行的聚合结果，用于计算和显示在 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_adamo1139__yi-34b-200k-rawrr-dpo-1", "harness_winogrande_5", split="train")

最新结果

总体结果

python { "all": { "acc": 0.7557329155625617, "acc_stderr": 0.02836045891045506, "acc_norm": 0.7606955903500686, "acc_norm_stderr": 0.02889015510293627, "mc1": 0.3929008567931457, "mc1_stderr": 0.017097248285233065, "mc2": 0.5399803687437482, "mc2_stderr": 0.014956918567738575 } }

具体任务结果

• harness|arc:challenge|25: python { "acc": 0.6271331058020477, "acc_stderr": 0.014131176760131172, "acc_norm": 0.6544368600682594, "acc_norm_stderr": 0.013896938461145675 }

• harness|hellaswag|10: python { "acc": 0.6570404301931886, "acc_stderr": 0.00473727969103619, "acc_norm": 0.8569010157339175, "acc_norm_stderr": 0.0034945810763985265 }

• harness|hendrycksTest-abstract_algebra|5: python { "acc": 0.4, "acc_stderr": 0.049236596391733084, "acc_norm": 0.4, "acc_norm_stderr": 0.049236596391733084 }

• harness|hendrycksTest-anatomy|5: python { "acc": 0.7185185185185186, "acc_stderr": 0.038850042458002526, "acc_norm": 0.7185185185185186, "acc_norm_stderr": 0.038850042458002526 }

• harness|hendrycksTest-astronomy|5: python { "acc": 0.8618421052631579, "acc_stderr": 0.028081042939576552, "acc_norm": 0.8618421052631579, "acc_norm_stderr": 0.028081042939576552 }

• harness|hendrycksTest-business_ethics|5: python { "acc": 0.79, "acc_stderr": 0.04093601807403326, "acc_norm": 0.79, "acc_norm_stderr": 0.04093601807403326 }

• harness|hendrycksTest-clinical_knowledge|5: python { "acc": 0.8226415094339623, "acc_stderr": 0.02350873921884694, "acc_norm": 0.8226415094339623, "acc_norm_stderr": 0.02350873921884694 }

• harness|hendrycksTest-college_biology|5: python { "acc": 0.875, "acc_stderr": 0.02765610492929436, "acc_norm": 0.875, "acc_norm_stderr": 0.02765610492929436 }

• harness|hendrycksTest-college_chemistry|5: python { "acc": 0.5, "acc_stderr": 0.050251890762960605, "acc_norm": 0.5, "acc_norm_stderr": 0.050251890762960605 }

• harness|hendrycksTest-college_computer_science|5: python { "acc": 0.62, "acc_stderr": 0.04878317312145633, "acc_norm": 0.62, "acc_norm_stderr": 0.04878317312145633 }

• harness|hendrycksTest-college_mathematics|5: python { "acc": 0.53, "acc_stderr": 0.050161355804659205, "acc_norm": 0.53, "acc_norm_stderr": 0.050161355804659205 }

• harness|hendrycksTest-college_medicine|5: python { "acc": 0.7398843930635838, "acc_stderr": 0.033450369167889904, "acc_norm": 0.7398843930635838, "acc_norm_stderr": 0.033450369167889904 }

• harness|hendrycksTest-college_physics|5: python { "acc": 0.5098039215686274, "acc_stderr": 0.04974229460422817, "acc_norm": 0.5098039215686274, "acc_norm_stderr": 0.04974229460422817 }

• harness|hendrycksTest-computer_security|5: python { "acc": 0.84, "acc_stderr": 0.03684529491774709, "acc_norm": 0.84, "acc_norm_stderr": 0.03684529491774709 }

• harness|hendrycksTest-conceptual_physics|5: python { "acc": 0.7702127659574468, "acc_stderr": 0.02750175294441242, "acc_norm": 0.7702127659574468, "acc_norm_stderr": 0.02750175294441242 }

• harness|hendrycksTest-econometrics|5: python { "acc": 0.5877192982456141, "acc_stderr": 0.04630653203366596, "acc_norm": 0.5877192982456141, "acc_norm_stderr": 0.04630653203366596 }

• harness|hendrycksTest-electrical_engineering|5: python { "acc": 0.7655172413793103, "acc_stderr": 0.035306258743465914, "acc_norm": 0.7655172413793103, "acc_norm_stderr": 0.035306258743465914 }

• harness|hendrycksTest-elementary_mathematics|5: python { "acc": 0.6402116402116402, "acc_stderr": 0.024718075944129277, "acc_norm": 0.6402116402116402, "acc_norm_stderr": 0.024718075944129277 }

• harness|hendrycksTest-formal_logic|5: python { "acc": 0.5952380952380952, "acc_stderr": 0.043902592653775635, "acc_norm": 0.5952380952380952, "acc_norm_stderr": 0.043902592653775635 }

• harness|hendrycksTest-global_facts|5: python { "acc": 0.56, "acc_stderr": 0.049888765156985884, "acc_norm": 0.56, "acc_norm_stderr": 0.049888765156985884 }

• harness|hendrycksTest-high_school_biology|5: python { "acc": 0.896774193548387, "acc_stderr": 0.01730838128103453, "acc_norm": 0.896774193548387, "acc_norm_stderr": 0.01730838128103453 }

• harness|hendrycksTest-high_school_chemistry|5: python { "acc": 0.6699507389162561, "acc_stderr": 0.033085304262282574, "acc_norm": 0.6699507389162561, "acc_norm_stderr": 0.033085304262282574 }

• harness|hendrycksTest-high_school_computer_science|5: python { "acc": 0.76, "acc_stderr": 0.04292346959909283, "acc_norm": 0.76, "acc_norm_stderr": 0.04292346959909283 }

• harness|hendrycksTest-high_school_european_history|5: python { "acc": 0.8484848484848485, "acc_stderr": 0.027998073798781675, "acc_norm": 0.8484848484848485, "acc_norm_stderr": 0.027998073798781675 }

• harness|hendrycksTest-high_school_geography|5: python { "acc": 0.9292929292929293, "acc_stderr": 0.01826310542019949, "acc_norm": 0.9292929292929293, "acc_norm_stderr": 0.01826310542019949 }

• harness|hendrycksTest-high_school_government_and_politics|5: python { "acc": 0.9792746113989638, "acc_stderr": 0.010281417011909039, "acc_norm": 0.9792746113989638, "acc_norm_stderr": 0.010281417011909039 }

• harness|hendrycksTest-high_school_macroeconomics|5: python { "acc": 0.8128205128205128, "acc_stderr": 0.019776601086550036, "acc_norm": 0.81282