open-llm-leaderboard-old/details_aboros98__merlin1

该数据集是在Open LLM Leaderboard上对模型aboros98/merlin1进行评估时自动创建的。它由63个配置组成，每个配置对应一个评估任务。数据集是从1次运行中生成的，每次运行在每个配置中表示为特定的分割。train分割始终指向最新的结果。一个额外的配置results存储了所有运行的聚合结果，用于计算和显示Open LLM Leaderboard上的聚合指标。可以使用Hugging Face的datasets库加载数据集，README中提供了特定运行的最新结果。

原始信息汇总

数据集概述

数据集简介

该数据集是在评估模型 aboros98/merlin1 在 Open LLM Leaderboard 上的自动创建的。数据集包含 63 个配置，每个配置对应一个评估任务。

数据集结构

数据集从 1 次运行中创建，每个运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。"train" 分割始终指向最新的结果。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_aboros98__merlin1", "harness_winogrande_5", split="train")

最新结果

以下是 2024-03-12T17:04:34.752225 运行的最新结果：

python { "all": { "acc": 0.5803631365007319, "acc_stderr": 0.03379683671031974, "acc_norm": 0.5819117908480674, "acc_norm_stderr": 0.034490432887019654, "mc1": 0.3292533659730722, "mc1_stderr": 0.016451264440068235, "mc2": 0.4834841796846446, "mc2_stderr": 0.01517611623380428 }, "harness|arc:challenge|25": { "acc": 0.5802047781569966, "acc_stderr": 0.014422181226303028, "acc_norm": 0.606655290102389, "acc_norm_stderr": 0.014275101465693028 }, "harness|hellaswag|10": { "acc": 0.5592511451902011, "acc_stderr": 0.004954622308738999, "acc_norm": 0.7454690300736905, "acc_norm_stderr": 0.004347070019527476 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.29, "acc_stderr": 0.045604802157206845, "acc_norm": 0.29, "acc_norm_stderr": 0.045604802157206845 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.4444444444444444, "acc_stderr": 0.04292596718256981, "acc_norm": 0.4444444444444444, "acc_norm_stderr": 0.04292596718256981 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.5460526315789473, "acc_stderr": 0.04051646342874142, "acc_norm": 0.5460526315789473, "acc_norm_stderr": 0.04051646342874142 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.58, "acc_stderr": 0.049604496374885836, "acc_norm": 0.58, "acc_norm_stderr": 0.049604496374885836 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.5735849056603773, "acc_stderr": 0.030437794342983052, "acc_norm": 0.5735849056603773, "acc_norm_stderr": 0.030437794342983052 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.6180555555555556, "acc_stderr": 0.040629907841466674, "acc_norm": 0.6180555555555556, "acc_norm_stderr": 0.040629907841466674 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.41, "acc_stderr": 0.049431107042371025, "acc_norm": 0.41, "acc_norm_stderr": 0.049431107042371025 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.46, "acc_stderr": 0.05009082659620332, "acc_norm": 0.46, "acc_norm_stderr": 0.05009082659620332 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.41, "acc_stderr": 0.04943110704237102, "acc_norm": 0.41, "acc_norm_stderr": 0.04943110704237102 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.630057803468208, "acc_stderr": 0.036812296333943194, "acc_norm": 0.630057803468208, "acc_norm_stderr": 0.036812296333943194 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.38235294117647056, "acc_stderr": 0.04835503696107224, "acc_norm": 0.38235294117647056, "acc_norm_stderr": 0.04835503696107224 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.73, "acc_stderr": 0.04461960433384739, "acc_norm": 0.73, "acc_norm_stderr": 0.04461960433384739 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.502127659574468, "acc_stderr": 0.03268572658667492, "acc_norm": 0.502127659574468, "acc_norm_stderr": 0.03268572658667492 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.35964912280701755, "acc_stderr": 0.045144961328736334, "acc_norm": 0.35964912280701755, "acc_norm_stderr": 0.045144961328736334 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5448275862068965, "acc_stderr": 0.04149886942192118, "acc_norm": 0.5448275862068965, "acc_norm_stderr": 0.04149886942192118 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.4603174603174603, "acc_stderr": 0.025670080636909193, "acc_norm": 0.4603174603174603, "acc_norm_stderr": 0.025670080636909193 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.36507936507936506, "acc_stderr": 0.04306241259127153, "acc_norm": 0.36507936507936506, "acc_norm_stderr": 0.04306241259127153 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.42, "acc_stderr": 0.049604496374885836, "acc_norm": 0.42, "acc_norm_stderr": 0.049604496374885836 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.6870967741935484, "acc_stderr": 0.026377567028645854, "acc_norm": 0.6870967741935484, "acc_norm_stderr": 0.026377567028645854 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.4729064039408867, "acc_stderr": 0.03512819077876106, "acc_norm": 0.4729064039408867, "acc_norm_stderr": 0.03512819077876106 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.65, "acc_stderr": 0.047937248544110196, "acc_norm": 0.65, "acc_norm_stderr": 0.047937248544110196 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.696969696969697, "acc_stderr": 0.03588624800091707, "acc_norm": 0.696969696969697, "acc_norm_stderr": 0.03588624800091707 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.7424242424242424, "acc_stderr": 0.03115626951964683, "acc_norm": 0.7424242424242424, "acc_norm_stderr": 0.03115626951964683 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.7927461139896373, "acc_stderr": 0.029252823291803624, "acc_norm": 0.7927461139896373, "acc_norm_stderr": 0.029252823291803624 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.5717948717948718, "acc_stderr": 0.025088301454694834, "acc_norm": 0.5717948717948718, "acc_norm_stderr": 0.025088301454694834 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.34444444444444444, "acc_stderr": 0.02897264888484427, "acc_norm": 0.34444444444444444, "acc_norm_stderr": 0.02897264888484427 }, "harness|hendrycksTest-high_school_microeconomics|5": { "acc": 0.6386554621848739, "acc_stderr": 0.03