open-llm-leaderboard-old/details_janhq__Mistral-7B-Instruct-v0.2-DARE

该数据集是在Open LLM Leaderboard上对模型janhq/Mistral-7B-Instruct-v0.2-DARE进行评估时自动创建的。数据集由63个配置组成，每个配置对应一个评估任务。数据集包含一次运行的结果，每次运行在每个配置中表示为特定的分割，train分割始终指向最新结果。此外，名为results的配置存储了所有运行的聚合结果，用于计算和显示Open LLM Leaderboard上的聚合指标。README还提供了如何使用`datasets`库中的`load_dataset`函数加载运行细节的示例。

原始信息汇总

数据集概述

数据集摘要

该数据集是在对模型 janhq/Mistral-7B-Instruct-v0.2-DARE 进行评估运行期间自动创建的，用于 Open LLM Leaderboard。

数据集组成

• 该数据集包含 63 个配置，每个配置对应一个评估任务。
• 数据集从 1 次运行中创建，每次运行可以在每个配置中找到特定的拆分，拆分名称使用运行的时间戳。
• "train" 拆分始终指向最新的结果。
• 一个额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_janhq__Mistral-7B-Instruct-v0.2-DARE", "harness_winogrande_5", split="train")

最新结果

以下是 2023-12-12T11:22:55.278603 运行的最新结果：

python { "all": { "acc": 0.5002671303432286, "acc_stderr": 0.03440023987934237, "acc_norm": 0.506255828811682, "acc_norm_stderr": 0.035162947112250174, "mc1": 0.3953488372093023, "mc1_stderr": 0.017115815632418194, "mc2": 0.5435910325313378, "mc2_stderr": 0.015385871725485683 }, "harness|arc:challenge|25": { "acc": 0.5580204778156996, "acc_stderr": 0.014512682523128342, "acc_norm": 0.6194539249146758, "acc_norm_stderr": 0.014188277712349814 }, "harness|hellaswag|10": { "acc": 0.5338577972515435, "acc_stderr": 0.0049783281907755245, "acc_norm": 0.7562238597888866, "acc_norm_stderr": 0.004284817238406704 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.29, "acc_stderr": 0.04560480215720684, "acc_norm": 0.29, "acc_norm_stderr": 0.04560480215720684 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.4962962962962963, "acc_stderr": 0.04319223625811331, "acc_norm": 0.4962962962962963, "acc_norm_stderr": 0.04319223625811331 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.5986842105263158, "acc_stderr": 0.03988903703336285, "acc_norm": 0.5986842105263158, "acc_norm_stderr": 0.03988903703336285 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.51, "acc_stderr": 0.05024183937956912, "acc_norm": 0.51, "acc_norm_stderr": 0.05024183937956912 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.569811320754717, "acc_stderr": 0.030471445867183235, "acc_norm": 0.569811320754717, "acc_norm_stderr": 0.030471445867183235 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.5763888888888888, "acc_stderr": 0.041321250197233685, "acc_norm": 0.5763888888888888, "acc_norm_stderr": 0.041321250197233685 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.4, "acc_stderr": 0.049236596391733084, "acc_norm": 0.4, "acc_norm_stderr": 0.049236596391733084 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.42, "acc_stderr": 0.049604496374885836, "acc_norm": 0.42, "acc_norm_stderr": 0.049604496374885836 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.31, "acc_stderr": 0.04648231987117316, "acc_norm": 0.31, "acc_norm_stderr": 0.04648231987117316 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5433526011560693, "acc_stderr": 0.03798106566014498, "acc_norm": 0.5433526011560693, "acc_norm_stderr": 0.03798106566014498 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.37254901960784315, "acc_stderr": 0.04810840148082635, "acc_norm": 0.37254901960784315, "acc_norm_stderr": 0.04810840148082635 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.62, "acc_stderr": 0.04878317312145633, "acc_norm": 0.62, "acc_norm_stderr": 0.04878317312145633 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.48936170212765956, "acc_stderr": 0.03267862331014063, "acc_norm": 0.48936170212765956, "acc_norm_stderr": 0.03267862331014063 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.38596491228070173, "acc_stderr": 0.04579639422070434, "acc_norm": 0.38596491228070173, "acc_norm_stderr": 0.04579639422070434 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5310344827586206, "acc_stderr": 0.04158632762097828, "acc_norm": 0.5310344827586206, "acc_norm_stderr": 0.04158632762097828 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.3783068783068783, "acc_stderr": 0.02497695405315526, "acc_norm": 0.3783068783068783, "acc_norm_stderr": 0.02497695405315526 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.2619047619047619, "acc_stderr": 0.03932537680392871, "acc_norm": 0.2619047619047619, "acc_norm_stderr": 0.03932537680392871 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.35, "acc_stderr": 0.047937248544110196, "acc_norm": 0.35, "acc_norm_stderr": 0.047937248544110196 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.47419354838709676, "acc_stderr": 0.02840609505765332, "acc_norm": 0.47419354838709676, "acc_norm_stderr": 0.02840609505765332 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.35960591133004927, "acc_stderr": 0.03376458246509567, "acc_norm": 0.35960591133004927, "acc_norm_stderr": 0.03376458246509567 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.53, "acc_stderr": 0.050161355804659205, "acc_norm": 0.53, "acc_norm_stderr": 0.050161355804659205 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.28484848484848485, "acc_stderr": 0.03524390844511784, "acc_norm": 0.28484848484848485, "acc_norm_stderr": 0.03524390844511784 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.6666666666666666, "acc_stderr": 0.033586181457325226, "acc_norm": 0.6666666666666666, "acc_norm_stderr": 0.033586181457325226 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.7564766839378239, "acc_stderr": 0.030975436386845436, "acc_norm": 0.7564766839378239, "acc_norm_stderr": 0.030975436386845436 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.5153846153846153, "acc_stderr": 0.02533900301010651, "acc_norm": 0.5153846153846153, "acc_norm_stderr": 0.02533900301010651 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.2814814814814815, "acc_stderr": 0.027420019350945277,