open-llm-leaderboard-old/details_fblgit__LUNA-SOLARkrautLM-Instruct

该数据集是在模型fblgit/LUNA-SOLARkrautLM-Instruct在Open LLM Leaderboard上的评估运行期间自动创建的。数据集由63个配置组成，每个配置对应一个被评估的任务。它包含一次运行的结果，每次运行在每个配置中表示为特定的分割。train分割始终指向最新的结果。一个名为results的额外配置存储了运行的所有聚合结果，这些结果用于计算和显示Open LLM Leaderboard上的聚合指标。README还提供了一个如何使用Python中的datasets库加载运行细节的示例。

原始信息汇总

数据集概述

数据集摘要

该数据集是在评估模型 fblgit/LUNA-SOLARkrautLM-Instruct 在 Open LLM Leaderboard 上的自动创建的。数据集由 63 个配置组成，每个配置对应一个评估任务。

数据集结构

• 配置数量：63
• 创建来源：1 次运行
• 分割命名：每个运行的分割以运行的时间戳命名，"train" 分割始终指向最新结果。
• 额外配置："results" 存储所有运行的聚合结果，用于计算和显示聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_fblgit__LUNA-SOLARkrautLM-Instruct", "harness_winogrande_5", split="train")

最新结果

以下是 2023-12-27T13:04:58.261893 运行的最新结果：

python { "all": { "acc": 0.6642541203854099, "acc_stderr": 0.0317093464542955, "acc_norm": 0.6656901555387255, "acc_norm_stderr": 0.03234983203431538, "mc1": 0.5826193390452876, "mc1_stderr": 0.017262891063272164, "mc2": 0.7336752254501507, "mc2_stderr": 0.014886399154960954 }, "harness|arc:challenge|25": { "acc": 0.6868600682593856, "acc_stderr": 0.013552671543623497, "acc_norm": 0.71160409556314, "acc_norm_stderr": 0.013238394422428173 }, "harness|hellaswag|10": { "acc": 0.7130053774148576, "acc_stderr": 0.004514345547780332, "acc_norm": 0.8827922724556861, "acc_norm_stderr": 0.003210102507177248 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.4, "acc_stderr": 0.049236596391733084, "acc_norm": 0.4, "acc_norm_stderr": 0.049236596391733084 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.6222222222222222, "acc_stderr": 0.04188307537595853, "acc_norm": 0.6222222222222222, "acc_norm_stderr": 0.04188307537595853 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.743421052631579, "acc_stderr": 0.0355418036802569, "acc_norm": 0.743421052631579, "acc_norm_stderr": 0.0355418036802569 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.78, "acc_stderr": 0.04163331998932261, "acc_norm": 0.78, "acc_norm_stderr": 0.04163331998932261 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.6830188679245283, "acc_stderr": 0.02863723563980089, "acc_norm": 0.6830188679245283, "acc_norm_stderr": 0.02863723563980089 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.7708333333333334, "acc_stderr": 0.03514697467862388, "acc_norm": 0.7708333333333334, "acc_norm_stderr": 0.03514697467862388 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.48, "acc_stderr": 0.050211673156867795, "acc_norm": 0.48, "acc_norm_stderr": 0.050211673156867795 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.52, "acc_stderr": 0.05021167315686779, "acc_norm": 0.52, "acc_norm_stderr": 0.05021167315686779 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.27, "acc_stderr": 0.044619604333847394, "acc_norm": 0.27, "acc_norm_stderr": 0.044619604333847394 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.6589595375722543, "acc_stderr": 0.03614665424180826, "acc_norm": 0.6589595375722543, "acc_norm_stderr": 0.03614665424180826 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.37254901960784315, "acc_stderr": 0.048108401480826346, "acc_norm": 0.37254901960784315, "acc_norm_stderr": 0.048108401480826346 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.74, "acc_stderr": 0.04408440022768078, "acc_norm": 0.74, "acc_norm_stderr": 0.04408440022768078 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.625531914893617, "acc_stderr": 0.03163910665367291, "acc_norm": 0.625531914893617, "acc_norm_stderr": 0.03163910665367291 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.5087719298245614, "acc_stderr": 0.04702880432049615, "acc_norm": 0.5087719298245614, "acc_norm_stderr": 0.04702880432049615 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.6413793103448275, "acc_stderr": 0.039966295748767186, "acc_norm": 0.6413793103448275, "acc_norm_stderr": 0.039966295748767186 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.4894179894179894, "acc_stderr": 0.025745542276045478, "acc_norm": 0.4894179894179894, "acc_norm_stderr": 0.025745542276045478 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.4523809523809524, "acc_stderr": 0.044518079590553275, "acc_norm": 0.4523809523809524, "acc_norm_stderr": 0.044518079590553275 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.35, "acc_stderr": 0.047937248544110196, "acc_norm": 0.35, "acc_norm_stderr": 0.047937248544110196 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.8161290322580645, "acc_stderr": 0.022037217340267826, "acc_norm": 0.8161290322580645, "acc_norm_stderr": 0.022037217340267826 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.5270935960591133, "acc_stderr": 0.03512819077876106, "acc_norm": 0.5270935960591133, "acc_norm_stderr": 0.03512819077876106 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.7, "acc_stderr": 0.046056618647183814, "acc_norm": 0.7, "acc_norm_stderr": 0.046056618647183814 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.806060606060606, "acc_stderr": 0.03087414513656209, "acc_norm": 0.806060606060606, "acc_norm_stderr": 0.03087414513656209 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.8585858585858586, "acc_stderr": 0.024825909793343343, "acc_norm": 0.8585858585858586, "acc_norm_stderr": 0.024825909793343343 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.8808290155440415, "acc_stderr": 0.023381935348121437, "acc_norm": 0.8808290155440415, "acc_norm_stderr": 0.023381935348121437 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.6692307692307692, "acc_stderr": 0.02385479568097113, "acc_norm": 0.6692307692307692, "acc_norm_stderr": 0.02385479568097113 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.37777777777777777, "acc_stderr": 0.029560707392465715, "acc_norm": 0.37777777777777777, "acc_norm_stderr": 0.029560707392465715 }, "har

搜集汇总

背景与挑战

背景概述

该数据集是模型fblgit/LUNA-SOLARkrautLM-Instruct在Open LLM Leaderboard评估期间自动生成的评估结果集合，包含63个任务配置和一次运行的详细数据，其中'train'分割代表最新结果，并存储聚合指标用于排行榜计算。数据集可通过datasets库加载，便于用户分析和复现评估过程。