open-llm-leaderboard-old/details_capleaf__T-Llama

该数据集是在模型capleaf/T-Llama在Open LLM Leaderboard上的评估运行期间自动创建的。它由63个配置组成，每个配置对应一个被评估的任务。数据集是从1次运行中生成的，每次运行在每个配置中表示为特定的分割，使用运行的时间戳命名。train分割始终指向最新的结果。一个名为results的额外配置存储了运行的所有聚合结果，用于计算和显示Open LLM Leaderboard上的聚合指标。可以使用datasets库中的load_dataset函数加载该数据集。

原始信息汇总

数据集概述

数据集简介

该数据集是在对模型 capleaf/T-Llama 进行评估运行期间自动创建的，用于 Open LLM Leaderboard。

数据集组成

• 数据集包含 63 个配置，每个配置对应一个评估任务。
• 数据集从 1 次运行中创建，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
• "train" 分割始终指向最新的结果。
• 一个额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示聚合指标在 Open LLM Leaderboard 上。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_capleaf__T-Llama", "harness_winogrande_5", split="train")

最新结果

以下是 2024-04-15T22:59:17.772846 运行的最新结果：

python { "all": { "acc": 0.48260613994162815, "acc_stderr": 0.034597958283579845, "acc_norm": 0.4856702378089888, "acc_norm_stderr": 0.03532936344005532, "mc1": 0.3072215422276622, "mc1_stderr": 0.016150201321323006, "mc2": 0.464708469527708, "mc2_stderr": 0.015628743915417735 }, "harness|arc:challenge|25": { "acc": 0.5034129692832765, "acc_stderr": 0.014611050403244077, "acc_norm": 0.5418088737201365, "acc_norm_stderr": 0.0145602203087147 }, "harness|hellaswag|10": { "acc": 0.5784704242182832, "acc_stderr": 0.004927948061486069, "acc_norm": 0.7647878908583947, "acc_norm_stderr": 0.004232645108976139 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.3, "acc_stderr": 0.046056618647183814, "acc_norm": 0.3, "acc_norm_stderr": 0.046056618647183814 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.45185185185185184, "acc_stderr": 0.04299268905480864, "acc_norm": 0.45185185185185184, "acc_norm_stderr": 0.04299268905480864 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.4473684210526316, "acc_stderr": 0.0404633688397825, "acc_norm": 0.4473684210526316, "acc_norm_stderr": 0.0404633688397825 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.55, "acc_stderr": 0.05, "acc_norm": 0.55, "acc_norm_stderr": 0.05 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.47547169811320755, "acc_stderr": 0.030735822206205615, "acc_norm": 0.47547169811320755, "acc_norm_stderr": 0.030735822206205615 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.4375, "acc_stderr": 0.04148415739394154, "acc_norm": 0.4375, "acc_norm_stderr": 0.04148415739394154 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.33, "acc_stderr": 0.04725815626252604, "acc_norm": 0.33, "acc_norm_stderr": 0.04725815626252604 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.44, "acc_stderr": 0.04988876515698589, "acc_norm": 0.44, "acc_norm_stderr": 0.04988876515698589 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.39, "acc_stderr": 0.04902071300001975, "acc_norm": 0.39, "acc_norm_stderr": 0.04902071300001975 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.42196531791907516, "acc_stderr": 0.0376574669386515, "acc_norm": 0.42196531791907516, "acc_norm_stderr": 0.0376574669386515 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.20588235294117646, "acc_stderr": 0.04023382273617746, "acc_norm": 0.20588235294117646, "acc_norm_stderr": 0.04023382273617746 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.57, "acc_stderr": 0.049756985195624284, "acc_norm": 0.57, "acc_norm_stderr": 0.049756985195624284 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.4595744680851064, "acc_stderr": 0.03257901482099835, "acc_norm": 0.4595744680851064, "acc_norm_stderr": 0.03257901482099835 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.32456140350877194, "acc_stderr": 0.044045561573747664, "acc_norm": 0.32456140350877194, "acc_norm_stderr": 0.044045561573747664 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.4689655172413793, "acc_stderr": 0.04158632762097828, "acc_norm": 0.4689655172413793, "acc_norm_stderr": 0.04158632762097828 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.30952380952380953, "acc_stderr": 0.023809523809523864, "acc_norm": 0.30952380952380953, "acc_norm_stderr": 0.023809523809523864 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.3253968253968254, "acc_stderr": 0.04190596438871136, "acc_norm": 0.3253968253968254, "acc_norm_stderr": 0.04190596438871136 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.34, "acc_stderr": 0.047609522856952344, "acc_norm": 0.34, "acc_norm_stderr": 0.047609522856952344 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.5225806451612903, "acc_stderr": 0.02841498501970786, "acc_norm": 0.5225806451612903, "acc_norm_stderr": 0.02841498501970786 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.3103448275862069, "acc_stderr": 0.03255086769970104, "acc_norm": 0.3103448275862069, "acc_norm_stderr": 0.03255086769970104 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.48, "acc_stderr": 0.050211673156867795, "acc_norm": 0.48, "acc_norm_stderr": 0.050211673156867795 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.6363636363636364, "acc_stderr": 0.03756335775187897, "acc_norm": 0.6363636363636364, "acc_norm_stderr": 0.03756335775187897 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.5757575757575758, "acc_stderr": 0.03521224908841585, "acc_norm": 0.5757575757575758, "acc_norm_stderr": 0.03521224908841585 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.7512953367875648, "acc_stderr": 0.0311958408777003, "acc_norm": 0.7512953367875648, "acc_norm_stderr": 0.0311958408777003 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.4564102564102564, "acc_stderr": 0.025254485424799605, "acc_norm": 0.4564102564102564, "acc_norm_stderr": 0.025254485424799605 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.28888888888888886, "acc_stderr": 0.02763490726417854, "acc_norm": 0.28888888888888886, "acc_norm_stderr": 0.02763490726417854 }, "harness|hendrycksTest-high_school_microeconomics|5": { "acc": 0.4411764705882353, "acc_stderr