open-llm-leaderboard-old/details_perlthoughts__Chupacabra-7B-v2.04

该数据集是在模型 perlthoughts/Chupacabra-7B-v2.04 在 Open LLM Leaderboard 上进行评估时自动生成的。数据集包含 63 个配置，每个配置对应一个被评估的任务。数据集由一次运行生成，每次运行在每个配置中表示为特定的分割，分割名称由运行的时间戳命名。train 分割始终指向最新的结果。一个名为 results 的额外配置存储了所有运行的聚合结果，这些结果用于计算和显示 Open LLM Leaderboard 上的聚合指标。README 还提供了如何使用 Hugging Face datasets 库加载数据集的示例。

原始信息汇总

数据集概述

该数据集是在评估模型 perlthoughts/Chupacabra-7B-v2.04 的过程中自动创建的，用于 Open LLM Leaderboard。数据集包含 63 个配置，每个配置对应一个评估任务。

数据集结构

• 配置数量：63 个配置
• 数据来源：从 1 次运行中创建，每个配置包含特定分片，分片名称使用运行的时间戳。
• 最新结果："train" 分片始终指向最新结果。
• 汇总结果：额外配置 "results" 存储所有运行结果的汇总，用于计算和展示在 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_perlthoughts__Chupacabra-7B-v2.04", "harness_winogrande_5", split="train")

最新结果

以下是 2024-01-05T10:12:55.038964 运行 的最新结果：

python { "all": { "acc": 0.6117535002239652, "acc_stderr": 0.03305067073551487, "acc_norm": 0.6144598700035333, "acc_norm_stderr": 0.033716352758886466, "mc1": 0.5275397796817626, "mc1_stderr": 0.01747693019071219, "mc2": 0.6775807253391397, "mc2_stderr": 0.014911725947999506 }, "harness|arc:challenge|25": { "acc": 0.6220136518771331, "acc_stderr": 0.0141696645203031, "acc_norm": 0.6629692832764505, "acc_norm_stderr": 0.013813476652902276 }, "harness|hellaswag|10": { "acc": 0.6577375024895439, "acc_stderr": 0.004734972668299617, "acc_norm": 0.8570005974905397, "acc_norm_stderr": 0.0034935679140933006 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.3, "acc_stderr": 0.046056618647183814, "acc_norm": 0.3, "acc_norm_stderr": 0.046056618647183814 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.6, "acc_stderr": 0.04232073695151589, "acc_norm": 0.6, "acc_norm_stderr": 0.04232073695151589 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.6710526315789473, "acc_stderr": 0.038234289699266046, "acc_norm": 0.6710526315789473, "acc_norm_stderr": 0.038234289699266046 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.58, "acc_stderr": 0.049604496374885836, "acc_norm": 0.58, "acc_norm_stderr": 0.049604496374885836 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.6754716981132075, "acc_stderr": 0.02881561571343211, "acc_norm": 0.6754716981132075, "acc_norm_stderr": 0.02881561571343211 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.7013888888888888, "acc_stderr": 0.03827052357950756, "acc_norm": 0.7013888888888888, "acc_norm_stderr": 0.03827052357950756 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.45, "acc_stderr": 0.049999999999999996, "acc_norm": 0.45, "acc_norm_stderr": 0.049999999999999996 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.46, "acc_stderr": 0.05009082659620332, "acc_norm": 0.46, "acc_norm_stderr": 0.05009082659620332 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.32, "acc_stderr": 0.04688261722621504, "acc_norm": 0.32, "acc_norm_stderr": 0.04688261722621504 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.6416184971098265, "acc_stderr": 0.036563436533531585, "acc_norm": 0.6416184971098265, "acc_norm_stderr": 0.036563436533531585 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.4019607843137255, "acc_stderr": 0.04878608714466996, "acc_norm": 0.4019607843137255, "acc_norm_stderr": 0.04878608714466996 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.71, "acc_stderr": 0.04560480215720684, "acc_norm": 0.71, "acc_norm_stderr": 0.04560480215720684 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.5446808510638298, "acc_stderr": 0.03255525359340355, "acc_norm": 0.5446808510638298, "acc_norm_stderr": 0.03255525359340355 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.4298245614035088, "acc_stderr": 0.04657047260594963, "acc_norm": 0.4298245614035088, "acc_norm_stderr": 0.04657047260594963 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5724137931034483, "acc_stderr": 0.04122737111370333, "acc_norm": 0.5724137931034483, "acc_norm_stderr": 0.04122737111370333 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.3994708994708995, "acc_stderr": 0.025225450284067887, "acc_norm": 0.3994708994708995, "acc_norm_stderr": 0.025225450284067887 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.4126984126984127, "acc_stderr": 0.04403438954768176, "acc_norm": 0.4126984126984127, "acc_norm_stderr": 0.04403438954768176 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.36, "acc_stderr": 0.04824181513244218, "acc_norm": 0.36, "acc_norm_stderr": 0.04824181513244218 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.5903225806451613, "acc_stderr": 0.027976054915347357, "acc_norm": 0.5903225806451613, "acc_norm_stderr": 0.027976054915347357 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.4876847290640394, "acc_stderr": 0.035169204442208966, "acc_norm": 0.4876847290640394, "acc_norm_stderr": 0.035169204442208966 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.61, "acc_stderr": 0.04902071300001974, "acc_norm": 0.61, "acc_norm_stderr": 0.04902071300001974 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.7333333333333333, "acc_stderr": 0.03453131801885416, "acc_norm": 0.7333333333333333, "acc_norm_stderr": 0.03453131801885416 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.7474747474747475, "acc_stderr": 0.03095405547036589, "acc_norm": 0.7474747474747475, "acc_norm_stderr": 0.03095405547036589 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.8549222797927462, "acc_stderr": 0.025416343096306422, "acc_norm": 0.8549222797927462, "acc_norm_stderr": 0.025416343096306422 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.6051282051282051, "acc_stderr": 0.0247843169421564, "acc_norm": 0.6051282051282051, "acc_norm_stderr": 0.0247843169421564 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.3037037037037037, "acc_stderr": 0.02803792996911499, "acc_norm": 0.3037037037037037, "acc_norm_stderr": 0.02803792996911499 }, "harness|hendrycksTest-high