open-llm-leaderboard-old/details_tushar310__MisGemma-7B

该数据集是在模型 tushar310/MisGemma-7B 在 Open LLM Leaderboard 上的评估运行期间自动创建的。数据集由 63 个配置组成，每个配置对应一个被评估的任务。数据集是从 3 次运行中生成的，每次运行在每个配置中表示为特定的分割。train 分割始终指向最新的结果。一个名为 results 的额外配置存储了所有运行的聚合结果，这些结果用于计算和显示 Open LLM Leaderboard 上的聚合指标。README 还提供了一个示例，展示了如何使用 `datasets` 库中的 `load_dataset` 函数加载运行中的详细信息。

原始信息汇总

数据集概述

数据集创建

• 该数据集是在评估模型 tushar310/MisGemma-7B 在 Open LLM Leaderboard 上的自动创建的。

数据集结构

• 数据集包含 63 个配置，每个配置对应一个评估任务。
• 数据集从 3 次运行中创建，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
• "train" 分割始终指向最新的结果。
• 一个额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示在 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_tushar310__MisGemma-7B", "harness_winogrande_5", split="train")

最新结果

• 最新结果来自 2024-03-14T17:11:47.572511 运行，包含多个任务的评估结果。

配置详情

• 数据集包含多个配置，每个配置对应不同的评估任务和运行结果。例如：
  • harness_arc_challenge_25
  • harness_gsm8k_5
  • harness_hellaswag_10
  • harness_hendrycksTest_5

数据文件路径

• 每个配置包含多个数据文件，每个文件对应不同的分割和时间戳。例如：
  • harness_arc_challenge_25 配置包含以下数据文件：
    • details_harness|arc:challenge|25_2024-03-14T16-41-43.215251.parquet
    • details_harness|arc:challenge|25_2024-03-14T16-48-19.540142.parquet
    • details_harness|arc:challenge|25_2024-03-14T17-11-47.572511.parquet
  • harness_gsm8k_5 配置包含以下数据文件：
    • details_harness|gsm8k|5_2024-03-14T16-41-43.215251.parquet
    • details_harness|gsm8k|5_2024-03-14T16-48-19.540142.parquet
    • details_harness|gsm8k|5_2024-03-14T17-11-47.572511.parquet

评估结果示例

• 以下是部分评估结果的示例： python { "all": { "acc": 0.6464595284102798, "acc_stderr": 0.032387861365881954, "acc_norm": 0.6488023822477677, "acc_norm_stderr": 0.03303715256874037, "mc1": 0.46266829865361075, "mc1_stderr": 0.01745464515097059, "mc2": 0.6221649218559862, "mc2_stderr": 0.015475153453510542 }, "harness|arc:challenge|25": { "acc": 0.6331058020477816, "acc_stderr": 0.014084133118104296, "acc_norm": 0.6689419795221843, "acc_norm_stderr": 0.01375206241981783 }, "harness|hellaswag|10": { "acc": 0.6742680740888269, "acc_stderr": 0.004676898861978911, "acc_norm": 0.8570005974905397, "acc_norm_stderr": 0.0034935679140932923 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.41, "acc_stderr": 0.049431107042371025, "acc_norm": 0.41, "acc_norm_stderr": 0.049431107042371025 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.6444444444444445, "acc_stderr": 0.04135176749720385, "acc_norm": 0.6444444444444445, "acc_norm_stderr": 0.04135176749720385 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.6842105263157895, "acc_stderr": 0.0378272898086547, "acc_norm": 0.6842105263157895, "acc_norm_stderr": 0.0378272898086547 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.6, "acc_stderr": 0.04923659639173309, "acc_norm": 0.6, "acc_norm_stderr": 0.04923659639173309 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.7018867924528301, "acc_stderr": 0.028152837942493857, "acc_norm": 0.7018867924528301, "acc_norm_stderr": 0.028152837942493857 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.7638888888888888, "acc_stderr": 0.03551446610810826, "acc_norm": 0.7638888888888888, "acc_norm_stderr": 0.03551446610810826 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.51, "acc_stderr": 0.05024183937956912, "acc_norm": 0.51, "acc_norm_stderr": 0.05024183937956912 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.52, "acc_stderr": 0.050211673156867795, "acc_norm": 0.52, "acc_norm_stderr": 0.050211673156867795 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.31, "acc_stderr": 0.04648231987117316, "acc_norm": 0.31, "acc_norm_stderr": 0.04648231987117316 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.6705202312138728, "acc_stderr": 0.03583901754736412, "acc_norm": 0.6705202312138728, "acc_norm_stderr": 0.03583901754736412 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.4215686274509804, "acc_stderr": 0.04913595201274498, "acc_norm": 0.4215686274509804, "acc_norm_stderr": 0.04913595201274498 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.8, "acc_stderr": 0.04020151261036846, "acc_norm": 0.8, "acc_norm_stderr": 0.04020151261036846 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.5787234042553191, "acc_stderr": 0.03227834510146268, "acc_norm": 0.5787234042553191, "acc_norm_stderr": 0.03227834510146268 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.49122807017543857, "acc_stderr": 0.04702880432049615, "acc_norm": 0.49122807017543857, "acc_norm_stderr": 0.04702880432049615 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5655172413793104, "acc_stderr": 0.04130740879555498, "acc_norm": 0.5655172413793104, "acc_norm_stderr": 0.04130740879555498 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.3968253968253968, "acc_stderr": 0.02519710107424648, "acc_norm": 0.3968253968253968, "acc_norm_stderr": 0.02519710107424648 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.42063492063492064, "acc_stderr": 0.04415438226743744, "acc_norm": 0.42063492063492064, "acc_norm_stderr": 0.04415438226743744 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.39, "acc_stderr": 0.04902071300001975, "acc_norm": 0.39, "acc_norm_stderr": 0.04902071300001975 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.7677419354838709, "acc_stderr": 0.024022256130308235, "acc_norm": 0.7677419354838709, "acc_norm_stderr": 0.024022256130308235 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.541871921182266, "acc_stderr": 0.03505630140785741, "acc_norm": 0.541871921182266, "acc_norm_stderr": 0.03505630140785741 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.67, "acc_stderr": 0.04725815626252609, "acc_norm": 0.67, "acc_norm_stderr": 0.04725815626252609 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.7636363636363637, "acc_stderr": 0.033175059300091805, "acc_norm": 0.7636363636363637, "acc_norm_stderr": 0.033175059300091805 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.803030303030303, "acc_stderr": 0.028335609732463362, "acc_norm": 0.803030303030303, "acc_norm_stderr": 0.028335609732463362 }, "harness|