open-llm-leaderboard/details_teknium__Mistral-Trismegistus-7B

该数据集是在评估模型teknium/Mistral-Trismegistus-7B时自动生成的，主要用于在Open LLM Leaderboard上展示模型的性能。数据集包含64个配置，每个配置对应一个评估任务。数据集由2次运行生成，每次运行的结果都存储为特定的分割，分割名称使用运行的时间戳。此外，数据集还包含一个名为"results"的配置，用于存储所有运行的聚合结果，并在Open LLM Leaderboard上展示聚合指标。

原始信息汇总

数据集概述

该数据集是在对模型 teknium/Mistral-Trismegistus-7B 进行评估运行期间自动创建的，用于 Open LLM Leaderboard。

数据集组成

• 数据集包含 64 个配置，每个配置对应一个评估任务。
• 数据集从 2 次运行中创建，每个运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
• "train" 分割始终指向最新的结果。
• 一个额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_teknium__Mistral-Trismegistus-7B", "harness_winogrande_5", split="train")

最新结果

以下是 2023-10-25T09:46:08.723071 运行的最新结果：

python { "all": { "em": 0.010591442953020135, "em_stderr": 0.0010483469790502314, "f1": 0.07238674496644287, "f1_stderr": 0.001675223530701393, "acc": 0.4004875617305928, "acc_stderr": 0.010548628211357203 }, "harness|drop|3": { "em": 0.010591442953020135, "em_stderr": 0.0010483469790502314, "f1": 0.07238674496644287, "f1_stderr": 0.001675223530701393 }, "harness|gsm8k|5": { "acc": 0.09931766489764973, "acc_stderr": 0.008238371412683985 }, "harness|winogrande|5": { "acc": 0.7016574585635359, "acc_stderr": 0.012858885010030421 } }

配置详情

• harness_arc_challenge_25

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|arc:challenge|25_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|arc:challenge|25_2023-10-12T08-45-24.509522.parquet

• harness_drop_3

  • 分割：2023_10_25T09_46_08.723071
    • 路径：**/details_harness|drop|3_2023-10-25T09-46-08.723071.parquet
  • 分割：latest
    • 路径：**/details_harness|drop|3_2023-10-25T09-46-08.723071.parquet

• harness_gsm8k_5

  • 分割：2023_10_25T09_46_08.723071
    • 路径：**/details_harness|gsm8k|5_2023-10-25T09-46-08.723071.parquet
  • 分割：latest
    • 路径：**/details_harness|gsm8k|5_2023-10-25T09-46-08.723071.parquet

• harness_hellaswag_10

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hellaswag|10_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hellaswag|10_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：
      • **/details_harness|hendrycksTest-abstract_algebra|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-anatomy|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-astronomy|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-business_ethics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-clinical_knowledge|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-college_biology|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-college_chemistry|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-college_computer_science|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-college_mathematics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-college_medicine|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-college_physics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-computer_security|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-conceptual_physics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-econometrics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-electrical_engineering|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-elementary_mathematics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-formal_logic|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-global_facts|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_biology|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_chemistry|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_computer_science|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_european_history|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_geography|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_mathematics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_microeconomics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_physics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_psychology|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_statistics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_us_history|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_world_history|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-human_aging|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-human_sexuality|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-international_law|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-jurisprudence|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-logical_fallacies|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-machine_learning|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-management|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-marketing|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-medical_genetics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-miscellaneous|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-moral_disputes|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-moral_scenarios|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-nutrition|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-philosophy|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-prehistory|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-professional_accounting|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-professional_law|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-professional_medicine|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-professional_psychology|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-public_relations|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-security_studies|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-sociology|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-us_foreign_policy|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-virology|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-world_religions|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：
      • **/details_harness|hendrycksTest-abstract_algebra|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-anatomy|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-astronomy|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-business_ethics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-clinical_knowledge|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-college_biology|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-college_chemistry|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-college_computer_science|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-college_mathematics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-college_medicine|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-college_physics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-computer_security|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-conceptual_physics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-econometrics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-electrical_engineering|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-elementary_mathematics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-formal_logic|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-global_facts|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_biology|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_chemistry|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_computer_science|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_european_history|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_geography|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_mathematics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_microeconomics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_physics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_psychology|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_statistics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_us_history|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-high_school_world_history|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-human_aging|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-human_sexuality|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-international_law|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-jurisprudence|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-logical_fallacies|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-machine_learning|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-management|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-marketing|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-medical_genetics|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-miscellaneous|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-moral_disputes|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-moral_scenarios|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-nutrition|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-philosophy|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-prehistory|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-professional_accounting|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-professional_law|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-professional_medicine|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-professional_psychology|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-public_relations|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-security_studies|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-sociology|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-us_foreign_policy|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-virology|5_2023-10-12T08-45-24.509522.parquet
      • **/details_harness|hendrycksTest-world_religions|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_abstract_algebra_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-abstract_algebra|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-abstract_algebra|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_anatomy_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-anatomy|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-anatomy|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_astronomy_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-astronomy|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-astronomy|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_business_ethics_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-business_ethics|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-business_ethics|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_clinical_knowledge_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-clinical_knowledge|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-clinical_knowledge|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_college_biology_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-college_biology|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-college_biology|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_college_chemistry_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-college_chemistry|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-college_chemistry|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_college_computer_science_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-college_computer_science|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-college_computer_science|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_college_mathematics_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-college_mathematics|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-college_mathematics|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_college_medicine_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-college_medicine|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-college_medicine|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_college_physics_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-college_physics|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-college_physics|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_computer_security_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-computer_security|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-computer_security|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_conceptual_physics_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-conceptual_physics|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-conceptual_physics|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_econometrics_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-econometrics|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-econometrics|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_electrical_engineering_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-electrical_engineering|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-electrical_engineering|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_elementary_mathematics_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-elementary_mathematics|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-elementary_mathematics|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_formal_logic_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-formal_logic|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-formal_logic|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_global_facts_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-global_facts|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-global_facts|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_high_school_biology_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-high_school_biology|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-high_school_biology|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_high_school_chemistry_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-high_school_chemistry|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-high_school_chemistry|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_high_school_computer_science_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-high_school_computer_science|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-high_school_computer_science|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_high_school_european_history_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-high_school_european_history|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-high_school_european_history|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_high_school_geography_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-high_school_geography|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-high_school_geography|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_high_school_government_and_politics_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_high_school_macroeconomics_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_high_school_mathematics_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-high_school_mathematics|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-high_school_mathematics|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_high_school_microeconomics_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_high_school_physics_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-high_school_physics|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-high_school_physics|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_high_school_psychology_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-high_school_psychology|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-high_school_psychology|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_high_school_statistics_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-high_school_statistics|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-high_school_statistics|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_high_school_us_history_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-high_school_us_history|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-high_school_us_history|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_high_school_world_history_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-high_school_world_history|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-high_school_world_history|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_human_aging_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-human_aging|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-human_aging|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_human_sexuality_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-human_sexuality|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-human_sexuality|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_international_law_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-international_law|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-international_law|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_jurisprudence_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-jurisprudence|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-jurisprudence|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_logical_fallacies_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-logical_fallacies|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-logical_fallacies|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_machine_learning_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-machine_learning|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-machine_learning|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_management_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-management|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-management|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_marketing_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-marketing|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-marketing|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_medical_genetics_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-medical_genetics|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-medical_genetics|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_miscellaneous_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-miscellaneous|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-miscellaneous|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_moral_disputes_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-moral_disputes|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-moral_disputes|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_moral_scenarios_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-moral_scenarios|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-moral_scenarios|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_nutrition_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-nutrition|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-nutrition|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_philosophy_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-philosophy|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-philosophy|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_prehistory_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-prehistory|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-prehistory|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_professional_accounting_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-professional_accounting|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-professional_accounting|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_professional_law_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-professional_law|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-professional_law|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_professional_medicine_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-professional_medicine|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-professional_medicine|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_professional_psychology_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-professional_psychology|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-professional_psychology|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_public_relations_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-public_relations|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-public_relations|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_security_studies_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-security_studies|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-security_studies|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_sociology_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-sociology|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-sociology|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_us_foreign_policy_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-us_foreign_policy|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-us_foreign_policy|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_virology_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-virology|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-virology|5_2023-10-12T08-45-24.509522.parquet

• harness_hendrycksTest_world_religions_5

  • 分割：2023_10_12T08_45_24.509522
    • 路径：**/details_harness|hendrycksTest-world_religions|5_2023-10-12T08-45-24.509522.parquet
  • 分割：latest
    • 路径：**/details_harness|hendrycksTest-world_religions|5_2023-10-12T08-45-24.509522.parquet

搜集汇总

数据集介绍

构建方式

在大型语言模型评估的广阔领域中，Open LLM Leaderboard 扮演着至关重要的角色，为模型性能提供了标准化度量。本数据集正是为记录模型 teknium/Mistral-Trismegistus-7B 在排行榜上的评估过程而自动创建的。其构建方式基于两次独立的评估运行，每次运行的结果被存储为特定配置下的独立数据分割，分割以运行时间戳命名，而 'train' 分割则始终指向最新结果。数据集包含 64 个配置，每个配置对应一个评估任务，另设一个名为 'results' 的配置专门存储所有聚合结果，用于计算并展示排行榜上的整体度量指标。

特点

该数据集的核心特点在于其结构化的任务组织与版本管理能力。通过将不同评估任务（如 ARC Challenge、DROP、GSM8K 等）分别封装为独立配置，研究者能够精准定位并分析模型在特定基准上的表现。每个配置内的时间戳分割忠实记录了历次评估的原始细节，确保了结果的可追溯性与复现性。此外，'train' 分割自动指向最新运行结果的设计，简化了持续追踪模型性能演变的流程，而 'results' 配置则提供了宏观的聚合视角，便于整体性能的横向对比。

使用方法

使用该数据集时，研究者可通过 Hugging Face 的 datasets 库灵活加载所需信息。例如，通过指定配置名称（如 'harness_winogrande_5'）和分割（'train'），即可获取最新评估的详细数据。若需回溯历史版本，则可通过时间戳分割名称加载对应运行的结果。数据集以 Parquet 格式存储，支持高效的列式读取与筛选，适用于大规模性能分析。加载后的数据可直接用于复现排行榜结果、绘制模型能力雷达图，或作为进一步元分析的基础，助力于理解模型在不同自然语言理解与推理任务上的优劣。

背景与挑战

背景概述

大语言模型能力的客观评估是推动该领域发展的关键环节，然而传统人工评测的高昂成本与主观偏差催生了标准化基准测试的需求。在此背景下，Hugging Face团队于2023年发起了开放大语言模型排行榜（Open LLM Leaderboard），旨在为社区提供一个透明、可复现的模型性能对比平台。该数据集正是为评估teknium团队开发的Mistral-Trismegistus-7B模型而自动生成的评测记录，由Clementine Fourrier等研究人员主导构建，通过集成语言模型评估框架（LM Evaluation Harness）对模型在ARC、HellaSwag、MMLU等多项经典基准上的表现进行系统化度量。这一评测体系不仅为Mistral-Trismegistus-7B模型提供了多维度的能力画像，更通过标准化流程树立了开源模型评估的范例，深刻影响了后续大语言模型的性能比较与迭代优化方向。

当前挑战

该数据集所承载的核心挑战在于如何构建一个既全面又公平的模型评估体系。从领域问题层面看，大语言模型在常识推理（如Winogrande任务）、数学求解（如GSM8K任务）以及阅读理解（如DROP任务）等异构任务上的表现差异悬殊，单一维度的评测无法反映模型真实能力，需要设计涵盖多领域、多难度的任务集合。从数据构建层面看，评测过程面临多重技术挑战：不同基准测试的提示格式与评估指标需统一规范，以确保结果可比性；模型推理时采样策略（如few-shot示例数量）的微小差异可能导致评分波动，需要严格固定超参数；此外，多次运行间的随机性控制、评测结果的可复现性保证，以及大规模评测任务的计算资源优化，均是构建可靠数据集必须攻克的难题。

常用场景

经典使用场景

在大规模语言模型评估领域，该数据集作为Open LLM Leaderboard的评测记录载体，其核心用途在于系统性地追踪和比较不同模型在多样化基准任务上的表现。数据集涵盖了ARC推理挑战、HellaSwag常识推理、MMLU多学科知识、GSM8K数学推理及Winogrande共指消解等64个经典测试配置，为研究者提供了标准化的模型性能对比平台。通过记录每次评测的详细日志与聚合结果，该数据集使得跨时间维度的模型演进分析成为可能，是衡量模型在知识理解、逻辑推理和语言生成等维度综合能力的权威工具。

衍生相关工作

基于该数据集衍生了多项具有影响力的研究工作。一方面，其详细评测日志催生了针对特定任务错误模式的分析，如通过剖析Mistral-Trismegistus-7B在DROP任务上1.06%的精确匹配率，研究者设计了增强数值推理能力的微调策略。另一方面，该数据集作为Open LLM Leaderboard的底层数据基础设施，支撑了Hugging Face社区中数千个模型的排行榜排名计算，激发了诸如模型融合、知识蒸馏等性能提升方法的广泛探索。此外，其多任务评测架构也启发了后续如HELM等综合评估框架的设计理念，推动了大模型评估从单一指标向多维能力矩阵的范式演进。

数据集最近研究

最新研究方向

在大语言模型评测领域，Open LLM Leaderboard已成为衡量模型性能的黄金标准。Mistral-Trismegistus-7B作为一款经微调优化的7B参数模型，其在排行榜上的评估数据揭示了当前研究的前沿方向：从单一任务评测向多维度、细粒度能力评估的范式转变。该数据集通过64个配置覆盖了ARC挑战、DROP、GSM8K、HellaSwag及涵盖57个学科的MMLU等核心基准，尤其关注模型在数学推理（GSM8K准确率9.93%）、常识推理（Winogrande准确率70.17%）和阅读理解（DROP的F1值7.24%）等关键能力上的表现。这一评估体系不仅反映了研究者对模型泛化能力与知识广度的双重追求，更推动了社区对微调策略、数据质量与评测公平性的深入探讨，为开源大模型的迭代优化提供了重要参考。