open-llm-leaderboard/details_invalid-coder__Starling-LM-7B-beta-laser-dpo

该数据集是在评估模型invalid-coder/Starling-LM-7B-beta-laser-dpo时自动创建的，评估过程在Open LLM Leaderboard上进行。数据集由63个配置组成，每个配置对应一个评估任务。数据集是从1次运行中创建的，每次运行都可以在特定配置中找到，运行的时间戳用于命名分割。train分割始终指向最新的结果。此外，results配置存储了所有运行的聚合结果，并用于计算和显示Open LLM Leaderboard上的聚合指标。

原始信息汇总

数据集概述

数据集名称: Evaluation run of invalid-coder/Starling-LM-7B-beta-laser-dpo

创建目的: 该数据集是自动创建的，用于评估模型invalid-coder/Starling-LM-7B-beta-laser-dpo在Open LLM Leaderboard上的性能。

数据集构成:

• 配置数量: 63个
• 每个配置: 对应一个评估任务
• 数据来源: 来自1次运行
• 数据分割: 每个配置中包含特定分割，分割名称使用运行的时间戳命名
• 额外配置: "results"，存储所有运行的聚合结果，用于计算和显示聚合指标

数据集加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_invalid-coder__Starling-LM-7B-beta-laser-dpo", "harness_winogrande_5", split="train")

最新结果

• 结果来源: 来自2024-03-29T18:53:46.963162的运行
• 结果内容: 包含多个任务的评估结果，如准确率(acc)、标准误差(acc_stderr)等
• 示例结果:
  • harness|arc:challenge|25: acc=0.633959, acc_norm=0.674061
  • harness|hellaswag|10: acc=0.642501, acc_norm=0.833798
  • harness|hendrycksTest-abstract_algebra|5: acc=0.35, acc_norm=0.35
  • harness|hendrycksTest-anatomy|5: acc=0.644444, acc_norm=0.644444
  • harness|hendrycksTest-astronomy|5: acc=0.710526, acc_norm=0.710526
  • harness|hendrycksTest-business_ethics|5: acc=0.68, acc_norm=0.68
  • harness|hendrycksTest-clinical_knowledge|5: acc=0.709434, acc_norm=0.709434
  • harness|hendrycksTest-college_biology|5: acc=0.777778, acc_norm=0.777778
  • harness|hendrycksTest-college_chemistry|5: acc=0.49, acc_norm=0.49
  • harness|hendrycksTest-college_computer_science|5: acc=0.57, acc_norm=0.57
  • harness|hendrycksTest-college_mathematics|5: acc=0.39, acc_norm=0.39
  • harness|hendrycksTest-college_medicine|5: acc=0.670520, acc_norm=0.670520
  • harness|hendrycksTest-college_physics|5: acc=0.362745, acc_norm=0.362745
  • harness|hendrycksTest-computer_security|5: acc=0.73, acc_norm=0.73
  • harness|hendrycksTest-conceptual_physics|5: acc=0.578723, acc_norm=0.578723
  • harness|hendrycksTest-econometrics|5: acc=0.464912, acc_norm=0.464912
  • harness|hendrycksTest-electrical_engineering|5: acc=0.586207, acc_norm=0.586207
  • harness|hendrycksTest-elementary_mathematics|5: acc=0.420635, acc_norm=0.420635
  • harness|hendrycksTest-formal_logic|5: acc=0.515873, acc_norm=0.515873
  • harness|hendrycksTest-global_facts|5: acc=0.29, acc_norm=0.29
  • harness|hendrycksTest-high_school_biology|5: acc=0.777419, acc_norm=0.777419
  • harness|hendrycksTest-high_school_chemistry|5: acc=0.497537, acc_norm=0.497537
  • harness|hendrycksTest-high_school_computer_science|5: acc=0.72, acc_norm=0.72
  • harness|hendrycksTest-high_school_european_history|5: acc=0.812121, acc_norm=0.812121
  • harness|hendrycksTest-high_school_geography|5: acc=0.767677, acc_norm=0.767677
  • harness|hendrycksTest-high_school_government_and_politics|5: acc=0.891192, acc_norm=0.891192
  • harness|hendrycksTest-high_school_macroeconomics|5: acc=0.669231, acc_norm=0.669231
  • harness|hendrycksTest-high_school_mathematics|5: acc=0.366667, acc_norm=0.366667
  • harness|hendrycksTest-high_school_microeconomics|5: acc=0.689076, acc_norm=0.689076
  • harness|hendrycksTest-high_school_physics|5: acc=0.344371, acc_norm=0.344371
  • harness|hendrycksTest-high_school_psychology|5: acc=0.844037, acc_norm=0.844037
  • harness|hendrycksTest-high_school_statistics|5: acc=0.509259, acc_norm=0.509259
  • harness|hendrycksTest-high_school_us_history|5: acc=0.833333, acc_norm=0.833333
  • harness|hendrycksTest-high_school_world_history|5: acc=0.810127, acc_norm=0.810127
  • harness|hendrycksTest-human_aging|5: acc=0.704036, acc_norm=0.704036
  • harness|hendrycksTest-human_sexuality|5: acc=0.786260, acc_norm=0.786260
  • harness|hendrycksTest-international_law|5: acc=0.818182, acc_norm=0.818182
  • harness|hendrycksTest-jurisprudence|5: acc=0.768519, acc_norm=0.768519
  • harness|hendrycksTest-logical_fallacies|5: acc=0.791411, acc_norm=0.791411
  • harness|hendrycksTest-machine_learning|5: acc=0.473214, acc_norm=0.473214
  • harness|hendrycksTest-management|5: acc=0.834951, acc_norm=0.834951
  • harness|hendrycksTest-marketing|5: acc=0.893162, acc_norm=0.893162
  • harness|hendrycksTest-medical_genetics|5: acc=0.78, acc_norm=0.78
  • harness|hendrycksTest-miscellaneous|5: acc=0.830140, acc_norm=0.830140
  • harness|hendrycksTest-moral_disputes|5: acc=0.754335, acc_norm=0.754335
  • harness|hendrycksTest-moral_scenarios|5: acc=0.264804, acc_norm=0.264804
  • harness|hendrycksTest-nutrition|5: acc=0.738562, acc_norm=0.738562
  • harness|hendrycksTest-philosophy|5: acc=0.707395, acc_norm=0.707395
  • harness|hendrycksTest-prehistory|5: acc=0.759259, acc_norm=0.759259
  • harness|hendrycksTest-professional_accounting|5: acc=0.489362, acc_norm=0.489362
  • harness|hendrycksTest-professional_law|5: acc=0.494785, acc_norm=0.494785
  • harness|hendrycksTest-professional_medicine|5: acc=0.724265, acc_norm=0.724265
  • harness|hendrycksTest-professional_psychology|5: acc=0.679739, acc_norm=0.679739
  • harness|hendrycksTest-public_relations|5: acc=0.663636, acc_norm=0.663636
  • harness|hendrycksTest-security_studies|5: acc=0.742857, acc_norm=0.742857
  • harness|hendrycksTest-sociology|5: acc=0.845771, acc_norm=0.845771
  • harness|hendrycksTest-us_foreign_policy|5: acc=0.84, acc_norm=0.84
  • harness|hendrycksTest-virology|5: acc=0.524096, acc_norm=0.524096
  • harness|hendrycksTest-world_religions|5: acc=0.830409, acc_norm=0.830409
  • harness|truthfulqa:mc|0: mc1=0.394125, mc2=0.554678
  • harness|winogrande|5: acc=0.813733
  • harness|gsm8k|5: acc=0.679303