open-llm-leaderboard-old/details_wenge-research__yayi-70b-llama2

该数据集是在Open LLM Leaderboard上对模型wenge-research/yayi-70b-llama2进行评估时自动创建的。数据集包含61个配置，每个配置对应一个评估任务。数据集由1次运行生成，每次运行的结果存储为特定配置中的一个分割，分割名称使用运行的时间戳。train分割始终指向最新的结果。此外，还有一个名为results的配置，存储了所有运行的聚合结果，并用于计算和显示Open LLM Leaderboard上的聚合指标。

原始信息汇总

数据集概述

数据集简介

该数据集是在评估模型 wenge-research/yayi-70b-llama2 在 Open LLM Leaderboard 上的运行过程中自动创建的。数据集包含61个配置，每个配置对应一个评估任务。

数据集结构

数据集由1次运行创建，每个运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。"train" 分割始终指向最新的结果。

额外配置

一个额外的配置 "results" 存储了所有运行的聚合结果，用于计算并在 Open LLM Leaderboard 上显示聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_wenge-research__yayi-70b-llama2", "harness_truthfulqa_mc_0", split="train")

最新结果

以下是 2023-09-13T20:08:14.965059 运行的最新结果：

python { "all": { "acc": 0.6427362614871128, "acc_stderr": 0.03251742836753478, "acc_norm": 0.6468766983428953, "acc_norm_stderr": 0.032494548846313066, "mc1": 0.30599755201958384, "mc1_stderr": 0.016132229728155045, "mc2": 0.4762734947955207, "mc2_stderr": 0.01439837288557781 }, "harness|arc:challenge|25": { "acc": 0.5614334470989761, "acc_stderr": 0.014500682618212862, "acc_norm": 0.606655290102389, "acc_norm_stderr": 0.014275101465693026 }, "harness|hellaswag|10": { "acc": 0.640211113324039, "acc_stderr": 0.0047895751634186535, "acc_norm": 0.8392750448117905, "acc_norm_stderr": 0.00366526456385775 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.38, "acc_stderr": 0.048783173121456316, "acc_norm": 0.38, "acc_norm_stderr": 0.048783173121456316 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.5777777777777777, "acc_stderr": 0.04266763404099582, "acc_norm": 0.5777777777777777, "acc_norm_stderr": 0.04266763404099582 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.6973684210526315, "acc_stderr": 0.03738520676119669, "acc_norm": 0.6973684210526315, "acc_norm_stderr": 0.03738520676119669 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.69, "acc_stderr": 0.04648231987117316, "acc_norm": 0.69, "acc_norm_stderr": 0.04648231987117316 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.6943396226415094, "acc_stderr": 0.028353298073322666, "acc_norm": 0.6943396226415094, "acc_norm_stderr": 0.028353298073322666 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.7013888888888888, "acc_stderr": 0.03827052357950756, "acc_norm": 0.7013888888888888, "acc_norm_stderr": 0.03827052357950756 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.37, "acc_stderr": 0.048523658709391, "acc_norm": 0.37, "acc_norm_stderr": 0.048523658709391 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.53, "acc_stderr": 0.05016135580465919, "acc_norm": 0.53, "acc_norm_stderr": 0.05016135580465919 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.29, "acc_stderr": 0.04560480215720683, "acc_norm": 0.29, "acc_norm_stderr": 0.04560480215720683 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5606936416184971, "acc_stderr": 0.037842719328874674, "acc_norm": 0.5606936416184971, "acc_norm_stderr": 0.037842719328874674 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.4019607843137255, "acc_stderr": 0.04878608714466996, "acc_norm": 0.4019607843137255, "acc_norm_stderr": 0.04878608714466996 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.76, "acc_stderr": 0.042923469599092816, "acc_norm": 0.76, "acc_norm_stderr": 0.042923469599092816 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.6170212765957447, "acc_stderr": 0.03177821250236922, "acc_norm": 0.6170212765957447, "acc_norm_stderr": 0.03177821250236922 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.42105263157894735, "acc_stderr": 0.04644602091222318, "acc_norm": 0.42105263157894735, "acc_norm_stderr": 0.04644602091222318 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.6068965517241379, "acc_stderr": 0.0407032901370707, "acc_norm": 0.6068965517241379, "acc_norm_stderr": 0.0407032901370707 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.42592592592592593, "acc_stderr": 0.025467149045469536, "acc_norm": 0.42592592592592593, "acc_norm_stderr": 0.025467149045469536 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.36507936507936506, "acc_stderr": 0.04306241259127153, "acc_norm": 0.36507936507936506, "acc_norm_stderr": 0.04306241259127153 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.46, "acc_stderr": 0.05009082659620333, "acc_norm": 0.46, "acc_norm_stderr": 0.05009082659620333 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.7870967741935484, "acc_stderr": 0.02328766512726854, "acc_norm": 0.7870967741935484, "acc_norm_stderr": 0.02328766512726854 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.5369458128078818, "acc_stderr": 0.035083705204426656, "acc_norm": 0.5369458128078818, "acc_norm_stderr": 0.035083705204426656 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.7, "acc_stderr": 0.046056618647183814, "acc_norm": 0.7, "acc_norm_stderr": 0.046056618647183814 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.793939393939394, "acc_stderr": 0.0315841532404771, "acc_norm": 0.793939393939394, "acc_norm_stderr": 0.0315841532404771 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.803030303030303, "acc_stderr": 0.028335609732463362, "acc_norm": 0.803030303030303, "acc_norm_stderr": 0.028335609732463362 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.9067357512953368, "acc_stderr": 0.020986854593289708, "acc_norm": 0.9067357512953368, "acc_norm_stderr": 0.020986854593289708 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.6487179487179487, "acc_stderr": 0.024203665177902803, "acc_norm": 0.6487179487179487, "acc_norm_stderr": 0.024203665177902803 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.34444444444444444, "acc_stderr": 0.028972648884844267, "acc_norm": 0.34444444444444444, "acc_norm_stderr": 0.028972648884844267 }, "harness|hendrycksTest-high_school_microeconomics|5": { "acc": 0.6386554621848739, "acc_stderr": 0.03120469122515002, "acc_norm": 0.6386554621848739, "acc_norm_stderr": 0.03120469122515002 }, "harness|hendrycksTest-high_school_physics|5": { "acc": 0.41721854304635764, "acc_stderr": 0.04026141497634612, "acc_norm": 0.41721854304635764, "acc_norm_stderr": 0.04026141497634612 }, "harness|hendrycksTest-high_school_psychology|5": { "acc": 0.8128440366972477, "acc_stderr": 0.016722684526200154, "acc_norm": 0.8128440366972477, "acc_norm_stderr": 0.016722684526200154 }, "harness|hendrycksTest-high_school_statistics|5": { "acc": 0.49074074074074076, "acc_stderr": 0.034093869469927006, "acc_norm": 0.49074074074074076, "acc_norm_stderr": 0.034093869469927006 }, "harness|hendrycksTest-high_school_us_history|5": { "acc": 0.8431372549019608, "acc_stderr": 0.02552472232455334, "acc_norm": 0.8431372549019608, "acc_norm_stderr": 0.02552472232455334 }, "harness|hendrycksTest-high_school_world_history|5": { "acc": 0.8143459915611815, "acc_stderr": 0.025310495376944853, "acc_norm": 0.8143459915611815, "acc_norm_stderr": 0.025310495376944853 }, "harness|hendrycksTest-human_aging|5": { "acc": 0.7354260089686099, "acc_stderr": 0.02960510321703832, "acc_norm": 0.7354260089686099, "acc_norm_stderr": 0.02960510321703832 }, "harness|hendrycksTest-human_sexuality|5": { "acc": 0.7938931297709924, "acc_stderr": 0.035477710041594654, "acc_norm": 0.7938931297709924, "acc_norm_stderr": 0.035477710041594654 }, "harness|hendrycksTest-international_law|5": { "acc": 0.8677685950413223, "acc_stderr": 0.0309227883204458, "acc_norm": 0.8677685950413223, "acc_norm_stderr": 0.0309227883204458 }, "harness|hendrycksTest-jurisprudence|5": { "acc": 0.7962962962962963, "acc_stderr": 0.038935425188248475, "acc_norm": 0.7962962962962963, "acc_norm_stderr": 0.038935425188248475 }, "harness|hendrycksTest-logical_fallacies|5": { "acc": 0.754601226993865, "acc_stderr": 0.03380939813943354, "acc_norm": 0.754601226993865, "acc_norm_stderr": 0.03380939813943354 }, "harness|hendrycksTest-machine_learning|5": { "acc": 0.4642857142857143, "acc_stderr": 0.04733667890053756, "acc_norm": 0.4642857142857143, "acc_norm_stderr": 0.04733667890053756 }, "harness|hendrycksTest-management|5": { "acc": 0.7961165048543689, "acc_stderr": 0.0398913985953177, "acc_norm": 0.7961165048543689, "acc_norm_stderr": 0.0398913985953177 }, "harness|hendrycksTest-marketing|5": { "acc": 0.8632478632478633, "acc_stderr": 0.022509033937077805, "acc_norm": 0.8632478632478633, "acc_norm_stderr": 0.022509033937077805 }, "harness|hendrycksTest-medical_genetics|5": { "acc": 0.64, "acc_stderr": 0.04824181513244218, "acc_norm": 0.64, "acc_norm_stderr": 0.04824181513244218 }, "harness|hendrycksTest-miscellaneous|5": { "acc": 0.8199233716475096, "acc_stderr": 0.01374079725857982, "acc_norm": 0.8199233716475096, "acc_norm_stderr": 0.01374079725857982 }, "harness|hendrycksTest-moral_disputes|5": { "acc": 0.7312138728323699, "acc_stderr": 0.023868003262500104, "acc_norm": 0.7312138728323699, "acc_norm_stderr": 0.023868003262500104 }, "harness|hendrycksTest-moral_scenarios|5": { "acc": 0.2737430167597765, "acc_stderr": 0.014912413096372434, "acc_norm": 0.2737430167597765, "acc_norm_stderr": 0.014912413096372434 }, "harness|hendrycksTest-nutrition|5": { "acc": 0.6993464052287581, "acc_stderr": 0.02625605383571896, "acc_norm": 0.6993464052287581, "acc_norm_stderr": 0.02625605383571896 }, "harness|hendrycksTest-philosophy|5": { "acc": 0.7363344051446945, "acc_stderr": 0.02502553850053234, "acc_norm": 0.7363344051446945, "acc_norm_stderr": 0.02502553850053234 }, "harness|hendrycksTest-prehistory|5": { "acc": 0.7314814814814815, "acc_stderr": 0.0246596851859673, "acc_norm": 0.7314814814814815, "acc_norm_stderr": 0.0246596851859673 }, "harness|hendrycksTest-professional_accounting|5": { "acc": 0.5177304964539007, "acc_stderr": 0.02980873964223777, "acc_norm": 0.5177304964539007, "acc_norm_stderr": 0.02980873964223777 }, "harness|hendrycksTest-professional_law|5": { "acc": 0.5110821382007823, "acc_stderr": 0.012767098998525826, "acc_norm": 0.5110821382007823, "acc_norm_stderr": 0.012767098998525826 }, "harness|hendrycksTest-professional_medicine|5": { "acc": 0.5772058823529411, "acc_stderr": 0.030008562845003476, "acc_norm": 0.5772058823529411, "acc_norm_stderr": 0.030008562845003476 }, "harness|hendrycksTest-professional_psychology|5": { "acc": 0.6764705882352942, "acc_stderr": 0.018926082916083376, "acc_norm": 0.6764705882352942, "acc_norm_stderr": 0.018926082916083376 }, "harness|hendrycksTest-public_relations|5": { "acc": 0.7181818181818181, "acc_stderr": 0.043091187099464585, "acc_norm": 0.7181818181818181, "acc_norm_stderr": 0.043091187099464585 }, "harness|hendrycksTest-security_studies|5": { "acc": 0.7673469387755102, "acc_stderr": 0.027049257915896175, "acc_norm": 0.7673469387755102, "acc_norm_stderr": 0.027049257915896175 }, "harness|hendrycksTest-sociology|5": { "acc": 0.8407960199004975, "acc_stderr": 0.02587064676616913, "acc_norm": 0.8407960199004975, "acc_norm_stderr": 0.02587064676616913 }, "harness|hendrycksTest-us_foreign_policy|5": { "acc": 0.93, "acc_stderr": 0.025643239997624294, "acc_norm": 0.93, "acc_norm_stderr": 0.025643239997624294 }, "harness|hendrycksTest-virology|5": { "acc": 0.536144578313253, "acc_stderr": 0.03882310850890594, "acc_norm": 0.536144578313253, "acc_norm_stderr": 0.03882310850890594 }, "harness|hendrycksTest-world_religions|5": { "acc": 0.8538011695906432, "acc_stderr": 0.027097290118070803, "acc_norm": 0.8538011695906432, "acc_norm_stderr": 0.027097290118070803 }, "harness|truthfulqa:mc|0": { "mc1": 0.30599755201958384, "mc1_stderr": 0.016132229728155045, "mc2": 0.4762734947955207, "mc2_stderr": 0.01439837288557781 } }

数据集配置

• 配置名称: harness_arc_challenge_25

  • 数据文件:
    • 分割: 2023_09_13T20_08_14.965059
      • 路径: **/details_harness|arc:challenge|25_2023-09-13T20-08-14.965059.parquet
    • 分割: latest
      • 路径: **/details_harness|arc:challenge|25_2023-09-13T20-08-14.965059.parquet

• 配置名称: harness_hellaswag_10

  • 数据文件:
    • 分割: 2023_09_13T20_08_14.965059
      • 路径: **/details_harness|hellaswag|10_2023-09-13T20-08-14.965059.parquet
    • 分割: latest
      • 路径: **/details_harness|hellaswag|10_2023-09-13T20-08-14.965059.parquet

• 配置名称: harness_hendrycksTest_5

  • 数据文件:
    • 分割: 2023_09_13T20_08_14.965059
      • 路径:
        • **/details_harness|hendrycksTest-abstract_algebra|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-anatomy|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-astronomy|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-business_ethics|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-clinical_knowledge|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-college_biology|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-college_chemistry|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-college_computer_science|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-college_mathematics|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-college_medicine|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-college_physics|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-computer_security|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-conceptual_physics|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-econometrics|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-electrical_engineering|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-elementary_mathematics|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-formal_logic|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-global_facts|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-high_school_biology|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-high_school_chemistry|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-high_school_computer_science|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-high_school_european_history|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-high_school_geography|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-high_school_mathematics|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-high_school_microeconomics|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-high_school_physics|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-high_school_psychology|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-high_school_statistics|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-high_school_us_history|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-high_school_world_history|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-human_aging|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-human_sexuality|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-international_law|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-jurisprudence|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-logical_fallacies|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-machine_learning|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-management|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-marketing|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-medical_genetics|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-miscellaneous|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-moral_disputes|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-moral_scenarios|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-nutrition|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-philosophy|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-prehistory|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-professional_accounting|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-professional_law|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-professional_medicine|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-professional_psychology|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-public_relations|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-security_studies|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-sociology|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-us_foreign_policy|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-virology|5_2023-09-13T20-08-14.965059.parquet
        • **/details_harness|hendrycksTest-world_religions|5_2023-09-13T20-08-14.965059.parquet