open-llm-leaderboard-old/details_migtissera__Synthia-34B-v1.2

该数据集是在模型migtissera/Synthia-34B-v1.2在Open LLM Leaderboard上的评估运行期间自动创建的。数据集由61个配置组成，每个配置对应一个被评估的任务。数据集是从一次运行中生成的，每次运行在每个配置中表示为特定的拆分，使用运行的时间戳命名。train拆分始终指向最新的结果。一个名为results的额外配置存储了运行的所有聚合结果，用于计算和显示Open LLM Leaderboard上的聚合指标。README还提供了如何使用Hugging Face datasets库加载数据集的示例，并包含了特定运行的最新结果。

原始信息汇总

数据集概述

数据集简介

该数据集是在对模型 migtissera/Synthia-34B-v1.2 进行评估运行期间自动创建的，用于 Open LLM Leaderboard。

数据集组成

• 数据集包含 61 个配置，每个配置对应一个评估任务。
• 数据集从 1 次运行中创建，每个运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
• "train" 分割始终指向最新的结果。
• 一个额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_migtissera__Synthia-34B-v1.2", "harness_truthfulqa_mc_0", split="train")

最新结果

以下是 2023-09-18T20:05:34.645170 运行的最新结果：

python { "all": { "acc": 0.5320903185183409, "acc_stderr": 0.03517517994960793, "acc_norm": 0.5358397153796313, "acc_norm_stderr": 0.03516397638431902, "mc1": 0.2998776009791922, "mc1_stderr": 0.01604035296671362, "mc2": 0.4467341818408572, "mc2_stderr": 0.014969799807071376 }, "harness|arc:challenge|25": { "acc": 0.5119453924914675, "acc_stderr": 0.014607220340597171, "acc_norm": 0.5486348122866894, "acc_norm_stderr": 0.01454210456995527 }, "harness|hellaswag|10": { "acc": 0.5587532364070902, "acc_stderr": 0.00495521278783238, "acc_norm": 0.7432782314280024, "acc_norm_stderr": 0.004359318206428689 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.26, "acc_stderr": 0.0440844002276808, "acc_norm": 0.26, "acc_norm_stderr": 0.0440844002276808 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.43703703703703706, "acc_stderr": 0.04284958639753399, "acc_norm": 0.43703703703703706, "acc_norm_stderr": 0.04284958639753399 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.506578947368421, "acc_stderr": 0.040685900502249704, "acc_norm": 0.506578947368421, "acc_norm_stderr": 0.040685900502249704 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.55, "acc_stderr": 0.05, "acc_norm": 0.55, "acc_norm_stderr": 0.05 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.5433962264150943, "acc_stderr": 0.030656748696739428, "acc_norm": 0.5433962264150943, "acc_norm_stderr": 0.030656748696739428 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.5694444444444444, "acc_stderr": 0.04140685639111502, "acc_norm": 0.5694444444444444, "acc_norm_stderr": 0.04140685639111502 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.46, "acc_stderr": 0.05009082659620332, "acc_norm": 0.46, "acc_norm_stderr": 0.05009082659620332 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.47, "acc_stderr": 0.050161355804659205, "acc_norm": 0.47, "acc_norm_stderr": 0.050161355804659205 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.3, "acc_stderr": 0.046056618647183814, "acc_norm": 0.3, "acc_norm_stderr": 0.046056618647183814 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5028901734104047, "acc_stderr": 0.03812400565974834, "acc_norm": 0.5028901734104047, "acc_norm_stderr": 0.03812400565974834 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.37254901960784315, "acc_stderr": 0.04810840148082635, "acc_norm": 0.37254901960784315, "acc_norm_stderr": 0.04810840148082635 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.72, "acc_stderr": 0.045126085985421296, "acc_norm": 0.72, "acc_norm_stderr": 0.045126085985421296 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.41702127659574467, "acc_stderr": 0.03223276266711712, "acc_norm": 0.41702127659574467, "acc_norm_stderr": 0.03223276266711712 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.39473684210526316, "acc_stderr": 0.045981880578165414, "acc_norm": 0.39473684210526316, "acc_norm_stderr": 0.045981880578165414 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.46206896551724136, "acc_stderr": 0.04154659671707548, "acc_norm": 0.46206896551724136, "acc_norm_stderr": 0.04154659671707548 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.4021164021164021, "acc_stderr": 0.025253032554997692, "acc_norm": 0.4021164021164021, "acc_norm_stderr": 0.025253032554997692 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.4365079365079365, "acc_stderr": 0.04435932892851466, "acc_norm": 0.4365079365079365, "acc_norm_stderr": 0.04435932892851466 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.38, "acc_stderr": 0.04878317312145634, "acc_norm": 0.38, "acc_norm_stderr": 0.04878317312145634 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.6225806451612903, "acc_stderr": 0.027575960723278243, "acc_norm": 0.6225806451612903, "acc_norm_stderr": 0.027575960723278243 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.3842364532019704, "acc_stderr": 0.0342239856565755, "acc_norm": 0.3842364532019704, "acc_norm_stderr": 0.0342239856565755 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.67, "acc_stderr": 0.04725815626252606, "acc_norm": 0.67, "acc_norm_stderr": 0.04725815626252606 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.6606060606060606, "acc_stderr": 0.03697442205031596, "acc_norm": 0.6606060606060606, "acc_norm_stderr": 0.03697442205031596 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.6666666666666666, "acc_stderr": 0.033586181457325226, "acc_norm": 0.6666666666666666, "acc_norm_stderr": 0.033586181457325226 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.7305699481865285, "acc_stderr": 0.032018671228777947, "acc_norm": 0.7305699481865285, "acc_norm_stderr": 0.032018671228777947 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.5128205128205128, "acc_stderr": 0.02534267129380725, "acc_norm": 0.5128205128205128, "acc_norm_stderr": 0.02534267129380725 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.3037037037037037, "acc_stderr": 0.028037929969114982, "acc_norm": 0.3037037037037037, "acc_norm_stderr": 0.028037929969114982 }, "harness|hendrycksTest-high_school_microeconomics|5": { "acc": 0.5672268907563025, "acc_stderr": 0.032183581077426124, "acc_norm": 0.5672268907563025, "acc_norm_stderr": 0.032183581077426124 }, "harness|hendrycksTest-high_school_physics|5": { "acc": 0.39072847682119205, "acc_stderr": 0.03983798306659806, "acc_norm": 0.39072847682119205, "acc_norm_stderr": 0.03983798306659806 }, "harness|hendrycksTest-high_school_psychology|5": { "acc": 0.6954128440366972, "acc_stderr": 0.019732299420354052, "acc_norm": 0.6954128440366972, "acc_norm_stderr": 0.019732299420354052 }, "harness|hendrycksTest-high_school_statistics|5": { "acc": 0.4027777777777778, "acc_stderr": 0.033448873829978666, "acc_norm": 0.4027777777777778, "acc_norm_stderr": 0.033448873829978666 }, "harness|hendrycksTest-high_school_us_history|5": { "acc": 0.7107843137254902, "acc_stderr": 0.031822318676475544, "acc_norm": 0.7107843137254902, "acc_norm_stderr": 0.031822318676475544 }, "harness|hendrycksTest-high_school_world_history|5": { "acc": 0.7172995780590717, "acc_stderr": 0.02931281415395593, "acc_norm": 0.7172995780590717, "acc_norm_stderr": 0.02931281415395593 }, "harness|hendrycksTest-human_aging|5": { "acc": 0.5426008968609866, "acc_stderr": 0.033435777055830646, "acc_norm": 0.5426008968609866, "acc_norm_stderr": 0.033435777055830646 }, "harness|hendrycksTest-human_sexuality|5": { "acc": 0.5343511450381679, "acc_stderr": 0.043749285605997376, "acc_norm": 0.5343511450381679, "acc_norm_stderr": 0.043749285605997376 }, "harness|hendrycksTest-international_law|5": { "acc": 0.6942148760330579, "acc_stderr": 0.04205953933884123, "acc_norm": 0.6942148760330579, "acc_norm_stderr": 0.04205953933884123 }, "harness|hendrycksTest-jurisprudence|5": { "acc": 0.6481481481481481, "acc_stderr": 0.04616631111801714, "acc_norm": 0.6481481481481481, "acc_norm_stderr": 0.04616631111801714 }, "harness|hendrycksTest-logical_fallacies|5": { "acc": 0.6380368098159509, "acc_stderr": 0.037757007291414416, "acc_norm": 0.6380368098159509, "acc_norm_stderr": 0.037757007291414416 }, "harness|hendrycksTest-machine_learning|5": { "acc": 0.38392857142857145, "acc_stderr": 0.04616143075028547, "acc_norm": 0.38392857142857145, "acc_norm_stderr": 0.04616143075028547 }, "harness|hendrycksTest-management|5": { "acc": 0.6601941747572816, "acc_stderr": 0.046897659372781335, "acc_norm": 0.6601941747572816, "acc_norm_stderr": 0.046897659372781335 }, "harness|hendrycksTest-marketing|5": { "acc": 0.7692307692307693, "acc_stderr": 0.0276019213814176, "acc_norm": 0.7692307692307693, "acc_norm_stderr": 0.0276019213814176 }, "harness|hendrycksTest-medical_genetics|5": { "acc": 0.51, "acc_stderr": 0.05024183937956912, "acc_norm": 0.51, "acc_norm_stderr": 0.05024183937956912 }, "harness|hendrycksTest-miscellaneous|5": { "acc": 0.6743295019157088, "acc_stderr": 0.016757989458549675, "acc_norm": 0.6743295019157088, "acc_norm_stderr": 0.016757989458549675 }, "harness|hendrycksTest-moral_disputes|5": { "acc": 0.5867052023121387, "acc_stderr": 0.02651126136940925, "acc_norm": 0.5867052023121387, "acc_norm_stderr": 0.02651126136940925 }, "harness|hendrycksTest-moral_scenarios|5": { "acc": 0.35977653631284917, "acc_stderr": 0.016051419760310263, "acc_norm": 0.35977653631284917, "acc_norm_stderr": 0.016051419760310263 }, "harness|hendrycksTest-nutrition|5": { "acc": 0.5294117647058824, "acc_stderr": 0.028580341065138296, "acc_norm": 0.5294117647058824, "acc_norm_stderr": 0.028580341065138296 }, "harness|hendrycksTest-philosophy|5": { "acc": 0.6141479099678456, "acc_stderr": 0.027648149599751464, "acc_norm": 0.6141479099678456, "acc_norm_stderr": 0.027648149599751464 }, "harness|hendrycksTest-prehistory|5": { "acc": 0.5524691358024691, "acc_stderr": 0.027667138569422708, "acc_norm": 0.5524691358024691, "acc_norm_stderr": 0.027667138569422708 }, "harness|hendrycksTest-professional_accounting|5": { "acc": 0.37943262411347517, "acc_stderr": 0.028947338851614105, "acc_norm": 0.37943262411347517, "acc_norm_stderr": 0.028947338851614105 }, "harness|hendrycksTest-professional_law|5": { "acc": 0.378748370273794, "acc_stderr": 0.012389052105003732, "acc_norm": 0.378748370273794, "acc_norm_stderr": 0.012389052105003732 }, "harness|hendrycksTest-professional_medicine|5": { "acc": 0.40808823529411764, "acc_stderr": 0.029855261393483924, "acc_norm": 0.40808823529411764, "acc_norm_stderr": 0.029855261393483924 }, "harness|hendrycksTest-professional_psychology|5": { "acc": 0.48366013071895425, "acc_stderr": 0.020217030653186457, "acc_norm": 0.48366013071895425, "acc_norm_stderr": 0.020217030653186457 }, "harness|hendrycksTest-public_relations|5": { "acc": 0.5727272727272728, "acc_stderr": 0.047381987035454834, "acc_norm": 0.5727272727272728, "acc_norm_stderr": 0.047381987035454834 }, "harness|hendrycksTest-security_studies|5": { "acc": 0.6448979591836734, "acc_stderr": 0.030635655150387638, "acc_norm": 0.6448979591836734, "acc_norm_stderr": 0.030635655150387638 }, "harness|hendrycksTest-sociology|5": { "acc": 0.7313432835820896, "acc_stderr": 0.031343283582089536, "acc_norm": 0.7313432835820896, "acc_norm_stderr": 0.031343283582089536 }, "harness|hendrycksTest-us_foreign_policy|5": { "acc": 0.7, "acc_stderr": 0.046056618647183814, "acc_norm": 0.7, "acc_norm_stderr": 0.046056618647183814 }, "harness|hendrycksTest-virology|5": { "acc": 0.4397590361445783, "acc_stderr": 0.03864139923699122, "acc_norm": 0.4397590361445783, "acc_norm_stderr": 0.03864139923699122 }, "harness|hendrycksTest-world_religions|5": { "acc": 0.6666666666666666, "acc_stderr": 0.03615507630310935, "acc_norm": 0.6666666666666666, "acc_norm_stderr": 0.03615507630310935 }, "harness|truthfulqa:mc|0": { "mc1": 0.2998776009791922, "mc1_stderr": 0.01604035296671362, "mc2": 0.4467341818408572, "mc2_stderr": 0.014969799807071376 } }

数据集配置

• 配置名称: harness_arc_challenge_25

  • 数据文件:
    • 分割: 2023_09_18T20_05_34.645170
      • 路径: **/details_harness|arc:challenge|25_2023-09-18T20-05-34.645170.parquet
    • 分割: latest
      • 路径: **/details_harness|arc:challenge|25_2023-09-18T20-05-34.645170.parquet

• 配置名称: harness_hellaswag_10

  • 数据文件:
    • 分割: 2023_09_18T20_05_34.645170
      • 路径: **/details_harness|hellaswag|10_2023-09-18T20-05-34.645170.parquet
    • 分割: latest
      • 路径: **/details_harness|hellaswag|10_2023-09-18T20-05-34.645170.parquet

• 配置名称: harness_hendrycksTest_5

  • 数据文件:
    • 分割: 2023_09_18T20_05_34.645170
      • 路径:
        • **/details_harness|hendrycksTest-abstract_algebra|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-anatomy|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-astronomy|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-business_ethics|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-clinical_knowledge|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-college_biology|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-college_chemistry|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-college_computer_science|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-college_mathematics|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-college_medicine|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-college_physics|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-computer_security|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-conceptual_physics|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-econometrics|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-electrical_engineering|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-elementary_mathematics|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-formal_logic|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-global_facts|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-high_school_biology|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-high_school_chemistry|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-high_school_computer_science|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-high_school_european_history|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-high_school_geography|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-high_school_mathematics|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-high_school_microeconomics|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-high_school_physics|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-high_school_psychology|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-high_school_statistics|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-high_school_us_history|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-high_school_world_history|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-human_aging|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-human_sexuality|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-international_law|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-jurisprudence|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-logical_fallacies|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-machine_learning|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-management|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-marketing|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-medical_genetics|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-miscellaneous|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-moral_disputes|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-moral_scenarios|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-nutrition|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-philosophy|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-prehistory|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-professional_accounting|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-professional_law|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-professional_medicine|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-professional_psychology|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-public_relations|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-security_studies|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-sociology|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-us_foreign_policy|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-virology|5_2023-09-18T20-05-34.645170.parquet
        • **/details_harness|hendrycksTest-world_religions|5_2023-09-18T20-05-34.645170.parquet