open-llm-leaderboard/details_ehartford__CodeLlama-34b-Instruct-hf

该数据集是在评估模型ehartford/CodeLlama-34b-Instruct-hf时自动创建的，包含61个配置，每个配置对应一个评估任务。数据集是从1次运行中创建的，每次运行都可以在特定配置中找到，分割名称使用运行的时间戳。train分割始终指向最新结果。此外，results配置存储了所有运行的聚合结果，并用于计算和显示在Open LLM Leaderboard上的聚合指标。

原始信息汇总

数据集概述

该数据集是在对模型 ehartford/CodeLlama-34b-Instruct-hf 进行评估运行期间自动创建的。数据集包含 61 个配置，每个配置对应一个评估任务。数据集从 1 次运行中创建，每次运行的详细结果可以在每个配置中找到，分割名称使用运行的时间戳。"train" 分割始终指向最新的结果。

数据集结构

配置列表

1. config_name: harness_arc_challenge_25

   • data_files:
     • split: 2023_08_26T00_11_17.332215
       • path: **/details_harness|arc:challenge|25_2023-08-26T00:11:17.332215.parquet
     • split: latest
       • path: **/details_harness|arc:challenge|25_2023-08-26T00:11:17.332215.parquet

2. config_name: harness_hellaswag_10

   • data_files:
     • split: 2023_08_26T00_11_17.332215
       • path: **/details_harness|hellaswag|10_2023-08-26T00:11:17.332215.parquet
     • split: latest
       • path: **/details_harness|hellaswag|10_2023-08-26T00:11:17.332215.parquet

3. config_name: harness_hendrycksTest_5

   • data_files:
     • split: 2023_08_26T00_11_17.332215
       • path:
         • **/details_harness|hendrycksTest-abstract_algebra|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-anatomy|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-astronomy|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-business_ethics|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-college_biology|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-college_chemistry|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-college_computer_science|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-college_mathematics|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-college_medicine|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-college_physics|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-computer_security|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-conceptual_physics|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-econometrics|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-electrical_engineering|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-formal_logic|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-global_facts|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-high_school_biology|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-high_school_european_history|5_2023-08-26T00:11:17.332215.parquet
         • **/details_harness|hendrycksTest-high_school_geography|5_2023-08-26T00:11:17.332215.parquet

最新结果

以下是来自 2023-08-26T00:11:17.332215 运行的最新结果：

python { "all": { "acc": 0.3954825543560614, "acc_stderr": 0.034996131407759465, "acc_norm": 0.39693969001192136, "acc_norm_stderr": 0.03500279971831286, "mc1": 0.29008567931456547, "mc1_stderr": 0.01588623687420952, "mc2": 0.4428923144531004, "mc2_stderr": 0.014810370517699043 }, "harness|arc:challenge|25": { "acc": 0.378839590443686, "acc_stderr": 0.01417591549000032, "acc_norm": 0.40784982935153585, "acc_norm_stderr": 0.014361097288449708 }, "harness|hellaswag|10": { "acc": 0.2998406691894045, "acc_stderr": 0.004572515919210699, "acc_norm": 0.35680143397729536, "acc_norm_stderr": 0.004780764443411313 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.36, "acc_stderr": 0.04824181513244218, "acc_norm": 0.36, "acc_norm_stderr": 0.04824181513244218 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.35555555555555557, "acc_stderr": 0.04135176749720386, "acc_norm": 0.35555555555555557, "acc_norm_stderr": 0.04135176749720386 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.3684210526315789, "acc_stderr": 0.03925523381052932, "acc_norm": 0.3684210526315789, "acc_norm_stderr": 0.03925523381052932 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.44, "acc_stderr": 0.04988876515698589, "acc_norm": 0.44, "acc_norm_stderr": 0.04988876515698589 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.4037735849056604, "acc_stderr": 0.030197611600197953, "acc_norm": 0.4037735849056604, "acc_norm_stderr": 0.030197611600197953 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.3680555555555556, "acc_stderr": 0.04032999053960718, "acc_norm": 0.3680555555555556, "acc_norm_stderr": 0.04032999053960718 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.29, "acc_stderr": 0.045604802157206845, "acc_norm": 0.29, "acc_norm_stderr": 0.045604802157206845 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.33, "acc_stderr": 0.047258156262526045, "acc_norm": 0.33, "acc_norm_stderr": 0.047258156262526045 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.41, "acc_stderr": 0.04943110704237102, "acc_norm": 0.41, "acc_norm_stderr": 0.04943110704237102 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.3583815028901734, "acc_stderr": 0.036563436533531585, "acc_norm": 0.3583815028901734, "acc_norm_stderr": 0.036563436533531585 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.22549019607843138, "acc_stderr": 0.041583075330832865, "acc_norm": 0.22549019607843138, "acc_norm_stderr": 0.041583075330832865 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.54, "acc_stderr": 0.05009082659620332, "acc_norm": 0.54, "acc_norm_stderr": 0.05009082659620332 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.3872340425531915, "acc_stderr": 0.03184389265339525, "acc_norm": 0.3872340425531915, "acc_norm_stderr": 0.03184389265339525 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.2543859649122807, "acc_stderr": 0.04096985139843672, "acc_norm": 0.2543859649122807, "acc_norm_stderr": 0.04096985139843672 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.3448275862068966, "acc_stderr": 0.03960933549451208, "acc_norm": 0.3448275862068966, "acc_norm_stderr": 0.03960933549451208 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.304232804