open-llm-leaderboard-old/details_lorinma__yi6B_Vicuna

该数据集是在评估模型lorinma/yi6B_Vicuna在Open LLM Leaderboard上的表现时自动生成的。数据集包含63个配置，每个配置对应一个评估任务。数据集由1次运行生成，每次运行的结果作为一个特定的分割存储，分割名称使用运行的时间戳。train分割始终指向最新的结果。此外，数据集还包含一个名为results的配置，用于存储所有运行的聚合结果，并用于计算和显示Open LLM Leaderboard上的聚合指标。

原始信息汇总

数据集概述

该数据集是在对模型 lorinma/yi6B_Vicuna 进行评估运行期间自动创建的。数据集包含 63 个配置，每个配置对应一个评估任务。数据集由 1 次运行创建，每次运行的详细信息可以在每个配置中找到，使用运行的时间戳作为分割名称。"train" 分割始终指向最新结果。

数据集结构

数据集包含以下配置：

• harness_arc_challenge_25

  • 分割：2024_01_10T18_32_12.113639
  • 路径：**/details_harness|arc:challenge|25_2024-01-10T18-32-12.113639.parquet
  • 分割：latest
  • 路径：**/details_harness|arc:challenge|25_2024-01-10T18-32-12.113639.parquet

• harness_gsm8k_5

  • 分割：2024_01_10T18_32_12.113639
  • 路径：**/details_harness|gsm8k|5_2024-01-10T18-32-12.113639.parquet
  • 分割：latest
  • 路径：**/details_harness|gsm8k|5_2024-01-10T18-32-12.113639.parquet

• harness_hellaswag_10

  • 分割：2024_01_10T18_32_12.113639
  • 路径：**/details_harness|hellaswag|10_2024-01-10T18-32-12.113639.parquet
  • 分割：latest
  • 路径：**/details_harness|hellaswag|10_2024-01-10T18-32-12.113639.parquet

• harness_hendrycksTest_5

  • 分割：2024_01_10T18_32_12.113639
  • 路径：
    • **/details_harness|hendrycksTest-abstract_algebra|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-anatomy|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-astronomy|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-business_ethics|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-clinical_knowledge|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-college_biology|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-college_chemistry|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-college_computer_science|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-college_mathematics|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-college_medicine|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-college_physics|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-computer_security|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-conceptual_physics|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-econometrics|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-electrical_engineering|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-elementary_mathematics|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-formal_logic|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-global_facts|5_2024-01-10T18-32-12.113639.parquet
    • **/details_harness|hendrycksTest-high_school_biology|5_2024-01-10T18-32-12.113639.parquet
    • ...

最新结果

以下是 2024-01-10T18:32:12.113639 运行的最新结果：

python { "all": { "acc": 0.5754850758494996, "acc_stderr": 0.033294207054371715, "acc_norm": 0.5840580871302226, "acc_norm_stderr": 0.03401101149690249, "mc1": 0.3243574051407589, "mc1_stderr": 0.01638797677964794, "mc2": 0.48112670332781104, "mc2_stderr": 0.01615176767349498 }, "harness|arc:challenge|25": { "acc": 0.42918088737201365, "acc_stderr": 0.014464085894870651, "acc_norm": 0.4616040955631399, "acc_norm_stderr": 0.014568245550296363 }, "harness|hellaswag|10": { "acc": 0.5296753634734117, "acc_stderr": 0.0049809853841529, "acc_norm": 0.692989444333798, "acc_norm_stderr": 0.004603111343213064 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.28, "acc_stderr": 0.04512608598542127, "acc_norm": 0.28, "acc_norm_stderr": 0.04512608598542127 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.4666666666666667, "acc_stderr": 0.043097329010363554, "acc_norm": 0.4666666666666667, "acc_norm_stderr": 0.043097329010363554 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.6118421052631579, "acc_stderr": 0.03965842097512744, "acc_norm": 0.6118421052631579, "acc_norm_stderr": 0.03965842097512744 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.65, "acc_stderr": 0.0479372485441102, "acc_norm": 0.65, "acc_norm_stderr": 0.0479372485441102 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.630188679245283, "acc_stderr": 0.02971142188010793, "acc_norm": 0.630188679245283, "acc_norm_stderr": 0.02971142188010793 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.6388888888888888, "acc_stderr": 0.04016660030451233, "acc_norm": 0.6388888888888888, "acc_norm_stderr": 0.04016660030451233 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.4, "acc_stderr": 0.04923659639173309, "acc_norm": 0.4, "acc_norm_stderr": 0.04923659639173309 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.52, "acc_stderr": 0.050211673156867795, "acc_norm": 0.52, "acc_norm_stderr": 0.050211673156867795 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.32, "acc_stderr": 0.04688261722621505, "acc_norm": 0.32, "acc_norm_stderr": 0.04688261722621505 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5606936416184971, "acc_stderr": 0.03784271932887467, "acc_norm": 0.5606936416184971, "acc_norm_stderr": 0.03784271932887467 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.35294117647058826, "acc_stderr": 0.04755129616062946, "acc_norm": 0.35294117647058826, "acc_norm_stderr": 0.04755129616062946 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.8, "acc_stderr": 0.04020151261036846, "acc_norm": 0.8, "acc_norm_stderr": 0.04020151261036846 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.5404255319148936, "acc_stderr": 0.03257901482099834, "acc_norm": 0.5404255319148936, "acc_norm_stderr": 0.03257901482099834 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.34210526315789475, "acc_stderr": 0.04462917535336936, "acc_norm": 0.34210526315789475, "acc_norm_stderr": 0.04462917535336936 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.6275862068965518, "acc_stderr": 0.04028731532947559, "acc_norm": 0.6275862068965518, "acc_norm_stderr": 0.04028731532947559 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.42592592592592593, "acc_stderr": 0.025467149045469536, "acc_norm": 0.42592592592592593, "acc_norm_stderr": 0.025467149045469536 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.404761904761904