open-llm-leaderboard-old/details_azarafrooz__mistral-7b-selfplay-v0

该数据集是在模型 azarafrooz/mistral-7b-selfplay-v0 在 Open LLM Leaderboard 上的评估运行期间自动创建的。数据集由 63 个配置组成，每个配置对应一个被评估的任务。它包含 1 次运行的结果，每次运行在每个配置中表示为特定的分割。train 分割始终指向最新的结果。一个名为 results 的额外配置存储了所有运行的聚合结果，这些结果用于计算和显示 Open LLM Leaderboard 上的聚合指标。README 还提供了如何使用 Python 中的 datasets 库加载运行中的详细信息的示例。

原始信息汇总

数据集概述

数据集摘要

该数据集是在对模型 azarafrooz/mistral-7b-selfplay-v0 进行评估运行期间自动创建的，用于 Open LLM Leaderboard。数据集包含 63 个配置，每个配置对应一个评估任务。数据集从 1 次运行中创建，每个运行的详细信息可以在每个配置中找到，使用运行的时间戳作为分割名称。"train" 分割始终指向最新的结果。

数据集结构

数据集包含多个配置，每个配置对应不同的评估任务。以下是部分配置的示例：

• harness_arc_challenge_25

  • 分割：2024_03_05T19_17_43.573204
  • 路径：**/details_harness|arc:challenge|25_2024-03-05T19-17-43.573204.parquet
  • 分割：latest
  • 路径：**/details_harness|arc:challenge|25_2024-03-05T19-17-43.573204.parquet

• harness_gsm8k_5

  • 分割：2024_03_05T19_17_43.573204
  • 路径：**/details_harness|gsm8k|5_2024-03-05T19-17-43.573204.parquet
  • 分割：latest
  • 路径：**/details_harness|gsm8k|5_2024-03-05T19-17-43.573204.parquet

• harness_hellaswag_10

  • 分割：2024_03_05T19_17_43.573204
  • 路径：**/details_harness|hellaswag|10_2024-03-05T19-17-43.573204.parquet
  • 分割：latest
  • 路径：**/details_harness|hellaswag|10_2024-03-05T19-17-43.573204.parquet

• harness_hendrycksTest_5

  • 分割：2024_03_05T19_17_43.573204
  • 路径：
    • **/details_harness|hendrycksTest-abstract_algebra|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-anatomy|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-astronomy|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-business_ethics|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-clinical_knowledge|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-college_biology|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-college_chemistry|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-college_computer_science|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-college_mathematics|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-college_medicine|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-college_physics|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-computer_security|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-conceptual_physics|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-econometrics|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-electrical_engineering|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-elementary_mathematics|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-formal_logic|5_2024-03-05T19-17-43.573204.parquet
    • **/details_harness|hendrycksTest-global_facts|5_2024-03-05T19-17-43.573204.parquet

最新结果

以下是 2024-03-05T19:17:43.573204 运行 的最新结果：

python { "all": { "acc": 0.5528308683620587, "acc_stderr": 0.03406015270677547, "acc_norm": 0.5573659242978083, "acc_norm_stderr": 0.034776472103005815, "mc1": 0.39657282741738065, "mc1_stderr": 0.017124930942023518, "mc2": 0.5628188850184273, "mc2_stderr": 0.015349135711816259 }, "harness|arc:challenge|25": { "acc": 0.5238907849829352, "acc_stderr": 0.014594701798071654, "acc_norm": 0.5469283276450512, "acc_norm_stderr": 0.014546892052005628 }, "harness|hellaswag|10": { "acc": 0.569308902609042, "acc_stderr": 0.0049416098207635826, "acc_norm": 0.756920932085242, "acc_norm_stderr": 0.00428065823471877 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.35, "acc_stderr": 0.04793724854411021, "acc_norm": 0.35, "acc_norm_stderr": 0.04793724854411021 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.4222222222222222, "acc_stderr": 0.04266763404099582, "acc_norm": 0.4222222222222222, "acc_norm_stderr": 0.04266763404099582 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.5723684210526315, "acc_stderr": 0.04026097083296563, "acc_norm": 0.5723684210526315, "acc_norm_stderr": 0.04026097083296563 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.52, "acc_stderr": 0.05021167315686779, "acc_norm": 0.52, "acc_norm_stderr": 0.05021167315686779 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.5849056603773585, "acc_stderr": 0.030325945789286105, "acc_norm": 0.5849056603773585, "acc_norm_stderr": 0.030325945789286105 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.5833333333333334, "acc_stderr": 0.04122728707651282, "acc_norm": 0.5833333333333334, "acc_norm_stderr": 0.04122728707651282 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.4, "acc_stderr": 0.049236596391733084, "acc_norm": 0.4, "acc_norm_stderr": 0.049236596391733084 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.52, "acc_stderr": 0.050211673156867795, "acc_norm": 0.52, "acc_norm_stderr": 0.050211673156867795 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.38, "acc_stderr": 0.04878317312145633, "acc_norm": 0.38, "acc_norm_stderr": 0.04878317312145633 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5144508670520231, "acc_stderr": 0.03810871630454764, "acc_norm": 0.5144508670520231, "acc_norm_stderr": 0.03810871630454764 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.29411764705882354, "acc_stderr": 0.04533838195929775, "acc_norm": 0.29411764705882354, "acc_norm_stderr": 0.04533838195929775 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.68, "acc_stderr": 0.046882617226215034, "acc_norm": 0.68, "acc_norm_stderr": 0.046882617226215034 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.4851063829787234, "acc_stderr": 0.032671518489247764, "acc_norm": 0.4851063829787234, "acc_norm_stderr": 0.032671518489247764 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.37719298245614036, "acc_stderr": 0.04559522141958216, "acc_norm": 0.37719298245614036, "acc_norm_stderr": 0.04559522141958216 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5448275862068965, "acc_stderr": 0.04149886942192117, "acc_norm": 0.5448275862068965, "acc_norm_stderr": 0.04149886942192117 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.37037037037037035, "acc_stderr": 0.024870815251057096, "acc_norm": 0.3