open-llm-leaderboard/details_acrastt__Puma-3B

数据集`Evaluation run of acrastt/Puma-3B`是在模型`acrastt/Puma-3B`的评估运行期间自动创建的，用于在Open LLM Leaderboard上进行评估。数据集包含64个配置，每个配置对应一个评估任务。数据集由5次运行生成，每次运行的结果作为一个特定的split存储，split的名称使用运行的时间戳。此外，数据集还包含一个名为`results`的配置，用于存储所有运行的聚合结果，并用于在Open LLM Leaderboard上计算和显示聚合指标。The dataset `Evaluation run of acrastt/Puma-3B` was automatically generated during the evaluation run of the model `acrastt/Puma-3B`, and is intended for evaluation on the Open LLM Leaderboard. The dataset consists of 64 configurations, each corresponding to a single evaluation task. It is compiled from five independent runs, with the results of each run stored as a dedicated split, where the name of the split is the timestamp of the corresponding run. Additionally, the dataset includes a configuration named `results` that stores the aggregated results across all runs, and is used to compute and display the aggregated metrics on the Open LLM Leaderboard.

原始信息汇总

数据集概述

数据集来源

该数据集是在对模型 acrastt/Puma-3B 进行评估运行期间自动创建的，评估运行在 Open LLM Leaderboard 上进行。

数据集结构

• 配置数量：数据集包含 64 个配置，每个配置对应一个评估任务。
• 运行次数：数据集从 5 次运行中创建。每个运行可以在每个配置中作为一个特定的分割找到，分割名称使用运行的时间戳。
• 最新结果："train" 分割总是指向最新的结果。
• 结果汇总：一个额外的配置 "results" 存储所有运行的汇总结果，用于计算和显示在 Open LLM Leaderboard 上的汇总指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_acrastt__Puma-3B", "harness_winogrande_5", split="train")

最新结果

以下是 2023-10-18T02:31:20.356957 运行的最新结果： python { "all": { "em": 0.0008389261744966443, "em_stderr": 0.0002964962989801232, "f1": 0.05384123322147677, "f1_stderr": 0.0012835343678864997, "acc": 0.335677096267113, "acc_stderr": 0.00783328405303534 }, "harness|drop|3": { "em": 0.0008389261744966443, "em_stderr": 0.0002964962989801232, "f1": 0.05384123322147677, "f1_stderr": 0.0012835343678864997 }, "harness|gsm8k|5": { "acc": 0.0075815011372251705, "acc_stderr": 0.0023892815120772322 }, "harness|winogrande|5": { "acc": 0.6637726913970008, "acc_stderr": 0.013277286593993447 } }

配置详情

• harness_arc_challenge_25

  • 分割：2023_08_17T19_20_55.722583, 2023_08_17T23_25_19.325666, 2023_08_17T23_52_04.586597, 2023_08_18T01_05_27.057546, latest
  • 路径：**/details_harness|arc:challenge|25_2023-08-17T19:20:55.722583.parquet 等

• harness_drop_3

  • 分割：2023_10_18T02_31_20.356957, latest
  • 路径：**/details_harness|drop|3_2023-10-18T02-31-20.356957.parquet

• harness_gsm8k_5

  • 分割：2023_10_18T02_31_20.356957, latest
  • 路径：**/details_harness|gsm8k|5_2023-10-18T02-31-20.356957.parquet

• harness_hellaswag_10

  • 分割：2023_08_17T19_20_55.722583, 2023_08_17T23_25_19.325666, 2023_08_17T23_52_04.586597, 2023_08_18T01_05_27.057546, latest
  • 路径：**/details_harness|hellaswag|10_2023-08-17T19:20:55.722583.parquet 等

• harness_hendrycksTest_5

  • 分割：2023_08_17T19_20_55.722583, 2023_08_17T23_25_19.325666, 2023_08_17T23_52_04.586597
  • 路径：**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-17T19:20:55.722583.parquet 等