open-llm-leaderboard/details_Fredithefish__RedPajama-INCITE-Chat-3B-Instruction-Tuning-with-GPT-4

该数据集是在Open LLM Leaderboard上对模型Fredithefish/RedPajama-INCITE-Chat-3B-Instruction-Tuning-with-GPT-4的评估运行过程中自动创建的。数据集由64个配置组成，每个配置对应一个评估任务。数据集由2次运行生成，每次运行的结果作为特定配置中的一个分割，分割名称使用运行的时间戳。train分割始终指向最新的结果。此外，还有一个名为results的配置存储了所有运行的聚合结果，用于在Open LLM Leaderboard上计算和显示聚合指标。

This dataset was automatically created during the evaluation run of the model Fredithefish/RedPajama-INCITE-Chat-3B-Instruction-Tuning-with-GPT-4 on the Open LLM Leaderboard. The dataset consists of 64 configurations, each corresponding to one evaluation task. It is generated from two runs, with the results of each run serving as a split in a specific configuration, where the split name uses the timestamp of the run. The train split always points to the most recent results. Additionally, there is a configuration named "results" that stores the aggregated results of all runs, which is used to calculate and display aggregated metrics on the Open LLM Leaderboard.

原始信息汇总

数据集概述

数据集摘要

该数据集是在评估模型 Fredithefish/RedPajama-INCITE-Chat-3B-Instruction-Tuning-with-GPT-4 在 Open LLM Leaderboard 上的自动创建的。数据集包含64个配置，每个配置对应一个评估任务。数据集从2次运行中创建，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。"train" 分割始终指向最新结果。

数据集结构

数据集包含多个配置，每个配置对应不同的任务和运行结果。以下是部分配置的详细信息：

配置列表

• harness_arc_challenge_25

  • 分割: 2023_07_19T14_47_41.742069
  • 路径: **/details_harness|arc:challenge|25_2023-07-19T14:47:41.742069.parquet
  • 分割: latest
  • 路径: **/details_harness|arc:challenge|25_2023-07-19T14:47:41.742069.parquet

• harness_drop_3

  • 分割: 2023_09_28T15_50_00.560199
  • 路径: **/details_harness|drop|3_2023-09-28T15-50-00.560199.parquet
  • 分割: latest
  • 路径: **/details_harness|drop|3_2023-09-28T15-50-00.560199.parquet

• harness_gsm8k_5

  • 分割: 2023_09_28T15_50_00.560199
  • 路径: **/details_harness|gsm8k|5_2023-09-28T15-50-00.560199.parquet
  • 分割: latest
  • 路径: **/details_harness|gsm8k|5_2023-09-28T15-50-00.560199.parquet

• harness_hellaswag_10

  • 分割: 2023_07_19T14_47_41.742069
  • 路径: **/details_harness|hellaswag|10_2023-07-19T14:47:41.742069.parquet
  • 分割: latest
  • 路径: **/details_harness|hellaswag|10_2023-07-19T14:47:41.742069.parquet

• harness_hendrycksTest_5

  • 分割: 2023_07_19T14_47_41.742069
  • 路径:
    • **/details_harness|hendrycksTest-abstract_algebra|5_2023-07-19T14:47:41.742069.parquet
    • **/details_harness|hendrycksTest-anatomy|5_2023-07-19T14:47:41.742069.parquet
    • ... (其他路径省略)
  • 分割: latest
  • 路径:
    • **/details_harness|hendrycksTest-abstract_algebra|5_2023-07-19T14:47:41.742069.parquet
    • **/details_harness|hendrycksTest-anatomy|5_2023-07-19T14:47:41.742069.parquet
    • ... (其他路径省略)

最新结果

以下是 2023-09-28T15:50:00.560199 运行 的最新结果：

python { "all": { "em": 0.018246644295302015, "em_stderr": 0.0013706682452812888, "f1": 0.0714765100671141, "f1_stderr": 0.0018411955158404013, "acc": 0.32543219642729987, "acc_stderr": 0.007862138879264232 }, "harness|drop|3": { "em": 0.018246644295302015, "em_stderr": 0.0013706682452812888, "f1": 0.0714765100671141, "f1_stderr": 0.0018411955158404013 }, "harness|gsm8k|5": { "acc": 0.006823351023502654, "acc_stderr": 0.0022675371022545044 }, "harness|winogrande|5": { "acc": 0.6440410418310971, "acc_stderr": 0.013456740656273959 } }