open-llm-leaderboard/details_Undi95__MLewd-v2.4-13B

该数据集是在评估模型Undi95/MLewd-v2.4-13B时自动生成的，包含3个配置，每个配置对应一个评估任务。数据集由2次运行生成，每次运行的结果作为特定分割存储在配置中，分割名称使用运行的时间戳。train分割始终指向最新结果。此外，还有一个results配置存储所有运行的聚合结果，用于在Open LLM Leaderboard上计算和显示聚合指标。

This dataset was automatically generated during the evaluation of the model Undi95/MLewd-v2.4-13B. It includes three configurations, each mapped to a distinct evaluation task. The dataset is compiled from two evaluation runs, with the outputs of each run stored as a dedicated split within its corresponding configuration, where the split name is derived from the timestamp of the respective run. The `train` split consistently references the most recently generated results. Furthermore, a `results` configuration is provided to store the aggregated results across all runs, which is utilized for computing and presenting aggregated metrics on the Open LLM Leaderboard.

原始信息汇总

数据集概述

数据集描述

• 数据集摘要： 该数据集是在对模型 Undi95/MLewd-v2.4-13B 进行评估运行期间自动创建的，用于 Open LLM Leaderboard。

• 数据集组成： 数据集包含 3 个配置，每个配置对应一个评估任务。

• 数据集创建： 数据集从 2 次运行中创建。每次运行在每个配置中作为一个特定的分割存在，分割名称使用运行的时间戳。"train" 分割始终指向最新的结果。

• 额外配置： 一个额外的配置 "results" 存储所有运行的聚合结果，并用于计算和显示 Open LLM Leaderboard 上的聚合指标。

最新结果

以下是来自 2023-11-06T15:01:09.022171 运行的最新结果：

python { "all": { "em": 0.37153942953020136, "em_stderr": 0.004948586020359345, "f1": 0.4432686661073842, "f1_stderr": 0.0047496461477472855, "acc": 0.4214342261393644, "acc_stderr": 0.010215463395612735 }, "harness|drop|3": { "em": 0.37153942953020136, "em_stderr": 0.004948586020359345, "f1": 0.4432686661073842, "f1_stderr": 0.0047496461477472855 }, "harness|gsm8k|5": { "acc": 0.0978013646702047, "acc_stderr": 0.008182119821849047 }, "harness|winogrande|5": { "acc": 0.745067087608524, "acc_stderr": 0.012248806969376422 } }

数据集结构

配置

• harness_drop_3：

  • 分割：2023_11_05T06_43_46.123528
    • 路径：**/details_harness|drop|3_2023-11-05T06-43-46.123528.parquet
  • 分割：2023_11_06T15_01_09.022171
    • 路径：**/details_harness|drop|3_2023-11-06T15-01-09.022171.parquet
  • 分割：latest
    • 路径：**/details_harness|drop|3_2023-11-06T15-01-09.022171.parquet

• harness_gsm8k_5：

  • 分割：2023_11_05T06_43_46.123528
    • 路径：**/details_harness|gsm8k|5_2023-11-05T06-43-46.123528.parquet
  • 分割：2023_11_06T15_01_09.022171
    • 路径：**/details_harness|gsm8k|5_2023-11-06T15-01-09.022171.parquet
  • 分割：latest
    • 路径：**/details_harness|gsm8k|5_2023-11-06T15-01-09.022171.parquet

• harness_winogrande_5：

  • 分割：2023_11_05T06_43_46.123528
    • 路径：**/details_harness|winogrande|5_2023-11-05T06-43-46.123528.parquet
  • 分割：2023_11_06T15_01_09.022171
    • 路径：**/details_harness|winogrande|5_2023-11-06T15-01-09.022171.parquet
  • 分割：latest
    • 路径：**/details_harness|winogrande|5_2023-11-06T15-01-09.022171.parquet

• results：

  • 分割：2023_11_05T06_43_46.123528
    • 路径：results_2023-11-05T06-43-46.123528.parquet
  • 分割：2023_11_06T15_01_09.022171
    • 路径：results_2023-11-06T15-01-09.022171.parquet
  • 分割：latest
    • 路径：results_2023-11-06T15-01-09.022171.parquet