open-llm-leaderboard/details_synapsoft__Llama-2-7b-chat-hf-flan2022-1.2M

该数据集是在Open LLM Leaderboard上对模型synapsoft/Llama-2-7b-chat-hf-flan2022-1.2M进行评估时自动生成的。数据集包含64个配置，每个配置对应一个被评估的任务。数据集由2次运行生成，每次运行在每个配置中表示为特定的分割，分割名称由运行的时间戳命名。train分割始终指向最新的结果。此外，名为results的配置存储了所有运行的聚合结果，这些结果用于计算和显示Open LLM Leaderboard上的聚合指标。README还提供了如何使用Hugging Face datasets库加载运行中的详细信息的示例。

This dataset was automatically generated during the evaluation of the model synapsoft/Llama-2-7b-chat-hf-flan2022-1.2M on the Open LLM Leaderboard. The dataset contains 64 configurations, with each configuration corresponding to one evaluated task. The dataset is generated from two runs, where each run is represented as a specific split under each configuration, and the split names are named after the timestamps of the corresponding runs. The `train` split always points to the most recent evaluation results. Additionally, the configuration named `results` stores the aggregated results across all runs, which are used to calculate and display the aggregate metrics on the Open LLM Leaderboard. The README also provides examples detailing how to load the runs using the Hugging Face Datasets library.

原始信息汇总

数据集概述

数据集简介

该数据集是在对模型 synapsoft/Llama-2-7b-chat-hf-flan2022-1.2M 进行评估运行期间自动创建的。数据集包含64个配置，每个配置对应一个评估任务。数据集从2次运行中创建，每次运行的结果可以在每个配置中找到，使用运行的时间戳作为分割名称。"train" 分割始终指向最新的结果。

数据集结构

数据集包含以下配置：

• harness_arc_challenge_25
• harness_drop_3
• harness_gsm8k_5
• harness_hellaswag_10
• harness_hendrycksTest_5
• harness_hendrycksTest_abstract_algebra_5
• harness_hendrycksTest_anatomy_5
• harness_hendrycksTest_astronomy_5
• harness_hendrycksTest_business_ethics_5
• harness_hendrycksTest_clinical_knowledge_5
• harness_hendrycksTest_college_biology_5
• harness_hendrycksTest_college_chemistry_5
• harness_hendrycksTest_college_computer_science_5
• harness_hendrycksTest_college_mathematics_5
• harness_hendrycksTest_college_medicine_5
• harness_hendrycksTest_college_physics_5
• harness_hendrycksTest_computer_security_5
• harness_hendrycksTest_conceptual_physics_5
• harness_hendrycksTest_econometrics_5
• harness_hendrycksTest_electrical_engineering_5
• harness_hendrycksTest_elementary_mathematics_5

数据文件

每个配置包含多个分割，每个分割对应不同的运行时间戳。例如：

• harness_arc_challenge_25 包含 2023_09_04T22_45_47.858606 和 latest 分割。
• harness_drop_3 包含 2023_09_23T08_39_00.771555 和 latest 分割。

最新结果

最新结果来自 2023-09-23T08:39:00.771555 运行，包含以下指标： python { "all": { "em": 0.2627936241610738, "em_stderr": 0.004507560917898865, "f1": 0.30115981543624176, "f1_stderr": 0.004494140287139199, "acc": 0.3666975232366727, "acc_stderr": 0.008004674480789642 }, "harness|drop|3": { "em": 0.2627936241610738, "em_stderr": 0.004507560917898865, "f1": 0.30115981543624176, "f1_stderr": 0.004494140287139199 }, "harness|gsm8k|5": { "acc": 0.015163002274450341, "acc_stderr": 0.003366022949726345 }, "harness|winogrande|5": { "acc": 0.7182320441988951, "acc_stderr": 0.01264332601185294 } }