open-llm-leaderboard-old/details_Telugu-LLM-Labs__Indic-gemma-7b-finetuned-sft-Navarasa-2.0

该数据集是在评估模型Telugu-LLM-Labs/Indic-gemma-7b-finetuned-sft-Navarasa-2.0时自动创建的，包含63个配置，每个配置对应一个评估任务。数据集由1次运行生成，每次运行的结果作为一个特定的分割存储在配置中，分割名称使用运行的时间戳。train分割始终指向最新的结果。此外，results配置存储了所有运行的聚合结果，用于计算和显示在Open LLM Leaderboard上的聚合指标。

原始信息汇总

数据集概述

数据集简介

该数据集是在对模型 Telugu-LLM-Labs/Indic-gemma-7b-finetuned-sft-Navarasa-2.0 进行评估运行时自动创建的，评估结果展示在 Open LLM Leaderboard 上。

数据集组成

• 数据集包含 63 个配置，每个配置对应一个评估任务。
• 数据集从 1 次运行中创建，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
• "train" 分割始终指向最新的结果。
• 额外的 "results" 配置存储所有运行的聚合结果，用于计算和显示聚合指标在 Open LLM Leaderboard 上。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_Telugu-LLM-Labs__Indic-gemma-7b-finetuned-sft-Navarasa-2.0", "harness_winogrande_5", split="train")

最新结果

以下是 2024-03-22T17:31:24.848459 运行的最新结果：

python { "all": { "acc": 0.540078202110863, "acc_stderr": 0.033879824791246044, "acc_norm": 0.5449910711013712, "acc_norm_stderr": 0.03457989471408299, "mc1": 0.3268053855569155, "mc1_stderr": 0.016419874731135032, "mc2": 0.49588787997553246, "mc2_stderr": 0.015429495608726489 }, "harness|arc:challenge|25": { "acc": 0.515358361774744, "acc_stderr": 0.014604496129394908, "acc_norm": 0.5460750853242321, "acc_norm_stderr": 0.01454922110517187 }, "harness|hellaswag|10": { "acc": 0.54690300736905, "acc_stderr": 0.0049677789400119346, "acc_norm": 0.7434773949412468, "acc_norm_stderr": 0.0043582106894422675 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.3, "acc_stderr": 0.046056618647183814, "acc_norm": 0.3, "acc_norm_stderr": 0.046056618647183814 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.4740740740740741, "acc_stderr": 0.04313531696750575, "acc_norm": 0.4740740740740741, "acc_norm_stderr": 0.04313531696750575 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.5855263157894737, "acc_stderr": 0.04008973785779205, "acc_norm": 0.5855263157894737, "acc_norm_stderr": 0.04008973785779205 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.59, "acc_stderr": 0.04943110704237102, "acc_norm": 0.59, "acc_norm_stderr": 0.04943110704237102 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.5584905660377358, "acc_stderr": 0.030561590426731837, "acc_norm": 0.5584905660377358, "acc_norm_stderr": 0.030561590426731837 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.5833333333333334, "acc_stderr": 0.041227287076512825, "acc_norm": 0.5833333333333334, "acc_norm_stderr": 0.041227287076512825 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.44, "acc_stderr": 0.049888765156985884, "acc_norm": 0.44, "acc_norm_stderr": 0.049888765156985884 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.41, "acc_stderr": 0.049431107042371025, "acc_norm": 0.41, "acc_norm_stderr": 0.049431107042371025 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.37, "acc_stderr": 0.04852365870939099, "acc_norm": 0.37, "acc_norm_stderr": 0.04852365870939099 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5144508670520231, "acc_stderr": 0.03810871630454764, "acc_norm": 0.5144508670520231, "acc_norm_stderr": 0.03810871630454764 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.28431372549019607, "acc_stderr": 0.04488482852329017, "acc_norm": 0.28431372549019607, "acc_norm_stderr": 0.04488482852329017 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.63, "acc_stderr": 0.04852365870939099, "acc_norm": 0.63, "acc_norm_stderr": 0.04852365870939099 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.5234042553191489, "acc_stderr": 0.03265019475033581, "acc_norm": 0.5234042553191489, "acc_norm_stderr": 0.03265019475033581 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.34210526315789475, "acc_stderr": 0.04462917535336936, "acc_norm": 0.34210526315789475, "acc_norm_stderr": 0.04462917535336936 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.503448275862069, "acc_stderr": 0.04166567577101579, "acc_norm": 0.503448275862069, "acc_norm_stderr": 0.04166567577101579 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.37566137566137564, "acc_stderr": 0.024942368931159795, "acc_norm": 0.37566137566137564, "acc_norm_stderr": 0.024942368931159795 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.30952380952380953, "acc_stderr": 0.04134913018303316, "acc_norm": 0.30952380952380953, "acc_norm_stderr": 0.04134913018303316 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.29, "acc_stderr": 0.04560480215720683, "acc_norm": 0.29, "acc_norm_stderr": 0.04560480215720683 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.6451612903225806, "acc_stderr": 0.027218889773308757, "acc_norm": 0.6451612903225806, "acc_norm_stderr": 0.027218889773308757 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.3645320197044335, "acc_stderr": 0.0338640574606209, "acc_norm": 0.3645320197044335, "acc_norm_stderr": 0.0338640574606209 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.56, "acc_stderr": 0.04988876515698589, "acc_norm": 0.56, "acc_norm_stderr": 0.04988876515698589 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.7212121212121212, "acc_stderr": 0.0350143870629678, "acc_norm": 0.7212121212121212, "acc_norm_stderr": 0.0350143870629678 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.7323232323232324, "acc_stderr": 0.03154449888270286, "acc_norm": 0.7323232323232324, "acc_norm_stderr": 0.03154449888270286 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.7616580310880829, "acc_stderr": 0.030748905363909868, "acc_norm": 0.7616580310880829, "acc_norm_stderr": 0.030748905363909868 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.4897435897435897, "acc_stderr": 0.025345672221942374, "acc_norm": 0.4897435897435897, "acc_norm_stderr": 0.025345672221942374 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.270370