open-llm-leaderboard-old/details_lemon-mint__gemma-ko-7b-instruct-v0.71

该数据集是在模型 lemon-mint/gemma-ko-7b-instruct-v0.71 在 Open LLM Leaderboard 上的评估运行期间自动创建的。数据集由 63 个配置组成，每个配置对应一个评估任务。它包含 1 次运行的结果，每次运行在每个配置中表示为特定的分割。train 分割始终指向最新结果。一个名为 results 的额外配置存储了所有运行的聚合结果，这些结果用于计算和显示 Open LLM Leaderboard 上的聚合指标。README 还提供了一个示例，展示了如何使用 Python 中的 datasets 库加载运行中的详细信息。

原始信息汇总

数据集概述

数据集摘要

该数据集是在评估模型lemon-mint/gemma-ko-7b-instruct-v0.71在Open LLM Leaderboard上的运行过程中自动创建的。

数据集组成

• 该数据集包含63个配置，每个配置对应一个评估任务。
• 数据集从1次运行中创建，每个运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
• "train"分割始终指向最新的结果。
• 一个额外的配置"results"存储所有运行的聚合结果，用于计算和显示在Open LLM Leaderboard上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_lemon-mint__gemma-ko-7b-instruct-v0.71", "harness_winogrande_5", split="train")

最新结果

以下是最新结果来自2024-04-09T06:37:34.656936： python { "all": { "acc": 0.5942899564374612, "acc_stderr": 0.03331686070230768, "acc_norm": 0.5993757855966777, "acc_norm_stderr": 0.03398353543840741, "mc1": 0.33414932680538556, "mc1_stderr": 0.016512530677150538, "mc2": 0.5172148516529776, "mc2_stderr": 0.015636005438812176 }, "harness|arc:challenge|25": { "acc": 0.5093856655290102, "acc_stderr": 0.014608816322065, "acc_norm": 0.5503412969283277, "acc_norm_stderr": 0.01453714444428473 }, "harness|hellaswag|10": { "acc": 0.5894244174467238, "acc_stderr": 0.004909328992915072, "acc_norm": 0.7746464847639912, "acc_norm_stderr": 0.004169610254807967 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.3, "acc_stderr": 0.046056618647183814, "acc_norm": 0.3, "acc_norm_stderr": 0.046056618647183814 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.5185185185185185, "acc_stderr": 0.043163785995113245, "acc_norm": 0.5185185185185185, "acc_norm_stderr": 0.043163785995113245 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.625, "acc_stderr": 0.039397364351956274, "acc_norm": 0.625, "acc_norm_stderr": 0.039397364351956274 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.6, "acc_stderr": 0.04923659639173309, "acc_norm": 0.6, "acc_norm_stderr": 0.04923659639173309 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.660377358490566, "acc_stderr": 0.029146904747798335, "acc_norm": 0.660377358490566, "acc_norm_stderr": 0.029146904747798335 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.7291666666666666, "acc_stderr": 0.037161774375660164, "acc_norm": 0.7291666666666666, "acc_norm_stderr": 0.037161774375660164 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.43, "acc_stderr": 0.04975698519562428, "acc_norm": 0.43, "acc_norm_stderr": 0.04975698519562428 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.48, "acc_stderr": 0.050211673156867795, "acc_norm": 0.48, "acc_norm_stderr": 0.050211673156867795 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.38, "acc_stderr": 0.04878317312145634, "acc_norm": 0.38, "acc_norm_stderr": 0.04878317312145634 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5317919075144508, "acc_stderr": 0.03804749744364764, "acc_norm": 0.5317919075144508, "acc_norm_stderr": 0.03804749744364764 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.3627450980392157, "acc_stderr": 0.047840607041056527, "acc_norm": 0.3627450980392157, "acc_norm_stderr": 0.047840607041056527 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.69, "acc_stderr": 0.04648231987117316, "acc_norm": 0.69, "acc_norm_stderr": 0.04648231987117316 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.5574468085106383, "acc_stderr": 0.03246956919789958, "acc_norm": 0.5574468085106383, "acc_norm_stderr": 0.03246956919789958 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.4473684210526316, "acc_stderr": 0.046774730044911984, "acc_norm": 0.4473684210526316, "acc_norm_stderr": 0.046774730044911984 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5517241379310345, "acc_stderr": 0.04144311810878151, "acc_norm": 0.5517241379310345, "acc_norm_stderr": 0.04144311810878151 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.43386243386243384, "acc_stderr": 0.025525034382474894, "acc_norm": 0.43386243386243384, "acc_norm_stderr": 0.025525034382474894 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.4444444444444444, "acc_stderr": 0.04444444444444449, "acc_norm": 0.4444444444444444, "acc_norm_stderr": 0.04444444444444449 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.32, "acc_stderr": 0.04688261722621505, "acc_norm": 0.32, "acc_norm_stderr": 0.04688261722621505 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.7161290322580646, "acc_stderr": 0.025649381063029268, "acc_norm": 0.7161290322580646, "acc_norm_stderr": 0.025649381063029268 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.49261083743842365, "acc_stderr": 0.03517603540361008, "acc_norm": 0.49261083743842365, "acc_norm_stderr": 0.03517603540361008 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.67, "acc_stderr": 0.04725815626252607, "acc_norm": 0.67, "acc_norm_stderr": 0.04725815626252607 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.7090909090909091, "acc_stderr": 0.03546563019624336, "acc_norm": 0.7090909090909091, "acc_norm_stderr": 0.03546563019624336 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.7828282828282829, "acc_stderr": 0.02937661648494562, "acc_norm": 0.7828282828282829, "acc_norm_stderr": 0.02937661648494562 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.8756476683937824, "acc_stderr": 0.023814477086593552, "acc_norm": 0.8756476683937824, "acc_norm_stderr": 0.023814477086593552 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.5923076923076923, "acc_stderr": 0.024915243985987847, "acc_norm": 0.5923076923076923, "acc_norm_stderr": 0.024915243985987847 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.29259259259259257, "acc_stderr": 0.027738969632176088, "acc_norm": 0.29259259259259257, "acc_norm_stderr": 0.027