open-llm-leaderboard-old/details_AiMavenAi__AiMaven-Prometheus

该数据集是在Open LLM Leaderboard上对模型AiMavenAi/AiMaven-Prometheus进行评估运行时自动创建的。数据集由63个配置组成，每个配置对应一个评估任务。数据集是从一个或多个运行中生成的，每个运行在每个配置中表示为特定的分割，train分割始终指向最新结果。此外，还有一个results配置，存储了运行的所有聚合结果，用于计算和显示Open LLM Leaderboard上的聚合指标。README文件还包含一个Python代码片段，展示了如何从运行中加载详细信息，并提供了特定运行的最新结果。

原始信息汇总

数据集概述

该数据集是在模型 AiMavenAi/AiMaven-Prometheus 在 Open LLM Leaderboard 上的评估运行期间自动创建的。

数据集组成

• 数据集包含 63 个配置，每个配置对应一个评估任务。
• 数据集从 1 次运行中创建，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
• "train" 分割始终指向最新的结果。
• 一个额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示在 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_AiMavenAi__AiMaven-Prometheus", "harness_winogrande_5", split="train")

最新结果

以下是 2024-02-02T20:40:21.719204 运行的最新结果：

python { "all": { "acc": 0.6576383819309891, "acc_stderr": 0.03197046028999131, "acc_norm": 0.6572056676322405, "acc_norm_stderr": 0.032638819506296185, "mc1": 0.5789473684210527, "mc1_stderr": 0.017283936248136476, "mc2": 0.7222211820734462, "mc2_stderr": 0.014706410203602844 }, "harness|arc:challenge|25": { "acc": 0.7107508532423208, "acc_stderr": 0.013250012579393443, "acc_norm": 0.7397610921501706, "acc_norm_stderr": 0.01282193022511257 }, "harness|hellaswag|10": { "acc": 0.7157936666002789, "acc_stderr": 0.004501137895230724, "acc_norm": 0.8882692690699064, "acc_norm_stderr": 0.003143910361779264 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.34, "acc_stderr": 0.04760952285695235, "acc_norm": 0.34, "acc_norm_stderr": 0.04760952285695235 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.6814814814814815, "acc_stderr": 0.04024778401977108, "acc_norm": 0.6814814814814815, "acc_norm_stderr": 0.04024778401977108 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.6842105263157895, "acc_stderr": 0.0378272898086547, "acc_norm": 0.6842105263157895, "acc_norm_stderr": 0.0378272898086547 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.64, "acc_stderr": 0.04824181513244218, "acc_norm": 0.64, "acc_norm_stderr": 0.04824181513244218 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.7169811320754716, "acc_stderr": 0.027724236492700914, "acc_norm": 0.7169811320754716, "acc_norm_stderr": 0.027724236492700914 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.7708333333333334, "acc_stderr": 0.03514697467862388, "acc_norm": 0.7708333333333334, "acc_norm_stderr": 0.03514697467862388 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.48, "acc_stderr": 0.050211673156867795, "acc_norm": 0.48, "acc_norm_stderr": 0.050211673156867795 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.54, "acc_stderr": 0.05009082659620333, "acc_norm": 0.54, "acc_norm_stderr": 0.05009082659620333 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.3, "acc_stderr": 0.046056618647183814, "acc_norm": 0.3, "acc_norm_stderr": 0.046056618647183814 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.6705202312138728, "acc_stderr": 0.03583901754736412, "acc_norm": 0.6705202312138728, "acc_norm_stderr": 0.03583901754736412 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.39215686274509803, "acc_stderr": 0.048580835742663454, "acc_norm": 0.39215686274509803, "acc_norm_stderr": 0.048580835742663454 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.74, "acc_stderr": 0.04408440022768078, "acc_norm": 0.74, "acc_norm_stderr": 0.04408440022768078 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.5702127659574469, "acc_stderr": 0.03236214467715564, "acc_norm": 0.5702127659574469, "acc_norm_stderr": 0.03236214467715564 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.4824561403508772, "acc_stderr": 0.04700708033551038, "acc_norm": 0.4824561403508772, "acc_norm_stderr": 0.04700708033551038 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5448275862068965, "acc_stderr": 0.04149886942192117, "acc_norm": 0.5448275862068965, "acc_norm_stderr": 0.04149886942192117 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.42592592592592593, "acc_stderr": 0.025467149045469546, "acc_norm": 0.42592592592592593, "acc_norm_stderr": 0.025467149045469546 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.5079365079365079, "acc_stderr": 0.044715725362943486, "acc_norm": 0.5079365079365079, "acc_norm_stderr": 0.044715725362943486 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.34, "acc_stderr": 0.04760952285695235, "acc_norm": 0.34, "acc_norm_stderr": 0.04760952285695235 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.7967741935483871, "acc_stderr": 0.02289168798455496, "acc_norm": 0.7967741935483871, "acc_norm_stderr": 0.02289168798455496 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.5320197044334976, "acc_stderr": 0.03510766597959215, "acc_norm": 0.5320197044334976, "acc_norm_stderr": 0.03510766597959215 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.69, "acc_stderr": 0.04648231987117316, "acc_norm": 0.69, "acc_norm_stderr": 0.04648231987117316 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.7757575757575758, "acc_stderr": 0.032568666616811015, "acc_norm": 0.7757575757575758, "acc_norm_stderr": 0.032568666616811015 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.8181818181818182, "acc_stderr": 0.0274796030105388, "acc_norm": 0.8181818181818182, "acc_norm_stderr": 0.0274796030105388 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.9119170984455959, "acc_stderr": 0.02045374660160103, "acc_norm": 0.9119170984455959, "acc_norm_stderr": 0.02045374660160103 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.6743589743589744, "acc_stderr": 0.02375966576741229, "acc_norm": 0.6743589743589744, "acc_norm_stderr": 0.02375966576741229 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.337037037037037, "acc_stderr": 0.02882088466625326, "acc_norm": 0.337037037037037, "acc_norm_stderr": 0.02882088466625326