open-llm-leaderboard-old/details_nicholasKluge__Aira-Instruct-124M

该数据集是在评估模型nicholasKluge/Aira-Instruct-124M时自动创建的，评估在Open LLM Leaderboard上进行。数据集由61个配置组成，每个配置对应一个评估任务。数据集包含1次运行的详细信息，每次运行都作为一个特定的分割存储，分割名称使用运行的时间戳。train分割始终指向最新的结果。此外，results配置存储了所有运行的聚合结果，用于在Open LLM Leaderboard上计算和显示聚合指标。

原始信息汇总

数据集概述

数据集简介

该数据集是在对模型 nicholasKluge/Aira-Instruct-124M 进行评估时自动创建的，用于 Open LLM Leaderboard。

数据集结构

• 配置数量：61个配置，每个配置对应一个评估任务。
• 运行次数：数据集来自1次运行。每个运行在每个配置中都有一个特定的分割，分割名称使用运行的时间戳。
• 训练分割："train" 分割始终指向最新的结果。
• 结果配置：一个额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_nicholasKluge__Aira-Instruct-124M", "harness_truthfulqa_mc_0", split="train")

最新结果

以下是 2023-08-10T09:14:16.516035 运行的最新结果：

python { "all": { "acc": 0.25097821278031224, "acc_stderr": 0.03126312568682377, "acc_norm": 0.25197883172295243, "acc_norm_stderr": 0.03127882498671644, "mc1": 0.22766217870257038, "mc1_stderr": 0.014679255032111075, "mc2": 0.3793773096260545, "mc2_stderr": 0.01493606177741941 }, "harness|arc:challenge|25": { "acc": 0.19368600682593856, "acc_stderr": 0.01154842540997854, "acc_norm": 0.2354948805460751, "acc_norm_stderr": 0.012399451855004753 }, "harness|hellaswag|10": { "acc": 0.2909778928500299, "acc_stderr": 0.004532850566893526, "acc_norm": 0.3082055367456682, "acc_norm_stderr": 0.004608082815535503 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.22, "acc_stderr": 0.04163331998932268, "acc_norm": 0.22, "acc_norm_stderr": 0.04163331998932268 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.3111111111111111, "acc_stderr": 0.03999262876617721, "acc_norm": 0.3111111111111111, "acc_norm_stderr": 0.03999262876617721 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.18421052631578946, "acc_stderr": 0.0315469804508223, "acc_norm": 0.18421052631578946, "acc_norm_stderr": 0.0315469804508223 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.23, "acc_stderr": 0.042295258468165065, "acc_norm": 0.23, "acc_norm_stderr": 0.042295258468165065 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.2339622641509434, "acc_stderr": 0.02605529690115292, "acc_norm": 0.2339622641509434, "acc_norm_stderr": 0.02605529690115292 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.2569444444444444, "acc_stderr": 0.03653946969442099, "acc_norm": 0.2569444444444444, "acc_norm_stderr": 0.03653946969442099 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.14, "acc_stderr": 0.03487350880197769, "acc_norm": 0.14, "acc_norm_stderr": 0.03487350880197769 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.29, "acc_stderr": 0.045604802157206845, "acc_norm": 0.29, "acc_norm_stderr": 0.045604802157206845 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.2, "acc_stderr": 0.04020151261036846, "acc_norm": 0.2, "acc_norm_stderr": 0.04020151261036846 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.26011560693641617, "acc_stderr": 0.03345036916788992, "acc_norm": 0.26011560693641617, "acc_norm_stderr": 0.03345036916788992 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.21568627450980393, "acc_stderr": 0.04092563958237654, "acc_norm": 0.21568627450980393, "acc_norm_stderr": 0.04092563958237654 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.25, "acc_stderr": 0.04351941398892446, "acc_norm": 0.25, "acc_norm_stderr": 0.04351941398892446 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.2553191489361702, "acc_stderr": 0.0285048564705142, "acc_norm": 0.2553191489361702, "acc_norm_stderr": 0.0285048564705142 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.2631578947368421, "acc_stderr": 0.04142439719489361, "acc_norm": 0.2631578947368421, "acc_norm_stderr": 0.04142439719489361 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.2413793103448276, "acc_stderr": 0.03565998174135303, "acc_norm": 0.2413793103448276, "acc_norm_stderr": 0.03565998174135303 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.2619047619047619, "acc_stderr": 0.022644212615525218, "acc_norm": 0.2619047619047619, "acc_norm_stderr": 0.022644212615525218 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.15079365079365079, "acc_stderr": 0.03200686497287392, "acc_norm": 0.15079365079365079, "acc_norm_stderr": 0.03200686497287392 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.17, "acc_stderr": 0.0377525168068637, "acc_norm": 0.17, "acc_norm_stderr": 0.0377525168068637 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.2, "acc_stderr": 0.022755204959542936, "acc_norm": 0.2, "acc_norm_stderr": 0.022755204959542936 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.29064039408866993, "acc_stderr": 0.0319474007226554, "acc_norm": 0.29064039408866993, "acc_norm_stderr": 0.0319474007226554 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.32, "acc_stderr": 0.04688261722621505, "acc_norm": 0.32, "acc_norm_stderr": 0.04688261722621505 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.21818181818181817, "acc_stderr": 0.03225078108306289, "acc_norm": 0.21818181818181817, "acc_norm_stderr": 0.03225078108306289 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.32323232323232326, "acc_stderr": 0.03332299921070644, "acc_norm": 0.32323232323232326, "acc_norm_stderr": 0.03332299921070644 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.29015544041450775, "acc_stderr": 0.03275264467791516, "acc_norm": 0.29015544041450775, "acc_norm_stderr": 0.03275264467791516 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.24102564102564103, "acc_stderr": 0.021685546665333195, "acc_norm": 0.24102564102564103, "acc_norm_stderr": 0.021685546665333195 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.22592592592592592, "acc_stderr": 0.02549753263960955, "acc_norm": 0.22592592592592592, "acc_norm_