open-llm-leaderboard-old/details_4season__alignment_model_test

该数据集是在评估模型4season/alignment_model_test时自动创建的，用于Open LLM Leaderboard的评估。数据集包含63个配置，每个配置对应一个评估任务。数据集由1次运行生成，每次运行的结果存储为特定配置中的一个分割，分割名称使用运行的时间戳。train分割始终指向最新的结果。此外，results配置存储了所有运行的聚合结果，用于计算和显示Open LLM Leaderboard上的聚合指标。

原始信息汇总

数据集概述

该数据集是在对模型 4season/alignment_model_test 进行评估运行期间自动创建的，用于 Open LLM Leaderboard。

数据集组成

• 数据集包含 63 个配置，每个配置对应一个评估任务。
• 数据集从 1 次运行中创建，每次运行可以在每个配置中找到特定的拆分，拆分名称使用运行的时间戳。
• "train" 拆分始终指向最新的结果。
• 一个额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_4season__alignment_model_test", "harness_winogrande_5", split="train")

最新结果

以下是 2024-03-16T13:39:48.127832 运行的最新结果：

python { "all": { "acc": 0.6847233774186882, "acc_stderr": 0.031376918102632344, "acc_norm": 0.6861672788340304, "acc_norm_stderr": 0.03201970285060687, "mc1": 0.6940024479804161, "mc1_stderr": 0.016132229728155038, "mc2": 0.8088413049033801, "mc2_stderr": 0.013121290704624325 }, "harness|arc:challenge|25": { "acc": 0.7627986348122867, "acc_stderr": 0.012430399829260856, "acc_norm": 0.7824232081911263, "acc_norm_stderr": 0.012057262020972499 }, "harness|hellaswag|10": { "acc": 0.7511451902011551, "acc_stderr": 0.004314659034649386, "acc_norm": 0.8968333001394144, "acc_norm_stderr": 0.003035548306420554 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.4, "acc_stderr": 0.04923659639173309, "acc_norm": 0.4, "acc_norm_stderr": 0.04923659639173309 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.6222222222222222, "acc_stderr": 0.04188307537595853, "acc_norm": 0.6222222222222222, "acc_norm_stderr": 0.04188307537595853 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.8092105263157895, "acc_stderr": 0.031975658210325, "acc_norm": 0.8092105263157895, "acc_norm_stderr": 0.031975658210325 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.73, "acc_stderr": 0.04461960433384741, "acc_norm": 0.73, "acc_norm_stderr": 0.04461960433384741 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.7396226415094339, "acc_stderr": 0.027008766090708045, "acc_norm": 0.7396226415094339, "acc_norm_stderr": 0.027008766090708045 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.8194444444444444, "acc_stderr": 0.03216600808802268, "acc_norm": 0.8194444444444444, "acc_norm_stderr": 0.03216600808802268 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.48, "acc_stderr": 0.050211673156867795, "acc_norm": 0.48, "acc_norm_stderr": 0.050211673156867795 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.62, "acc_stderr": 0.04878317312145632, "acc_norm": 0.62, "acc_norm_stderr": 0.04878317312145632 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.38, "acc_stderr": 0.04878317312145633, "acc_norm": 0.38, "acc_norm_stderr": 0.04878317312145633 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.6936416184971098, "acc_stderr": 0.035149425512674394, "acc_norm": 0.6936416184971098, "acc_norm_stderr": 0.035149425512674394 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.4411764705882353, "acc_stderr": 0.049406356306056595, "acc_norm": 0.4411764705882353, "acc_norm_stderr": 0.049406356306056595 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.82, "acc_stderr": 0.038612291966536955, "acc_norm": 0.82, "acc_norm_stderr": 0.038612291966536955 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.6595744680851063, "acc_stderr": 0.03097669299853443, "acc_norm": 0.6595744680851063, "acc_norm_stderr": 0.03097669299853443 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.5263157894736842, "acc_stderr": 0.046970851366478626, "acc_norm": 0.5263157894736842, "acc_norm_stderr": 0.046970851366478626 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.6551724137931034, "acc_stderr": 0.03960933549451207, "acc_norm": 0.6551724137931034, "acc_norm_stderr": 0.03960933549451207 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.4894179894179894, "acc_stderr": 0.02574554227604548, "acc_norm": 0.4894179894179894, "acc_norm_stderr": 0.02574554227604548 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.49206349206349204, "acc_stderr": 0.044715725362943486, "acc_norm": 0.49206349206349204, "acc_norm_stderr": 0.044715725362943486 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.32, "acc_stderr": 0.046882617226215034, "acc_norm": 0.32, "acc_norm_stderr": 0.046882617226215034 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.8387096774193549, "acc_stderr": 0.020923327006423294, "acc_norm": 0.8387096774193549, "acc_norm_stderr": 0.020923327006423294 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.5467980295566502, "acc_stderr": 0.03502544650845872, "acc_norm": 0.5467980295566502, "acc_norm_stderr": 0.03502544650845872 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.76, "acc_stderr": 0.04292346959909283, "acc_norm": 0.76, "acc_norm_stderr": 0.04292346959909283 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.8181818181818182, "acc_stderr": 0.030117688929503564, "acc_norm": 0.8181818181818182, "acc_norm_stderr": 0.030117688929503564 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.8535353535353535, "acc_stderr": 0.025190921114603908, "acc_norm": 0.8535353535353535, "acc_norm_stderr": 0.025190921114603908 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.8963730569948186, "acc_stderr": 0.021995311963644244, "acc_norm": 0.8963730569948186, "acc_norm_stderr": 0.021995311963644244 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.7025641025641025, "acc_stderr": 0.023177408131465946, "acc_norm": 0.7025641025641025, "acc_norm_stderr": 0.023177408131465946 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.3962962962962963, "acc_stderr": 0.029822619458534004, "acc_norm": 0.3962962962962963, "acc_norm_stderr": 0.029822619458