open-llm-leaderboard-old/details_sail__Sailor-1.8B-Chat

该数据集是在模型sail/Sailor-1.8B-Chat的评估运行期间自动创建的，用于在Open LLM Leaderboard上进行评估。数据集由63个配置组成，每个配置对应一个评估任务。数据集从1次运行中创建，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳命名。train分割始终指向最新的结果。此外，results配置存储了所有运行的聚合结果，并用于计算和显示Open LLM Leaderboard上的聚合指标。

原始信息汇总

数据集概述

该数据集是在对模型 sail/Sailor-1.8B-Chat 进行评估运行时自动创建的，用于 Open LLM Leaderboard。

数据集组成

• 数据集包含 63 个配置，每个配置对应一个评估任务。
• 数据集从 1 次运行中创建，每次运行的详细信息可以在每个配置中找到，以运行的时间戳命名的特定分片形式存储。
• "train" 分片始终指向最新的结果。
• 额外的 "results" 配置存储所有运行的聚合结果，用于计算和显示 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_sail__Sailor-1.8B-Chat", "harness_winogrande_5", split="train")

最新结果

以下是 2024-03-11T06:45:47.530917 运行的最新结果：

python { "all": { "acc": 0.3806365170008389, "acc_stderr": 0.034063765218371976, "acc_norm": 0.3858613655901287, "acc_norm_stderr": 0.03490323085374032, "mc1": 0.2423500611995104, "mc1_stderr": 0.015000674373570349, "mc2": 0.38711002085151214, "mc2_stderr": 0.014178729804966983 }, "harness|arc:challenge|25": { "acc": 0.3242320819112628, "acc_stderr": 0.01367881039951882, "acc_norm": 0.3575085324232082, "acc_norm_stderr": 0.014005494275916571 }, "harness|hellaswag|10": { "acc": 0.43069109739095796, "acc_stderr": 0.004941609820763587, "acc_norm": 0.5712009559848635, "acc_norm_stderr": 0.004938930143234449 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.29, "acc_stderr": 0.045604802157206845, "acc_norm": 0.29, "acc_norm_stderr": 0.045604802157206845 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.35555555555555557, "acc_stderr": 0.04135176749720386, "acc_norm": 0.35555555555555557, "acc_norm_stderr": 0.04135176749720386 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.4144736842105263, "acc_stderr": 0.04008973785779206, "acc_norm": 0.4144736842105263, "acc_norm_stderr": 0.04008973785779206 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.45, "acc_stderr": 0.05, "acc_norm": 0.45, "acc_norm_stderr": 0.05 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.37358490566037733, "acc_stderr": 0.029773082713319875, "acc_norm": 0.37358490566037733, "acc_norm_stderr": 0.029773082713319875 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.3194444444444444, "acc_stderr": 0.03899073687357335, "acc_norm": 0.3194444444444444, "acc_norm_stderr": 0.03899073687357335 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.32, "acc_stderr": 0.046882617226215034, "acc_norm": 0.32, "acc_norm_stderr": 0.046882617226215034 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.38, "acc_stderr": 0.04878317312145632, "acc_norm": 0.38, "acc_norm_stderr": 0.04878317312145632 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.35, "acc_stderr": 0.047937248544110196, "acc_norm": 0.35, "acc_norm_stderr": 0.047937248544110196 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.3583815028901734, "acc_stderr": 0.03656343653353157, "acc_norm": 0.3583815028901734, "acc_norm_stderr": 0.03656343653353157 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.20588235294117646, "acc_stderr": 0.04023382273617747, "acc_norm": 0.20588235294117646, "acc_norm_stderr": 0.04023382273617747 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.41, "acc_stderr": 0.049431107042371025, "acc_norm": 0.41, "acc_norm_stderr": 0.049431107042371025 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.3191489361702128, "acc_stderr": 0.03047297336338005, "acc_norm": 0.3191489361702128, "acc_norm_stderr": 0.03047297336338005 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.2631578947368421, "acc_stderr": 0.041424397194893624, "acc_norm": 0.2631578947368421, "acc_norm_stderr": 0.041424397194893624 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.43448275862068964, "acc_stderr": 0.041307408795554966, "acc_norm": 0.43448275862068964, "acc_norm_stderr": 0.041307408795554966 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.30158730158730157, "acc_stderr": 0.023636975996101806, "acc_norm": 0.30158730158730157, "acc_norm_stderr": 0.023636975996101806 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.31746031746031744, "acc_stderr": 0.04163453031302859, "acc_norm": 0.31746031746031744, "acc_norm_stderr": 0.04163453031302859 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.26, "acc_stderr": 0.0440844002276808, "acc_norm": 0.26, "acc_norm_stderr": 0.0440844002276808 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.4, "acc_stderr": 0.02786932057166464, "acc_norm": 0.4, "acc_norm_stderr": 0.02786932057166464 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.33004926108374383, "acc_stderr": 0.033085304262282574, "acc_norm": 0.33004926108374383, "acc_norm_stderr": 0.033085304262282574 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.34, "acc_stderr": 0.04760952285695235, "acc_norm": 0.34, "acc_norm_stderr": 0.04760952285695235 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.5272727272727272, "acc_stderr": 0.03898531605579418, "acc_norm": 0.5272727272727272, "acc_norm_stderr": 0.03898531605579418 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.4696969696969697, "acc_stderr": 0.03555804051763929, "acc_norm": 0.4696969696969697, "acc_norm_stderr": 0.03555804051763929 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.5025906735751295, "acc_stderr": 0.03608390745384487, "acc_norm": 0.5025906735751295, "acc_norm_stderr": 0.03608390745384487 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.4025641025641026, "acc_stderr": 0.024864995159767755, "acc_norm": 0.4025641025641026, "acc_norm_stderr": 0.024864995159767755 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.23703703703703705, "acc_stderr": 0.025928876132766118, "acc_norm": 0.23703703703703705, "acc_norm_stderr": 0.025928876132766118 }, "harness|hendrycksTest-high_school_microeconomics|5