OALL/details_HeshamHaroon__llama-3-instruct-slerp-arabic

数据集 Evaluation run of HeshamHaroon/llama-3-instruct-slerp-arabic 是在模型评估过程中自动生成的，包含136个配置，每个配置对应一个评估任务。数据集由1次运行生成，每次运行的结果作为一个特定的分割存储在配置中，分割名称使用运行的时间戳。train 分割始终指向最新的结果。此外，还有一个名为 results 的配置存储了所有运行的聚合结果。

原始信息汇总

数据集概述

数据集来源

• 数据集自动创建于模型HeshamHaroon/llama-3-instruct-slerp-arabic的评估运行期间。

数据集结构

• 数据集包含136个配置，每个配置对应一个评估任务。
• 数据集从1次运行中创建，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
• "train"分割始终指向最新的结果。
• 额外配置"results"存储所有运行的聚合结果。

数据加载示例

python from datasets import load_dataset data = load_dataset("OALL/details_HeshamHaroon__llama-3-instruct-slerp-arabic", "lighteval_xstory_cloze_ar_0", split="train")

最新结果

• 最新结果来自2024-06-11T16:13:58.470213的运行。
• 结果包括多个任务的准确率（acc_norm）和标准误差（acc_norm_stderr）。 python { "all": { "acc_norm": 0.41605439835036606, "acc_norm_stderr": 0.037750704302305184, "acc": 0.614824619457313, "acc_stderr": 0.012523231571141186 }, "community|acva:Algeria|0": { "acc_norm": 0.5230769230769231, "acc_norm_stderr": 0.0358596530894741 }, "community|acva:Ancient_Egypt|0": { "acc_norm": 0.05396825396825397, "acc_norm_stderr": 0.012751380783465837 }, "community|acva:Arab_Empire|0": { "acc_norm": 0.3132075471698113, "acc_norm_stderr": 0.02854479331905533 }, "community|acva:Arabic_Architecture|0": { "acc_norm": 0.4564102564102564, "acc_norm_stderr": 0.035761230969912135 }, "community|acva:Arabic_Art|0": { "acc_norm": 0.38974358974358975, "acc_norm_stderr": 0.03501424776256372 }, "community|acva:Arabic_Astronomy|0": { "acc_norm": 0.4666666666666667, "acc_norm_stderr": 0.03581804596782233 }, "community|acva:Arabic_Calligraphy|0": { "acc_norm": 0.5058823529411764, "acc_norm_stderr": 0.03137061905442377 }, "community|acva:Arabic_Ceremony|0": { "acc_norm": 0.518918918918919, "acc_norm_stderr": 0.036834092970087065 }, "community|acva:Arabic_Clothing|0": { "acc_norm": 0.5128205128205128, "acc_norm_stderr": 0.03588610523192215 }, "community|acva:Arabic_Culture|0": { "acc_norm": 0.2358974358974359, "acc_norm_stderr": 0.030481516761721537 }, "community|acva:Arabic_Food|0": { "acc_norm": 0.441025641025641, "acc_norm_stderr": 0.0356473293185358 }, "community|acva:Arabic_Funeral|0": { "acc_norm": 0.4, "acc_norm_stderr": 0.050529115263991134 }, "community|acva:Arabic_Geography|0": { "acc_norm": 0.6068965517241379, "acc_norm_stderr": 0.040703290137070705 }, "community|acva:Arabic_History|0": { "acc_norm": 0.30256410256410254, "acc_norm_stderr": 0.03298070870085619 }, "community|acva:Arabic_Language_Origin|0": { "acc_norm": 0.5473684210526316, "acc_norm_stderr": 0.051339113773544845 }, "community|acva:Arabic_Literature|0": { "acc_norm": 0.4689655172413793, "acc_norm_stderr": 0.04158632762097828 }, "community|acva:Arabic_Math|0": { "acc_norm": 0.30256410256410254, "acc_norm_stderr": 0.03298070870085618 }, "community|acva:Arabic_Medicine|0": { "acc_norm": 0.46206896551724136, "acc_norm_stderr": 0.041546596717075474 }, "community|acva:Arabic_Music|0": { "acc_norm": 0.23741007194244604, "acc_norm_stderr": 0.036220593237998276 }, "community|acva:Arabic_Ornament|0": { "acc_norm": 0.5282051282051282, "acc_norm_stderr": 0.03584074674920833 }, "community|acva:Arabic_Philosophy|0": { "acc_norm": 0.5793103448275863, "acc_norm_stderr": 0.0411391498118926 }, "community|acva:Arabic_Physics_and_Chemistry|0": { "acc_norm": 0.5333333333333333, "acc_norm_stderr": 0.03581804596782232 }, "community|acva:Arabic_Wedding|0": { "acc_norm": 0.4256410256410256, "acc_norm_stderr": 0.03549871080367708 }, "community|acva:Bahrain|0": { "acc_norm": 0.3111111111111111, "acc_norm_stderr": 0.06979205927323111 }, "community|acva:Comoros|0": { "acc_norm": 0.37777777777777777, "acc_norm_stderr": 0.07309112127323451 }, "community|acva:Egypt_modern|0": { "acc_norm": 0.3157894736842105, "acc_norm_stderr": 0.04794350420740798 }, "community|acva:InfluenceFromAncientEgypt|0": { "acc_norm": 0.6, "acc_norm_stderr": 0.035172622905632896 }, "community|acva:InfluenceFromByzantium|0": { "acc_norm": 0.7172413793103448, "acc_norm_stderr": 0.03752833958003337 }, "community|acva:InfluenceFromChina|0": { "acc_norm": 0.26666666666666666, "acc_norm_stderr": 0.0317493043641267 }, "community|acva:InfluenceFromGreece|0": { "acc_norm": 0.6307692307692307, "acc_norm_stderr": 0.034648411418637566 }, "community|acva:InfluenceFromIslam|0": { "acc_norm": 0.296551724137931, "acc_norm_stderr": 0.03806142687309993 }, "community|acva:InfluenceFromPersia|0": { "acc_norm": 0.6971428571428572, "acc_norm_stderr": 0.03483414676585986 }, "community|acva:InfluenceFromRome|0": { "acc_norm": 0.5743589743589743, "acc_norm_stderr": 0.03549871080367708 }, "community|acva:Iraq|0": { "acc_norm": 0.5058823529411764, "acc_norm_stderr": 0.05455069703232772 }, "community|acva:Islam_Education|0": { "acc_norm": 0.46153846153846156, "acc_norm_stderr": 0.03579154352544572 }, "community|acva:Islam_branches_and_schools|0": { "acc_norm": 0.4342857142857143, "acc_norm_stderr": 0.037576101528126626 }, "community|acva:Islamic_law_system|0": { "acc_norm": 0.4256410256410256, "acc_norm_stderr": 0.035498710803677086 }, "community|acva:Jordan|0": { "acc_norm": 0.3333333333333333, "acc_norm_stderr": 0.07106690545187012 }, "community|acva:Kuwait|0": { "acc_norm": 0.26666666666666666, "acc_norm_stderr": 0.06666666666666667 }, "community|acva:Lebanon|0": { "acc_norm": 0.17777777777777778, "acc_norm_stderr": 0.05763774795025094 }, "community|acva:Libya|0": { "acc_norm": 0.4444444444444444, "acc_norm_stderr": 0.07491109582924914 }, "community|acva:Mauritania|0": { "acc_norm": 0.4222222222222222, "acc_norm_stderr": 0.07446027270295805 }, "community|acva:Mesopotamia_civilization|0": { "acc_norm": 0.5225806451612903, "acc_norm_stderr": 0.0402500394824441 }, "community|acva:Morocco|0": { "acc_norm": 0.2222222222222222, "acc_norm_stderr": 0.06267511942419628 }, "community|acva:Oman|0": { "acc_norm": 0.17777777777777778, "acc_norm_stderr": 0.05763774795025094 }, "community|acva:Palestine|0": { "acc_norm": 0.24705882352941178, "acc_norm_stderr": 0.047058823529411785 }, "community|acva:Qatar|0": { "acc_norm": 0.4, "acc_norm_stderr": 0.07385489458759964 }, "community|acva:Saudi_Arabia|0": { "acc_norm": 0.3282051282051282, "acc_norm_stderr": 0.03371243782413707 }, "community|acva:Somalia|0": { "acc_norm": 0.37777777777777777, "acc_norm_stderr": 0.07309112127323451 }, "community|acva:Sudan|0": { "