Name: nyu-dice-lab/lm-eval-results-zhengr-MixTAO-7Bx2-MoE-v8.1-private
Creator: nyu-dice-lab
Published: 2024-12-05 05:38:25
License: 暂无描述

下载链接：

https://hf-mirror.com/datasets/nyu-dice-lab/lm-eval-results-zhengr-MixTAO-7Bx2-MoE-v8.1-private

下载链接

链接失效反馈

官方服务：

资源简介：

--- pretty_name: Evaluation run of zhengr/MixTAO-7Bx2-MoE-v8.1 dataset_summary: "Dataset automatically created during the evaluation run of model\ \ [zhengr/MixTAO-7Bx2-MoE-v8.1](https://huggingface.co/zhengr/MixTAO-7Bx2-MoE-v8.1)\n\ The dataset is composed of 62 configuration(s), each one corresponding to one of\ \ the evaluated task.\n\nThe dataset has been created from 2 run(s). Each run can\ \ be found as a specific split in each configuration, the split being named using\ \ the timestamp of the run.The \"train\" split is always pointing to the latest\ \ results.\n\nAn additional configuration \"results\" store all the aggregated results\ \ of the run.\n\nTo load the details from a run, you can for instance do the following:\n\ ```python\nfrom datasets import load_dataset\ndata = load_dataset(\n\t\"nyu-dice-lab/lm-eval-results-zhengr-MixTAO-7Bx2-MoE-v8.1-private\"\ ,\n\tname=\"zhengr__MixTAO-7Bx2-MoE-v8.1__BeaverTailsEval\",\n\tsplit=\"latest\"\ \n)\n```\n\n## Latest results\n\nThese are the [latest results from run 2024-12-05T00-28-46.710222](https://huggingface.co/datasets/nyu-dice-lab/lm-eval-results-zhengr-MixTAO-7Bx2-MoE-v8.1-private/blob/main/zhengr/MixTAO-7Bx2-MoE-v8.1/results_2024-12-05T00-28-46.710222.json)\ \ (note that there might be results for other tasks in the repos if successive evals\ \ didn't cover the same tasks. You find each in the results and the \"latest\" split\ \ for each eval):\n\n```python\n{\n \"all\": {\n \"BeaverTailsEval\":\ \ {\n \"alias\": \"BeaverTailsEval\",\n \"acc,none\": 0.8814285714285715,\n\ \ \"acc_stderr,none\": 0.01222770634375575,\n \"acc_norm,none\"\ : 0.13,\n \"acc_norm_stderr,none\": 0.012720164114794184\n },\n\ \ \"CDNA\": {\n \"alias\": \"CDNA\",\n \"acc,none\"\ : 0.9534115920763023,\n \"acc_stderr,none\": 0.004037345306870679,\n\ \ \"acc_norm,none\": 0.01173881144534116,\n \"acc_norm_stderr,none\"\ : 0.0020633109360408777\n },\n \"DTToxicity\": {\n \"alias\"\ : \"DTToxicity\",\n \"acc,none\": 0.4578464106844741,\n \"\ acc_stderr,none\": 0.010180481445723958,\n \"acc_norm,none\": 0.5029215358931552,\n\ \ \"acc_norm_stderr,none\": 0.010216680957241337\n },\n \ \ \"JailbreakHub\": {\n \"alias\": \"JailbreakHub\",\n \"\ acc,none\": 0.15792602377807133,\n \"acc_stderr,none\": 0.002963830609421817,\n\ \ \"acc_norm,none\": 0.09478203434610304,\n \"acc_norm_stderr,none\"\ : 0.002380625059276766\n },\n \"SGXSTest\": {\n \"alias\"\ : \"SGXSTest\",\n \"acc,none\": 0.51,\n \"acc_stderr,none\"\ : 0.035436970729343674,\n \"acc_norm,none\": 0.5,\n \"acc_norm_stderr,none\"\ : 0.0354440602504168\n },\n \"SaladBench\": {\n \"alias\"\ : \"SaladBench\",\n \"acc,none\": 0.49505208333333334,\n \"\ acc_stderr,none\": 0.008069370988058294,\n \"acc_norm,none\": 0.49505208333333334,\n\ \ \"acc_norm_stderr,none\": 0.008069370988058294\n },\n \ \ \"StrongREJECT\": {\n \"alias\": \"StrongREJECT\",\n \"\ acc,none\": 0.9424920127795527,\n \"acc_stderr,none\": 0.013180310031262344,\n\ \ \"acc_norm,none\": 0.19808306709265175,\n \"acc_norm_stderr,none\"\ : 0.022563739178544238\n },\n \"WildGuardTest\": {\n \"\ alias\": \"WildGuardTest\",\n \"acc,none\": 0.5710144927536231,\n \ \ \"acc_stderr,none\": 0.011919999713357268,\n \"acc_norm,none\"\ : 0.5640579710144927,\n \"acc_norm_stderr,none\": 0.011942839896636795\n\ \ },\n \"bbq\": {\n \"acc_norm,none\": 0.9325377829446762,\n\ \ \"acc_norm_stderr,none\": 0.0010218971033224293,\n \"acc,none\"\ : 0.9324864938795049,\n \"acc_stderr,none\": 0.0010226628876829763,\n\ \ \"alias\": \"bbq\"\n },\n \"bbq_age\": {\n \ \ \"alias\": \" - bbq_age\",\n \"acc,none\": 0.8301630434782609,\n \ \ \"acc_stderr,none\": 0.006190602280652597,\n \"acc_norm,none\"\ : 0.8301630434782609,\n \"acc_norm_stderr,none\": 0.006190602280652598\n\ \ },\n \"bbq_disabilitystatus\": {\n \"alias\": \" - bbq_disabilitystatus\"\ ,\n \"acc,none\": 0.9042416452442159,\n \"acc_stderr,none\"\ : 0.007462174379038646,\n \"acc_norm,none\": 0.9048843187660668,\n \ \ \"acc_norm_stderr,none\": 0.007439733789161572\n },\n \"\ bbq_genderidentity\": {\n \"alias\": \" - bbq_genderidentity\",\n \ \ \"acc,none\": 0.9442877291960508,\n \"acc_stderr,none\": 0.00304577473375189,\n\ \ \"acc_norm,none\": 0.942524682651622,\n \"acc_norm_stderr,none\"\ : 0.003090702699187024\n },\n \"bbq_nationality\": {\n \ \ \"alias\": \" - bbq_nationality\",\n \"acc,none\": 0.9168831168831169,\n\ \ \"acc_stderr,none\": 0.00497504218811962,\n \"acc_norm,none\"\ : 0.9181818181818182,\n \"acc_norm_stderr,none\": 0.0049395161759693985\n\ \ },\n \"bbq_physicalappearance\": {\n \"alias\": \" -\ \ bbq_physicalappearance\",\n \"acc,none\": 0.8299492385786802,\n \ \ \"acc_stderr,none\": 0.009466180402043615,\n \"acc_norm,none\"\ : 0.8280456852791879,\n \"acc_norm_stderr,none\": 0.009508092739152258\n\ \ },\n \"bbq_raceethnicity\": {\n \"alias\": \" - bbq_raceethnicity\"\ ,\n \"acc,none\": 0.9156976744186046,\n \"acc_stderr,none\"\ : 0.0033499103029905916,\n \"acc_norm,none\": 0.915406976744186,\n \ \ \"acc_norm_stderr,none\": 0.0033551483498529358\n },\n \ \ \"bbq_racexgender\": {\n \"alias\": \" - bbq_racexgender\",\n \ \ \"acc,none\": 0.9584586466165413,\n \"acc_stderr,none\": 0.0015795165224040926,\n\ \ \"acc_norm,none\": 0.9586466165413534,\n \"acc_norm_stderr,none\"\ : 0.0015760934304819407\n },\n \"bbq_racexses\": {\n \"\ alias\": \" - bbq_racexses\",\n \"acc,none\": 0.9760752688172043,\n \ \ \"acc_stderr,none\": 0.0014466136180125736,\n \"acc_norm,none\"\ : 0.9779569892473118,\n \"acc_norm_stderr,none\": 0.0013898972475625852\n\ \ },\n \"bbq_religion\": {\n \"alias\": \" - bbq_religion\"\ ,\n \"acc,none\": 0.835,\n \"acc_stderr,none\": 0.010719526896310953,\n\ \ \"acc_norm,none\": 0.8341666666666666,\n \"acc_norm_stderr,none\"\ : 0.01074119842135935\n },\n \"bbq_ses\": {\n \"alias\"\ : \" - bbq_ses\",\n \"acc,none\": 0.921037296037296,\n \"\ acc_stderr,none\": 0.003255311967010792,\n \"acc_norm,none\": 0.9195804195804196,\n\ \ \"acc_norm_stderr,none\": 0.0032826060053440993\n },\n \ \ \"bbq_sexualorientation\": {\n \"alias\": \" - bbq_sexualorientation\"\ ,\n \"acc,none\": 0.9016203703703703,\n \"acc_stderr,none\"\ : 0.010138157908353061,\n \"acc_norm,none\": 0.9016203703703703,\n \ \ \"acc_norm_stderr,none\": 0.01013815790835306\n },\n \"\ leaderboard\": {\n \" \": \" \",\n \"alias\": \"leaderboard\"\ \n },\n \"leaderboard_bbh\": {\n \" \": \" \",\n \ \ \"alias\": \" - leaderboard_bbh\"\n },\n \"leaderboard_bbh_boolean_expressions\"\ : {\n \"alias\": \" - leaderboard_bbh_boolean_expressions\",\n \ \ \"acc_norm,none\": 0.8,\n \"acc_norm_stderr,none\": 0.025348970020979078\n\ \ },\n \"leaderboard_bbh_causal_judgement\": {\n \"alias\"\ : \" - leaderboard_bbh_causal_judgement\",\n \"acc_norm,none\": 0.6096256684491979,\n\ \ \"acc_norm_stderr,none\": 0.03576973947986412\n },\n \ \ \"leaderboard_bbh_date_understanding\": {\n \"alias\": \" - leaderboard_bbh_date_understanding\"\ ,\n \"acc_norm,none\": 0.472,\n \"acc_norm_stderr,none\":\ \ 0.031636489531544396\n },\n \"leaderboard_bbh_disambiguation_qa\"\ : {\n \"alias\": \" - leaderboard_bbh_disambiguation_qa\",\n \ \ \"acc_norm,none\": 0.668,\n \"acc_norm_stderr,none\": 0.02984403904746591\n\ \ },\n \"leaderboard_bbh_formal_fallacies\": {\n \"alias\"\ : \" - leaderboard_bbh_formal_fallacies\",\n \"acc_norm,none\": 0.612,\n\ \ \"acc_norm_stderr,none\": 0.03088103874899391\n },\n \ \ \"leaderboard_bbh_geometric_shapes\": {\n \"alias\": \" - leaderboard_bbh_geometric_shapes\"\ ,\n \"acc_norm,none\": 0.376,\n \"acc_norm_stderr,none\":\ \ 0.030696336267394583\n },\n \"leaderboard_bbh_hyperbaton\": {\n\ \ \"alias\": \" - leaderboard_bbh_hyperbaton\",\n \"acc_norm,none\"\ : 0.708,\n \"acc_norm_stderr,none\": 0.02881432040220564\n },\n\ \ \"leaderboard_bbh_logical_deduction_five_objects\": {\n \"alias\"\ : \" - leaderboard_bbh_logical_deduction_five_objects\",\n \"acc_norm,none\"\ : 0.472,\n \"acc_norm_stderr,none\": 0.031636489531544396\n },\n\ \ \"leaderboard_bbh_logical_deduction_seven_objects\": {\n \"\ alias\": \" - leaderboard_bbh_logical_deduction_seven_objects\",\n \"\ acc_norm,none\": 0.448,\n \"acc_norm_stderr,none\": 0.03151438761115355\n\ \ },\n \"leaderboard_bbh_logical_deduction_three_objects\": {\n \ \ \"alias\": \" - leaderboard_bbh_logical_deduction_three_objects\",\n\ \ \"acc_norm,none\": 0.724,\n \"acc_norm_stderr,none\": 0.028328537274211352\n\ \ },\n \"leaderboard_bbh_movie_recommendation\": {\n \"\ alias\": \" - leaderboard_bbh_movie_recommendation\",\n \"acc_norm,none\"\ : 0.668,\n \"acc_norm_stderr,none\": 0.02984403904746591\n },\n\ \ \"leaderboard_bbh_navigate\": {\n \"alias\": \" - leaderboard_bbh_navigate\"\ ,\n \"acc_norm,none\": 0.58,\n \"acc_norm_stderr,none\": 0.03127799950463661\n\ \ },\n \"leaderboard_bbh_object_counting\": {\n \"alias\"\ : \" - leaderboard_bbh_object_counting\",\n \"acc_norm,none\": 0.34,\n\ \ \"acc_norm_stderr,none\": 0.030020073605457907\n },\n \ \ \"leaderboard_bbh_penguins_in_a_table\": {\n \"alias\": \" - leaderboard_bbh_penguins_in_a_table\"\ ,\n \"acc_norm,none\": 0.4657534246575342,\n \"acc_norm_stderr,none\"\ : 0.04142522736934775\n },\n \"leaderboard_bbh_reasoning_about_colored_objects\"\ : {\n \"alias\": \" - leaderboard_bbh_reasoning_about_colored_objects\"\ ,\n \"acc_norm,none\": 0.56,\n \"acc_norm_stderr,none\": 0.03145724452223563\n\ \ },\n \"leaderboard_bbh_ruin_names\": {\n \"alias\": \"\ \ - leaderboard_bbh_ruin_names\",\n \"acc_norm,none\": 0.632,\n \ \ \"acc_norm_stderr,none\": 0.030562070620993163\n },\n \"\ leaderboard_bbh_salient_translation_error_detection\": {\n \"alias\"\ : \" - leaderboard_bbh_salient_translation_error_detection\",\n \"acc_norm,none\"\ : 0.476,\n \"acc_norm_stderr,none\": 0.03164968895968782\n },\n\ \ \"leaderboard_bbh_snarks\": {\n \"alias\": \" - leaderboard_bbh_snarks\"\ ,\n \"acc_norm,none\": 0.7078651685393258,\n \"acc_norm_stderr,none\"\ : 0.03418064934010763\n },\n \"leaderboard_bbh_sports_understanding\"\ : {\n \"alias\": \" - leaderboard_bbh_sports_understanding\",\n \ \ \"acc_norm,none\": 0.78,\n \"acc_norm_stderr,none\": 0.026251792824605824\n\ \ },\n \"leaderboard_bbh_temporal_sequences\": {\n \"alias\"\ : \" - leaderboard_bbh_temporal_sequences\",\n \"acc_norm,none\": 0.256,\n\ \ \"acc_norm_stderr,none\": 0.02765710871820491\n },\n \ \ \"leaderboard_bbh_tracking_shuffled_objects_five_objects\": {\n \"\ alias\": \" - leaderboard_bbh_tracking_shuffled_objects_five_objects\",\n \ \ \"acc_norm,none\": 0.192,\n \"acc_norm_stderr,none\": 0.024960691989172015\n\ \ },\n \"leaderboard_bbh_tracking_shuffled_objects_seven_objects\"\ : {\n \"alias\": \" - leaderboard_bbh_tracking_shuffled_objects_seven_objects\"\ ,\n \"acc_norm,none\": 0.14,\n \"acc_norm_stderr,none\": 0.021989409645240272\n\ \ },\n \"leaderboard_bbh_tracking_shuffled_objects_three_objects\"\ : {\n \"alias\": \" - leaderboard_bbh_tracking_shuffled_objects_three_objects\"\ ,\n \"acc_norm,none\": 0.3,\n \"acc_norm_stderr,none\": 0.029040893477575845\n\ \ },\n \"leaderboard_bbh_web_of_lies\": {\n \"alias\":\ \ \" - leaderboard_bbh_web_of_lies\",\n \"acc_norm,none\": 0.48,\n \ \ \"acc_norm_stderr,none\": 0.03166085340849519\n },\n \"\ leaderboard_gpqa\": {\n \" \": \" \",\n \"alias\": \" - leaderboard_gpqa\"\ \n },\n \"leaderboard_gpqa_diamond\": {\n \"alias\": \"\ \ - leaderboard_gpqa_diamond\",\n \"acc_norm,none\": 0.29292929292929293,\n\ \ \"acc_norm_stderr,none\": 0.032424979581788166\n },\n \ \ \"leaderboard_gpqa_extended\": {\n \"alias\": \" - leaderboard_gpqa_extended\"\ ,\n \"acc_norm,none\": 0.30952380952380953,\n \"acc_norm_stderr,none\"\ : 0.019802641880170335\n },\n \"leaderboard_gpqa_main\": {\n \ \ \"alias\": \" - leaderboard_gpqa_main\",\n \"acc_norm,none\"\ : 0.29017857142857145,\n \"acc_norm_stderr,none\": 0.02146611544057122\n\ \ },\n \"leaderboard_ifeval\": {\n \"alias\": \" - leaderboard_ifeval\"\ ,\n \"prompt_level_strict_acc,none\": 0.3826247689463956,\n \ \ \"prompt_level_strict_acc_stderr,none\": 0.020915307841436647,\n \ \ \"inst_level_strict_acc,none\": 0.5179856115107914,\n \"inst_level_strict_acc_stderr,none\"\ : \"N/A\",\n \"prompt_level_loose_acc,none\": 0.4602587800369686,\n \ \ \"prompt_level_loose_acc_stderr,none\": 0.02144850143413504,\n \ \ \"inst_level_loose_acc,none\": 0.5935251798561151,\n \"inst_level_loose_acc_stderr,none\"\ : \"N/A\"\n },\n \"leaderboard_math_hard\": {\n \" \":\ \ \" \",\n \"alias\": \" - leaderboard_math_hard\"\n },\n \ \ \"leaderboard_math_algebra_hard\": {\n \"alias\": \" - leaderboard_math_algebra_hard\"\ ,\n \"exact_match,none\": 0.11400651465798045,\n \"exact_match_stderr,none\"\ : 0.018168513062629208\n },\n \"leaderboard_math_counting_and_prob_hard\"\ : {\n \"alias\": \" - leaderboard_math_counting_and_prob_hard\",\n \ \ \"exact_match,none\": 0.04878048780487805,\n \"exact_match_stderr,none\"\ : 0.019502196558588064\n },\n \"leaderboard_math_geometry_hard\":\ \ {\n \"alias\": \" - leaderboard_math_geometry_hard\",\n \ \ \"exact_match,none\": 0.022727272727272728,\n \"exact_match_stderr,none\"\ : 0.01302104690906372\n },\n \"leaderboard_math_intermediate_algebra_hard\"\ : {\n \"alias\": \" - leaderboard_math_intermediate_algebra_hard\",\n\ \ \"exact_match,none\": 0.02857142857142857,\n \"exact_match_stderr,none\"\ : 0.009973998820736029\n },\n \"leaderboard_math_num_theory_hard\"\ : {\n \"alias\": \" - leaderboard_math_num_theory_hard\",\n \ \ \"exact_match,none\": 0.07142857142857142,\n \"exact_match_stderr,none\"\ : 0.020820824576076355\n },\n \"leaderboard_math_prealgebra_hard\"\ : {\n \"alias\": \" - leaderboard_math_prealgebra_hard\",\n \ \ \"exact_match,none\": 0.19689119170984457,\n \"exact_match_stderr,none\"\ : 0.02869787397186069\n },\n \"leaderboard_math_precalculus_hard\"\ : {\n \"alias\": \" - leaderboard_math_precalculus_hard\",\n \ \ \"exact_match,none\": 0.007407407407407408,\n \"exact_match_stderr,none\"\ : 0.007407407407407402\n },\n \"leaderboard_mmlu_pro\": {\n \ \ \"alias\": \" - leaderboard_mmlu_pro\",\n \"acc,none\": 0.31299867021276595,\n\ \ \"acc_stderr,none\": 0.004227651421740313\n },\n \"leaderboard_musr\"\ : {\n \" \": \" \",\n \"alias\": \" - leaderboard_musr\"\n\ \ },\n \"leaderboard_musr_murder_mysteries\": {\n \"alias\"\ : \" - leaderboard_musr_murder_mysteries\",\n \"acc_norm,none\": 0.58,\n\ \ \"acc_norm_stderr,none\": 0.03127799950463661\n },\n \ \ \"leaderboard_musr_object_placements\": {\n \"alias\": \" - leaderboard_musr_object_placements\"\ ,\n \"acc_norm,none\": 0.3671875,\n \"acc_norm_stderr,none\"\ : 0.030186403889489913\n },\n \"leaderboard_musr_team_allocation\"\ : {\n \"alias\": \" - leaderboard_musr_team_allocation\",\n \ \ \"acc_norm,none\": 0.372,\n \"acc_norm_stderr,none\": 0.03063032594455831\n\ \ },\n \"toxigen\": {\n \"alias\": \"toxigen\",\n \ \ \"acc,none\": 0.6,\n \"acc_stderr,none\": 0.015987215339548645,\n\ \ \"acc_norm,none\": 0.5702127659574469,\n \"acc_norm_stderr,none\"\ : 0.016155203301509467\n },\n \"wmdp\": {\n \"acc,none\"\ : 0.531897491821156,\n \"acc_stderr,none\": 0.008068859365825751,\n \ \ \"alias\": \"wmdp\"\n },\n \"wmdp_bio\": {\n \ \ \"alias\": \" - wmdp_bio\",\n \"acc,none\": 0.6684996072270227,\n\ \ \"acc_stderr,none\": 0.013199243664138346\n },\n \"wmdp_chem\"\ : {\n \"alias\": \" - wmdp_chem\",\n \"acc,none\": 0.5171568627450981,\n\ \ \"acc_stderr,none\": 0.024769484776043047\n },\n \"wmdp_cyber\"\ : {\n \"alias\": \" - wmdp_cyber\",\n \"acc,none\": 0.447408152994464,\n\ \ \"acc_stderr,none\": 0.011157440033040955\n },\n \"xstest\"\ : {\n \"alias\": \"xstest\",\n \"acc,none\": 0.4666666666666667,\n\ \ \"acc_stderr,none\": 0.023543963941101545,\n \"acc_norm,none\"\ : 0.4444444444444444,\n \"acc_norm_stderr,none\": 0.023450349399618212\n\ \ }\n },\n \"BeaverTailsEval\": {\n \"alias\": \"BeaverTailsEval\"\ ,\n \"acc,none\": 0.8814285714285715,\n \"acc_stderr,none\": 0.01222770634375575,\n\ \ \"acc_norm,none\": 0.13,\n \"acc_norm_stderr,none\": 0.012720164114794184\n\ \ },\n \"CDNA\": {\n \"alias\": \"CDNA\",\n \"acc,none\": 0.9534115920763023,\n\ \ \"acc_stderr,none\": 0.004037345306870679,\n \"acc_norm,none\":\ \ 0.01173881144534116,\n \"acc_norm_stderr,none\": 0.0020633109360408777\n\ \ },\n \"DTToxicity\": {\n \"alias\": \"DTToxicity\",\n \"acc,none\"\ : 0.4578464106844741,\n \"acc_stderr,none\": 0.010180481445723958,\n \ \ \"acc_norm,none\": 0.5029215358931552,\n \"acc_norm_stderr,none\": 0.010216680957241337\n\ \ },\n \"JailbreakHub\": {\n \"alias\": \"JailbreakHub\",\n \ \ \"acc,none\": 0.15792602377807133,\n \"acc_stderr,none\": 0.002963830609421817,\n\ \ \"acc_norm,none\": 0.09478203434610304,\n \"acc_norm_stderr,none\"\ : 0.002380625059276766\n },\n \"SGXSTest\": {\n \"alias\": \"SGXSTest\"\ ,\n \"acc,none\": 0.51,\n \"acc_stderr,none\": 0.035436970729343674,\n\ \ \"acc_norm,none\": 0.5,\n \"acc_norm_stderr,none\": 0.0354440602504168\n\ \ },\n \"SaladBench\": {\n \"alias\": \"SaladBench\",\n \"acc,none\"\ : 0.49505208333333334,\n \"acc_stderr,none\": 0.008069370988058294,\n \ \ \"acc_norm,none\": 0.49505208333333334,\n \"acc_norm_stderr,none\"\ : 0.008069370988058294\n },\n \"StrongREJECT\": {\n \"alias\": \"StrongREJECT\"\ ,\n \"acc,none\": 0.9424920127795527,\n \"acc_stderr,none\": 0.013180310031262344,\n\ \ \"acc_norm,none\": 0.19808306709265175,\n \"acc_norm_stderr,none\"\ : 0.022563739178544238\n },\n \"WildGuardTest\": {\n \"alias\": \"\ WildGuardTest\",\n \"acc,none\": 0.5710144927536231,\n \"acc_stderr,none\"\ : 0.011919999713357268,\n \"acc_norm,none\": 0.5640579710144927,\n \ \ \"acc_norm_stderr,none\": 0.011942839896636795\n },\n \"bbq\": {\n \ \ \"acc_norm,none\": 0.9325377829446762,\n \"acc_norm_stderr,none\": 0.0010218971033224293,\n\ \ \"acc,none\": 0.9324864938795049,\n \"acc_stderr,none\": 0.0010226628876829763,\n\ \ \"alias\": \"bbq\"\n },\n \"bbq_age\": {\n \"alias\": \" -\ \ bbq_age\",\n \"acc,none\": 0.8301630434782609,\n \"acc_stderr,none\"\ : 0.006190602280652597,\n \"acc_norm,none\": 0.8301630434782609,\n \ \ \"acc_norm_stderr,none\": 0.006190602280652598\n },\n \"bbq_disabilitystatus\"\ : {\n \"alias\": \" - bbq_disabilitystatus\",\n \"acc,none\": 0.9042416452442159,\n\ \ \"acc_stderr,none\": 0.007462174379038646,\n \"acc_norm,none\":\ \ 0.9048843187660668,\n \"acc_norm_stderr,none\": 0.007439733789161572\n\ \ },\n \"bbq_genderidentity\": {\n \"alias\": \" - bbq_genderidentity\"\ ,\n \"acc,none\": 0.9442877291960508,\n \"acc_stderr,none\": 0.00304577473375189,\n\ \ \"acc_norm,none\": 0.942524682651622,\n \"acc_norm_stderr,none\"\ : 0.003090702699187024\n },\n \"bbq_nationality\": {\n \"alias\": \"\ \ - bbq_nationality\",\n \"acc,none\": 0.9168831168831169,\n \"acc_stderr,none\"\ : 0.00497504218811962,\n \"acc_norm,none\": 0.9181818181818182,\n \ \ \"acc_norm_stderr,none\": 0.0049395161759693985\n },\n \"bbq_physicalappearance\"\ : {\n \"alias\": \" - bbq_physicalappearance\",\n \"acc,none\": 0.8299492385786802,\n\ \ \"acc_stderr,none\": 0.009466180402043615,\n \"acc_norm,none\":\ \ 0.8280456852791879,\n \"acc_norm_stderr,none\": 0.009508092739152258\n\ \ },\n \"bbq_raceethnicity\": {\n \"alias\": \" - bbq_raceethnicity\"\ ,\n \"acc,none\": 0.9156976744186046,\n \"acc_stderr,none\": 0.0033499103029905916,\n\ \ \"acc_norm,none\": 0.915406976744186,\n \"acc_norm_stderr,none\"\ : 0.0033551483498529358\n },\n \"bbq_racexgender\": {\n \"alias\":\ \ \" - bbq_racexgender\",\n \"acc,none\": 0.9584586466165413,\n \"\ acc_stderr,none\": 0.0015795165224040926,\n \"acc_norm,none\": 0.9586466165413534,\n\ \ \"acc_norm_stderr,none\": 0.0015760934304819407\n },\n \"bbq_racexses\"\ : {\n \"alias\": \" - bbq_racexses\",\n \"acc,none\": 0.9760752688172043,\n\ \ \"acc_stderr,none\": 0.0014466136180125736,\n \"acc_norm,none\"\ : 0.9779569892473118,\n \"acc_norm_stderr,none\": 0.0013898972475625852\n\ \ },\n \"bbq_religion\": {\n \"alias\": \" - bbq_religion\",\n \ \ \"acc,none\": 0.835,\n \"acc_stderr,none\": 0.010719526896310953,\n\ \ \"acc_norm,none\": 0.8341666666666666,\n \"acc_norm_stderr,none\"\ : 0.01074119842135935\n },\n \"bbq_ses\": {\n \"alias\": \" - bbq_ses\"\ ,\n \"acc,none\": 0.921037296037296,\n \"acc_stderr,none\": 0.003255311967010792,\n\ \ \"acc_norm,none\": 0.9195804195804196,\n \"acc_norm_stderr,none\"\ : 0.0032826060053440993\n },\n \"bbq_sexualorientation\": {\n \"alias\"\ : \" - bbq_sexualorientation\",\n \"acc,none\": 0.9016203703703703,\n \ \ \"acc_stderr,none\": 0.010138157908353061,\n \"acc_norm,none\": 0.9016203703703703,\n\ \ \"acc_norm_stderr,none\": 0.01013815790835306\n },\n \"leaderboard\"\ : {\n \" \": \" \",\n \"alias\": \"leaderboard\"\n },\n \"leaderboard_bbh\"\ : {\n \" \": \" \",\n \"alias\": \" - leaderboard_bbh\"\n },\n\ \ \"leaderboard_bbh_boolean_expressions\": {\n \"alias\": \" - leaderboard_bbh_boolean_expressions\"\ ,\n \"acc_norm,none\": 0.8,\n \"acc_norm_stderr,none\": 0.025348970020979078\n\ \ },\n \"leaderboard_bbh_causal_judgement\": {\n \"alias\": \" - leaderboard_bbh_causal_judgement\"\ ,\n \"acc_norm,none\": 0.6096256684491979,\n \"acc_norm_stderr,none\"\ : 0.03576973947986412\n },\n \"leaderboard_bbh_date_understanding\": {\n \ \ \"alias\": \" - leaderboard_bbh_date_understanding\",\n \"acc_norm,none\"\ : 0.472,\n \"acc_norm_stderr,none\": 0.031636489531544396\n },\n \"\ leaderboard_bbh_disambiguation_qa\": {\n \"alias\": \" - leaderboard_bbh_disambiguation_qa\"\ ,\n \"acc_norm,none\": 0.668,\n \"acc_norm_stderr,none\": 0.02984403904746591\n\ \ },\n \"leaderboard_bbh_formal_fallacies\": {\n \"alias\": \" - leaderboard_bbh_formal_fallacies\"\ ,\n \"acc_norm,none\": 0.612,\n \"acc_norm_stderr,none\": 0.03088103874899391\n\ \ },\n \"leaderboard_bbh_geometric_shapes\": {\n \"alias\": \" - leaderboard_bbh_geometric_shapes\"\ ,\n \"acc_norm,none\": 0.376,\n \"acc_norm_stderr,none\": 0.030696336267394583\n\ \ },\n \"leaderboard_bbh_hyperbaton\": {\n \"alias\": \" - leaderboard_bbh_hyperbaton\"\ ,\n \"acc_norm,none\": 0.708,\n \"acc_norm_stderr,none\": 0.02881432040220564\n\ \ },\n \"leaderboard_bbh_logical_deduction_five_objects\": {\n \"alias\"\ : \" - leaderboard_bbh_logical_deduction_five_objects\",\n \"acc_norm,none\"\ : 0.472,\n \"acc_norm_stderr,none\": 0.031636489531544396\n },\n \"\ leaderboard_bbh_logical_deduction_seven_objects\": {\n \"alias\": \" - leaderboard_bbh_logical_deduction_seven_objects\"\ ,\n \"acc_norm,none\": 0.448,\n \"acc_norm_stderr,none\": 0.03151438761115355\n\ \ },\n \"leaderboard_bbh_logical_deduction_three_objects\": {\n \"\ alias\": \" - leaderboard_bbh_logical_deduction_three_objects\",\n \"acc_norm,none\"\ : 0.724,\n \"acc_norm_stderr,none\": 0.028328537274211352\n },\n \"\ leaderboard_bbh_movie_recommendation\": {\n \"alias\": \" - leaderboard_bbh_movie_recommendation\"\ ,\n \"acc_norm,none\": 0.668,\n \"acc_norm_stderr,none\": 0.02984403904746591\n\ \ },\n \"leaderboard_bbh_navigate\": {\n \"alias\": \" - leaderboard_bbh_navigate\"\ ,\n \"acc_norm,none\": 0.58,\n \"acc_norm_stderr,none\": 0.03127799950463661\n\ \ },\n \"leaderboard_bbh_object_counting\": {\n \"alias\": \" - leaderboard_bbh_object_counting\"\ ,\n \"acc_norm,none\": 0.34,\n \"acc_norm_stderr,none\": 0.030020073605457907\n\ \ },\n \"leaderboard_bbh_penguins_in_a_table\": {\n \"alias\": \" \ \ - leaderboard_bbh_penguins_in_a_table\",\n \"acc_norm,none\": 0.4657534246575342,\n\ \ \"acc_norm_stderr,none\": 0.04142522736934775\n },\n \"leaderboard_bbh_reasoning_about_colored_objects\"\ : {\n \"alias\": \" - leaderboard_bbh_reasoning_about_colored_objects\"\ ,\n \"acc_norm,none\": 0.56,\n \"acc_norm_stderr,none\": 0.03145724452223563\n\ \ },\n \"leaderboard_bbh_ruin_names\": {\n \"alias\": \" - leaderboard_bbh_ruin_names\"\ ,\n \"acc_norm,none\": 0.632,\n \"acc_norm_stderr,none\": 0.030562070620993163\n\ \ },\n \"leaderboard_bbh_salient_translation_error_detection\": {\n \ \ \"alias\": \" - leaderboard_bbh_salient_translation_error_detection\",\n \ \ \"acc_norm,none\": 0.476,\n \"acc_norm_stderr,none\": 0.03164968895968782\n\ \ },\n \"leaderboard_bbh_snarks\": {\n \"alias\": \" - leaderboard_bbh_snarks\"\ ,\n \"acc_norm,none\": 0.7078651685393258,\n \"acc_norm_stderr,none\"\ : 0.03418064934010763\n },\n \"leaderboard_bbh_sports_understanding\": {\n\ \ \"alias\": \" - leaderboard_bbh_sports_understanding\",\n \"acc_norm,none\"\ : 0.78,\n \"acc_norm_stderr,none\": 0.026251792824605824\n },\n \"\ leaderboard_bbh_temporal_sequences\": {\n \"alias\": \" - leaderboard_bbh_temporal_sequences\"\ ,\n \"acc_norm,none\": 0.256,\n \"acc_norm_stderr,none\": 0.02765710871820491\n\ \ },\n \"leaderboard_bbh_tracking_shuffled_objects_five_objects\": {\n \ \ \"alias\": \" - leaderboard_bbh_tracking_shuffled_objects_five_objects\"\ ,\n \"acc_norm,none\": 0.192,\n \"acc_norm_stderr,none\": 0.024960691989172015\n\ \ },\n \"leaderboard_bbh_tracking_shuffled_objects_seven_objects\": {\n \ \ \"alias\": \" - leaderboard_bbh_tracking_shuffled_objects_seven_objects\"\ ,\n \"acc_norm,none\": 0.14,\n \"acc_norm_stderr,none\": 0.021989409645240272\n\ \ },\n \"leaderboard_bbh_tracking_shuffled_objects_three_objects\": {\n \ \ \"alias\": \" - leaderboard_bbh_tracking_shuffled_objects_three_objects\"\ ,\n \"acc_norm,none\": 0.3,\n \"acc_norm_stderr,none\": 0.029040893477575845\n\ \ },\n \"leaderboard_bbh_web_of_lies\": {\n \"alias\": \" - leaderboard_bbh_web_of_lies\"\ ,\n \"acc_norm,none\": 0.48,\n \"acc_norm_stderr,none\": 0.03166085340849519\n\ \ },\n \"leaderboard_gpqa\": {\n \" \": \" \",\n \"alias\":\ \ \" - leaderboard_gpqa\"\n },\n \"leaderboard_gpqa_diamond\": {\n \ \ \"alias\": \" - leaderboard_gpqa_diamond\",\n \"acc_norm,none\": 0.29292929292929293,\n\ \ \"acc_norm_stderr,none\": 0.032424979581788166\n },\n \"leaderboard_gpqa_extended\"\ : {\n \"alias\": \" - leaderboard_gpqa_extended\",\n \"acc_norm,none\"\ : 0.30952380952380953,\n \"acc_norm_stderr,none\": 0.019802641880170335\n\ \ },\n \"leaderboard_gpqa_main\": {\n \"alias\": \" - leaderboard_gpqa_main\"\ ,\n \"acc_norm,none\": 0.29017857142857145,\n \"acc_norm_stderr,none\"\ : 0.02146611544057122\n },\n \"leaderboard_ifeval\": {\n \"alias\"\ : \" - leaderboard_ifeval\",\n \"prompt_level_strict_acc,none\": 0.3826247689463956,\n\ \ \"prompt_level_strict_acc_stderr,none\": 0.020915307841436647,\n \ \ \"inst_level_strict_acc,none\": 0.5179856115107914,\n \"inst_level_strict_acc_stderr,none\"\ : \"N/A\",\n \"prompt_level_loose_acc,none\": 0.4602587800369686,\n \ \ \"prompt_level_loose_acc_stderr,none\": 0.02144850143413504,\n \"inst_level_loose_acc,none\"\ : 0.5935251798561151,\n \"inst_level_loose_acc_stderr,none\": \"N/A\"\n \ \ },\n \"leaderboard_math_hard\": {\n \" \": \" \",\n \"alias\"\ : \" - leaderboard_math_hard\"\n },\n \"leaderboard_math_algebra_hard\": {\n\ \ \"alias\": \" - leaderboard_math_algebra_hard\",\n \"exact_match,none\"\ : 0.11400651465798045,\n \"exact_match_stderr,none\": 0.018168513062629208\n\ \ },\n \"leaderboard_math_counting_and_prob_hard\": {\n \"alias\":\ \ \" - leaderboard_math_counting_and_prob_hard\",\n \"exact_match,none\"\ : 0.04878048780487805,\n \"exact_match_stderr,none\": 0.019502196558588064\n\ \ },\n \"leaderboard_math_geometry_hard\": {\n \"alias\": \" - leaderboard_math_geometry_hard\"\ ,\n \"exact_match,none\": 0.022727272727272728,\n \"exact_match_stderr,none\"\ : 0.01302104690906372\n },\n \"leaderboard_math_intermediate_algebra_hard\"\ : {\n \"alias\": \" - leaderboard_math_intermediate_algebra_hard\",\n \ \ \"exact_match,none\": 0.02857142857142857,\n \"exact_match_stderr,none\"\ : 0.009973998820736029\n },\n \"leaderboard_math_num_theory_hard\": {\n \ \ \"alias\": \" - leaderboard_math_num_theory_hard\",\n \"exact_match,none\"\ : 0.07142857142857142,\n \"exact_match_stderr,none\": 0.020820824576076355\n\ \ },\n \"leaderboard_math_prealgebra_hard\": {\n \"alias\": \" - leaderboard_math_prealgebra_hard\"\ ,\n \"exact_match,none\": 0.19689119170984457,\n \"exact_match_stderr,none\"\ : 0.02869787397186069\n },\n \"leaderboard_math_precalculus_hard\": {\n \ \ \"alias\": \" - leaderboard_math_precalculus_hard\",\n \"exact_match,none\"\ : 0.007407407407407408,\n \"exact_match_stderr,none\": 0.007407407407407402\n\ \ },\n \"leaderboard_mmlu_pro\": {\n \"alias\": \" - leaderboard_mmlu_pro\"\ ,\n \"acc,none\": 0.31299867021276595,\n \"acc_stderr,none\": 0.004227651421740313\n\ \ },\n \"leaderboard_musr\": {\n \" \": \" \",\n \"alias\":\ \ \" - leaderboard_musr\"\n },\n \"leaderboard_musr_murder_mysteries\": {\n\ \ \"alias\": \" - leaderboard_musr_murder_mysteries\",\n \"acc_norm,none\"\ : 0.58,\n \"acc_norm_stderr,none\": 0.03127799950463661\n },\n \"leaderboard_musr_object_placements\"\ : {\n \"alias\": \" - leaderboard_musr_object_placements\",\n \"\ acc_norm,none\": 0.3671875,\n \"acc_norm_stderr,none\": 0.030186403889489913\n\ \ },\n \"leaderboard_musr_team_allocation\": {\n \"alias\": \" - leaderboard_musr_team_allocation\"\ ,\n \"acc_norm,none\": 0.372,\n \"acc_norm_stderr,none\": 0.03063032594455831\n\ \ },\n \"toxigen\": {\n \"alias\": \"toxigen\",\n \"acc,none\"\ : 0.6,\n \"acc_stderr,none\": 0.015987215339548645,\n \"acc_norm,none\"\ : 0.5702127659574469,\n \"acc_norm_stderr,none\": 0.016155203301509467\n\ \ },\n \"wmdp\": {\n \"acc,none\": 0.531897491821156,\n \"acc_stderr,none\"\ : 0.008068859365825751,\n \"alias\": \"wmdp\"\n },\n \"wmdp_bio\":\ \ {\n \"alias\": \" - wmdp_bio\",\n \"acc,none\": 0.6684996072270227,\n\ \ \"acc_stderr,none\": 0.013199243664138346\n },\n \"wmdp_chem\": {\n\ \ \"alias\": \" - wmdp_chem\",\n \"acc,none\": 0.5171568627450981,\n\ \ \"acc_stderr,none\": 0.024769484776043047\n },\n \"wmdp_cyber\":\ \ {\n \"alias\": \" - wmdp_cyber\",\n \"acc,none\": 0.447408152994464,\n\ \ \"acc_stderr,none\": 0.011157440033040955\n },\n \"xstest\": {\n\ \ \"alias\": \"xstest\",\n \"acc,none\": 0.4666666666666667,\n \ \ \"acc_stderr,none\": 0.023543963941101545,\n \"acc_norm,none\": 0.4444444444444444,\n\ \ \"acc_norm_stderr,none\": 0.023450349399618212\n }\n}\n```" repo_url: https://huggingface.co/zhengr/MixTAO-7Bx2-MoE-v8.1 leaderboard_url: '' point_of_contact: '' configs: - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__BeaverTailsEval data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_BeaverTailsEval_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_BeaverTailsEval_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__CDNA data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_CDNA_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_CDNA_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__DTToxicity data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_DTToxicity_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_DTToxicity_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__JailbreakHub data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_JailbreakHub_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_JailbreakHub_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__SGXSTest data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_SGXSTest_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_SGXSTest_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__SaladBench data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_SaladBench_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_SaladBench_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__StrongREJECT data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_StrongREJECT_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_StrongREJECT_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__WildGuardTest data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_WildGuardTest_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_WildGuardTest_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__bbq_age data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_bbq_age_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_bbq_age_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__bbq_disabilitystatus data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_bbq_disabilitystatus_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_bbq_disabilitystatus_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__bbq_genderidentity data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_bbq_genderidentity_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_bbq_genderidentity_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__bbq_nationality data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_bbq_nationality_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_bbq_nationality_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__bbq_physicalappearance data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_bbq_physicalappearance_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_bbq_physicalappearance_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__bbq_raceethnicity data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_bbq_raceethnicity_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_bbq_raceethnicity_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__bbq_racexgender data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_bbq_racexgender_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_bbq_racexgender_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__bbq_racexses data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_bbq_racexses_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_bbq_racexses_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__bbq_religion data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_bbq_religion_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_bbq_religion_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__bbq_ses data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_bbq_ses_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_bbq_ses_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__bbq_sexualorientation data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_bbq_sexualorientation_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_bbq_sexualorientation_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_boolean_expressions data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_boolean_expressions_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_boolean_expressions_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_causal_judgement data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_causal_judgement_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_causal_judgement_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_date_understanding data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_date_understanding_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_date_understanding_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_disambiguation_qa data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_disambiguation_qa_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_disambiguation_qa_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_formal_fallacies data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_formal_fallacies_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_formal_fallacies_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_geometric_shapes data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_geometric_shapes_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_geometric_shapes_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_hyperbaton data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_hyperbaton_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_hyperbaton_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_logical_deduction_five_objects data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_logical_deduction_five_objects_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_logical_deduction_five_objects_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_logical_deduction_seven_objects data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_logical_deduction_seven_objects_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_logical_deduction_seven_objects_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_logical_deduction_three_objects data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_logical_deduction_three_objects_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_logical_deduction_three_objects_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_movie_recommendation data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_movie_recommendation_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_movie_recommendation_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_navigate data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_navigate_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_navigate_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_object_counting data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_object_counting_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_object_counting_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_penguins_in_a_table data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_penguins_in_a_table_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_penguins_in_a_table_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_reasoning_about_colored_objects data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_reasoning_about_colored_objects_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_reasoning_about_colored_objects_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_ruin_names data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_ruin_names_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_ruin_names_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_salient_translation_error_detection data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_salient_translation_error_detection_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_salient_translation_error_detection_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_snarks data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_snarks_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_snarks_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_sports_understanding data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_sports_understanding_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_sports_understanding_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_temporal_sequences data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_temporal_sequences_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_temporal_sequences_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_tracking_shuffled_objects_five_objects data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_tracking_shuffled_objects_five_objects_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_tracking_shuffled_objects_five_objects_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_tracking_shuffled_objects_seven_objects data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_tracking_shuffled_objects_seven_objects_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_tracking_shuffled_objects_seven_objects_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_tracking_shuffled_objects_three_objects data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_tracking_shuffled_objects_three_objects_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_tracking_shuffled_objects_three_objects_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_bbh_web_of_lies data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_bbh_web_of_lies_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_bbh_web_of_lies_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_gpqa_diamond data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_gpqa_diamond_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_gpqa_diamond_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_gpqa_extended data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_gpqa_extended_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_gpqa_extended_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_gpqa_main data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_gpqa_main_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_gpqa_main_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_ifeval data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_ifeval_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_ifeval_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_math_algebra_hard data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_math_algebra_hard_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_math_algebra_hard_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_math_counting_and_prob_hard data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_math_counting_and_prob_hard_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_math_counting_and_prob_hard_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_math_geometry_hard data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_math_geometry_hard_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_math_geometry_hard_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_math_intermediate_algebra_hard data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_math_intermediate_algebra_hard_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_math_intermediate_algebra_hard_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_math_num_theory_hard data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_math_num_theory_hard_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_math_num_theory_hard_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_math_prealgebra_hard data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_math_prealgebra_hard_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_math_prealgebra_hard_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_math_precalculus_hard data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_math_precalculus_hard_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_math_precalculus_hard_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_mmlu_pro data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_mmlu_pro_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_mmlu_pro_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_musr_murder_mysteries data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_musr_murder_mysteries_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_musr_murder_mysteries_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_musr_object_placements data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_musr_object_placements_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_musr_object_placements_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__leaderboard_musr_team_allocation data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_leaderboard_musr_team_allocation_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_leaderboard_musr_team_allocation_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__toxigen data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_toxigen_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_toxigen_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__wmdp_bio data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_wmdp_bio_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_wmdp_bio_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__wmdp_chem data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_wmdp_chem_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_wmdp_chem_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__wmdp_cyber data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_wmdp_cyber_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_wmdp_cyber_2024-12-05T00-28-46.710222.jsonl' - config_name: zhengr__MixTAO-7Bx2-MoE-v8.1__xstest data_files: - split: 2024_12_05T00_28_46.710222 path: - '**/samples_xstest_2024-12-05T00-28-46.710222.jsonl' - split: latest path: - '**/samples_xstest_2024-12-05T00-28-46.710222.jsonl' --- # Dataset Card for Evaluation run of zhengr/MixTAO-7Bx2-MoE-v8.1  Dataset automatically created during the evaluation run of model [zhengr/MixTAO-7Bx2-MoE-v8.1](https://huggingface.co/zhengr/MixTAO-7Bx2-MoE-v8.1) The dataset is composed of 62 configuration(s), each one corresponding to one of the evaluated task. The dataset has been created from 2 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest results. An additional configuration "results" store all the aggregated results of the run. To load the details from a run, you can for instance do the following: ```python from datasets import load_dataset data = load_dataset( "nyu-dice-lab/lm-eval-results-zhengr-MixTAO-7Bx2-MoE-v8.1-private", name="zhengr__MixTAO-7Bx2-MoE-v8.1__BeaverTailsEval", split="latest" ) ``` ## Latest results These are the [latest results from run 2024-12-05T00-28-46.710222](https://huggingface.co/datasets/nyu-dice-lab/lm-eval-results-zhengr-MixTAO-7Bx2-MoE-v8.1-private/blob/main/zhengr/MixTAO-7Bx2-MoE-v8.1/results_2024-12-05T00-28-46.710222.json) (note that there might be results for other tasks in the repos if successive evals didn't cover the same tasks. You find each in the results and the "latest" split for each eval): ```python { "all": { "BeaverTailsEval": { "alias": "BeaverTailsEval", "acc,none": 0.8814285714285715, "acc_stderr,none": 0.01222770634375575, "acc_norm,none": 0.13, "acc_norm_stderr,none": 0.012720164114794184 }, "CDNA": { "alias": "CDNA", "acc,none": 0.9534115920763023, "acc_stderr,none": 0.004037345306870679, "acc_norm,none": 0.01173881144534116, "acc_norm_stderr,none": 0.0020633109360408777 }, "DTToxicity": { "alias": "DTToxicity", "acc,none": 0.4578464106844741, "acc_stderr,none": 0.010180481445723958, "acc_norm,none": 0.5029215358931552, "acc_norm_stderr,none": 0.010216680957241337 }, "JailbreakHub": { "alias": "JailbreakHub", "acc,none": 0.15792602377807133, "acc_stderr,none": 0.002963830609421817, "acc_norm,none": 0.09478203434610304, "acc_norm_stderr,none": 0.002380625059276766 }, "SGXSTest": { "alias": "SGXSTest", "acc,none": 0.51, "acc_stderr,none": 0.035436970729343674, "acc_norm,none": 0.5, "acc_norm_stderr,none": 0.0354440602504168 }, "SaladBench": { "alias": "SaladBench", "acc,none": 0.49505208333333334, "acc_stderr,none": 0.008069370988058294, "acc_norm,none": 0.49505208333333334, "acc_norm_stderr,none": 0.008069370988058294 }, "StrongREJECT": { "alias": "StrongREJECT", "acc,none": 0.9424920127795527, "acc_stderr,none": 0.013180310031262344, "acc_norm,none": 0.19808306709265175, "acc_norm_stderr,none": 0.022563739178544238 }, "WildGuardTest": { "alias": "WildGuardTest", "acc,none": 0.5710144927536231, "acc_stderr,none": 0.011919999713357268, "acc_norm,none": 0.5640579710144927, "acc_norm_stderr,none": 0.011942839896636795 }, "bbq": { "acc_norm,none": 0.9325377829446762, "acc_norm_stderr,none": 0.0010218971033224293, "acc,none": 0.9324864938795049, "acc_stderr,none": 0.0010226628876829763, "alias": "bbq" }, "bbq_age": { "alias": " - bbq_age", "acc,none": 0.8301630434782609, "acc_stderr,none": 0.006190602280652597, "acc_norm,none": 0.8301630434782609, "acc_norm_stderr,none": 0.006190602280652598 }, "bbq_disabilitystatus": { "alias": " - bbq_disabilitystatus", "acc,none": 0.9042416452442159, "acc_stderr,none": 0.007462174379038646, "acc_norm,none": 0.9048843187660668, "acc_norm_stderr,none": 0.007439733789161572 }, "bbq_genderidentity": { "alias": " - bbq_genderidentity", "acc,none": 0.9442877291960508, "acc_stderr,none": 0.00304577473375189, "acc_norm,none": 0.942524682651622, "acc_norm_stderr,none": 0.003090702699187024 }, "bbq_nationality": { "alias": " - bbq_nationality", "acc,none": 0.9168831168831169, "acc_stderr,none": 0.00497504218811962, "acc_norm,none": 0.9181818181818182, "acc_norm_stderr,none": 0.0049395161759693985 }, "bbq_physicalappearance": { "alias": " - bbq_physicalappearance", "acc,none": 0.8299492385786802, "acc_stderr,none": 0.009466180402043615, "acc_norm,none": 0.8280456852791879, "acc_norm_stderr,none": 0.009508092739152258 }, "bbq_raceethnicity": { "alias": " - bbq_raceethnicity", "acc,none": 0.9156976744186046, "acc_stderr,none": 0.0033499103029905916, "acc_norm,none": 0.915406976744186, "acc_norm_stderr,none": 0.0033551483498529358 }, "bbq_racexgender": { "alias": " - bbq_racexgender", "acc,none": 0.9584586466165413, "acc_stderr,none": 0.0015795165224040926, "acc_norm,none": 0.9586466165413534, "acc_norm_stderr,none": 0.0015760934304819407 }, "bbq_racexses": { "alias": " - bbq_racexses", "acc,none": 0.9760752688172043, "acc_stderr,none": 0.0014466136180125736, "acc_norm,none": 0.9779569892473118, "acc_norm_stderr,none": 0.0013898972475625852 }, "bbq_religion": { "alias": " - bbq_religion", "acc,none": 0.835, "acc_stderr,none": 0.010719526896310953, "acc_norm,none": 0.8341666666666666, "acc_norm_stderr,none": 0.01074119842135935 }, "bbq_ses": { "alias": " - bbq_ses", "acc,none": 0.921037296037296, "acc_stderr,none": 0.003255311967010792, "acc_norm,none": 0.9195804195804196, "acc_norm_stderr,none": 0.0032826060053440993 }, "bbq_sexualorientation": { "alias": " - bbq_sexualorientation", "acc,none": 0.9016203703703703, "acc_stderr,none": 0.010138157908353061, "acc_norm,none": 0.9016203703703703, "acc_norm_stderr,none": 0.01013815790835306 }, "leaderboard": { " ": " ", "alias": "leaderboard" }, "leaderboard_bbh": { " ": " ", "alias": " - leaderboard_bbh" }, "leaderboard_bbh_boolean_expressions": { "alias": " - leaderboard_bbh_boolean_expressions", "acc_norm,none": 0.8, "acc_norm_stderr,none": 0.025348970020979078 }, "leaderboard_bbh_causal_judgement": { "alias": " - leaderboard_bbh_causal_judgement", "acc_norm,none": 0.6096256684491979, "acc_norm_stderr,none": 0.03576973947986412 }, "leaderboard_bbh_date_understanding": { "alias": " - leaderboard_bbh_date_understanding", "acc_norm,none": 0.472, "acc_norm_stderr,none": 0.031636489531544396 }, "leaderboard_bbh_disambiguation_qa": { "alias": " - leaderboard_bbh_disambiguation_qa", "acc_norm,none": 0.668, "acc_norm_stderr,none": 0.02984403904746591 }, "leaderboard_bbh_formal_fallacies": { "alias": " - leaderboard_bbh_formal_fallacies", "acc_norm,none": 0.612, "acc_norm_stderr,none": 0.03088103874899391 }, "leaderboard_bbh_geometric_shapes": { "alias": " - leaderboard_bbh_geometric_shapes", "acc_norm,none": 0.376, "acc_norm_stderr,none": 0.030696336267394583 }, "leaderboard_bbh_hyperbaton": { "alias": " - leaderboard_bbh_hyperbaton", "acc_norm,none": 0.708, "acc_norm_stderr,none": 0.02881432040220564 }, "leaderboard_bbh_logical_deduction_five_objects": { "alias": " - leaderboard_bbh_logical_deduction_five_objects", "acc_norm,none": 0.472, "acc_norm_stderr,none": 0.031636489531544396 }, "leaderboard_bbh_logical_deduction_seven_objects": { "alias": " - leaderboard_bbh_logical_deduction_seven_objects", "acc_norm,none": 0.448, "acc_norm_stderr,none": 0.03151438761115355 }, "leaderboard_bbh_logical_deduction_three_objects": { "alias": " - leaderboard_bbh_logical_deduction_three_objects", "acc_norm,none": 0.724, "acc_norm_stderr,none": 0.028328537274211352 }, "leaderboard_bbh_movie_recommendation": { "alias": " - leaderboard_bbh_movie_recommendation", "acc_norm,none": 0.668, "acc_norm_stderr,none": 0.02984403904746591 }, "leaderboard_bbh_navigate": { "alias": " - leaderboard_bbh_navigate", "acc_norm,none": 0.58, "acc_norm_stderr,none": 0.03127799950463661 }, "leaderboard_bbh_object_counting": { "alias": " - leaderboard_bbh_object_counting", "acc_norm,none": 0.34, "acc_norm_stderr,none": 0.030020073605457907 }, "leaderboard_bbh_penguins_in_a_table": { "alias": " - leaderboard_bbh_penguins_in_a_table", "acc_norm,none": 0.4657534246575342, "acc_norm_stderr,none": 0.04142522736934775 }, "leaderboard_bbh_reasoning_about_colored_objects": { "alias": " - leaderboard_bbh_reasoning_about_colored_objects", "acc_norm,none": 0.56, "acc_norm_stderr,none": 0.03145724452223563 }, "leaderboard_bbh_ruin_names": { "alias": " - leaderboard_bbh_ruin_names", "acc_norm,none": 0.632, "acc_norm_stderr,none": 0.030562070620993163 }, "leaderboard_bbh_salient_translation_error_detection": { "alias": " - leaderboard_bbh_salient_translation_error_detection", "acc_norm,none": 0.476, "acc_norm_stderr,none": 0.03164968895968782 }, "leaderboard_bbh_snarks": { "alias": " - leaderboard_bbh_snarks", "acc_norm,none": 0.7078651685393258, "acc_norm_stderr,none": 0.03418064934010763 }, "leaderboard_bbh_sports_understanding": { "alias": " - leaderboard_bbh_sports_understanding", "acc_norm,none": 0.78, "acc_norm_stderr,none": 0.026251792824605824 }, "leaderboard_bbh_temporal_sequences": { "alias": " - leaderboard_bbh_temporal_sequences", "acc_norm,none": 0.256, "acc_norm_stderr,none": 0.02765710871820491 }, "leaderboard_bbh_tracking_shuffled_objects_five_objects": { "alias": " - leaderboard_bbh_tracking_shuffled_objects_five_objects", "acc_norm,none": 0.192, "acc_norm_stderr,none": 0.024960691989172015 }, "leaderboard_bbh_tracking_shuffled_objects_seven_objects": { "alias": " - leaderboard_bbh_tracking_shuffled_objects_seven_objects", "acc_norm,none": 0.14, "acc_norm_stderr,none": 0.021989409645240272 }, "leaderboard_bbh_tracking_shuffled_objects_three_objects": { "alias": " - leaderboard_bbh_tracking_shuffled_objects_three_objects", "acc_norm,none": 0.3, "acc_norm_stderr,none": 0.029040893477575845 }, "leaderboard_bbh_web_of_lies": { "alias": " - leaderboard_bbh_web_of_lies", "acc_norm,none": 0.48, "acc_norm_stderr,none": 0.03166085340849519 }, "leaderboard_gpqa": { " ": " ", "alias": " - leaderboard_gpqa" }, "leaderboard_gpqa_diamond": { "alias": " - leaderboard_gpqa_diamond", "acc_norm,none": 0.29292929292929293, "acc_norm_stderr,none": 0.032424979581788166 }, "leaderboard_gpqa_extended": { "alias": " - leaderboard_gpqa_extended", "acc_norm,none": 0.30952380952380953, "acc_norm_stderr,none": 0.019802641880170335 }, "leaderboard_gpqa_main": { "alias": " - leaderboard_gpqa_main", "acc_norm,none": 0.29017857142857145, "acc_norm_stderr,none": 0.02146611544057122 }, "leaderboard_ifeval": { "alias": " - leaderboard_ifeval", "prompt_level_strict_acc,none": 0.3826247689463956, "prompt_level_strict_acc_stderr,none": 0.020915307841436647, "inst_level_strict_acc,none": 0.5179856115107914, "inst_level_strict_acc_stderr,none": "N/A", "prompt_level_loose_acc,none": 0.4602587800369686, "prompt_level_loose_acc_stderr,none": 0.02144850143413504, "inst_level_loose_acc,none": 0.5935251798561151, "inst_level_loose_acc_stderr,none": "N/A" }, "leaderboard_math_hard": { " ": " ", "alias": " - leaderboard_math_hard" }, "leaderboard_math_algebra_hard": { "alias": " - leaderboard_math_algebra_hard", "exact_match,none": 0.11400651465798045, "exact_match_stderr,none": 0.018168513062629208 }, "leaderboard_math_counting_and_prob_hard": { "alias": " - leaderboard_math_counting_and_prob_hard", "exact_match,none": 0.04878048780487805, "exact_match_stderr,none": 0.019502196558588064 }, "leaderboard_math_geometry_hard": { "alias": " - leaderboard_math_geometry_hard", "exact_match,none": 0.022727272727272728, "exact_match_stderr,none": 0.01302104690906372 }, "leaderboard_math_intermediate_algebra_hard": { "alias": " - leaderboard_math_intermediate_algebra_hard", "exact_match,none": 0.02857142857142857, "exact_match_stderr,none": 0.009973998820736029 }, "leaderboard_math_num_theory_hard": { "alias": " - leaderboard_math_num_theory_hard", "exact_match,none": 0.07142857142857142, "exact_match_stderr,none": 0.020820824576076355 }, "leaderboard_math_prealgebra_hard": { "alias": " - leaderboard_math_prealgebra_hard", "exact_match,none": 0.19689119170984457, "exact_match_stderr,none": 0.02869787397186069 }, "leaderboard_math_precalculus_hard": { "alias": " - leaderboard_math_precalculus_hard", "exact_match,none": 0.007407407407407408, "exact_match_stderr,none": 0.007407407407407402 }, "leaderboard_mmlu_pro": { "alias": " - leaderboard_mmlu_pro", "acc,none": 0.31299867021276595, "acc_stderr,none": 0.004227651421740313 }, "leaderboard_musr": { " ": " ", "alias": " - leaderboard_musr" }, "leaderboard_musr_murder_mysteries": { "alias": " - leaderboard_musr_murder_mysteries", "acc_norm,none": 0.58, "acc_norm_stderr,none": 0.03127799950463661 }, "leaderboard_musr_object_placements": { "alias": " - leaderboard_musr_object_placements", "acc_norm,none": 0.3671875, "acc_norm_stderr,none": 0.030186403889489913 }, "leaderboard_musr_team_allocation": { "alias": " - leaderboard_musr_team_allocation", "acc_norm,none": 0.372, "acc_norm_stderr,none": 0.03063032594455831 }, "toxigen": { "alias": "toxigen", "acc,none": 0.6, "acc_stderr,none": 0.015987215339548645, "acc_norm,none": 0.5702127659574469, "acc_norm_stderr,none": 0.016155203301509467 }, "wmdp": { "acc,none": 0.531897491821156, "acc_stderr,none": 0.008068859365825751, "alias": "wmdp" }, "wmdp_bio": { "alias": " - wmdp_bio", "acc,none": 0.6684996072270227, "acc_stderr,none": 0.013199243664138346 }, "wmdp_chem": { "alias": " - wmdp_chem", "acc,none": 0.5171568627450981, "acc_stderr,none": 0.024769484776043047 }, "wmdp_cyber": { "alias": " - wmdp_cyber", "acc,none": 0.447408152994464, "acc_stderr,none": 0.011157440033040955 }, "xstest": { "alias": "xstest", "acc,none": 0.4666666666666667, "acc_stderr,none": 0.023543963941101545, "acc_norm,none": 0.4444444444444444, "acc_norm_stderr,none": 0.023450349399618212 } }, "BeaverTailsEval": { "alias": "BeaverTailsEval", "acc,none": 0.8814285714285715, "acc_stderr,none": 0.01222770634375575, "acc_norm,none": 0.13, "acc_norm_stderr,none": 0.012720164114794184 }, "CDNA": { "alias": "CDNA", "acc,none": 0.9534115920763023, "acc_stderr,none": 0.004037345306870679, "acc_norm,none": 0.01173881144534116, "acc_norm_stderr,none": 0.0020633109360408777 }, "DTToxicity": { "alias": "DTToxicity", "acc,none": 0.4578464106844741, "acc_stderr,none": 0.010180481445723958, "acc_norm,none": 0.5029215358931552, "acc_norm_stderr,none": 0.010216680957241337 }, "JailbreakHub": { "alias": "JailbreakHub", "acc,none": 0.15792602377807133, "acc_stderr,none": 0.002963830609421817, "acc_norm,none": 0.09478203434610304, "acc_norm_stderr,none": 0.002380625059276766 }, "SGXSTest": { "alias": "SGXSTest", "acc,none": 0.51, "acc_stderr,none": 0.035436970729343674, "acc_norm,none": 0.5, "acc_norm_stderr,none": 0.0354440602504168 }, "SaladBench": { "alias": "SaladBench", "acc,none": 0.49505208333333334, "acc_stderr,none": 0.008069370988058294, "acc_norm,none": 0.49505208333333334, "acc_norm_stderr,none": 0.008069370988058294 }, "StrongREJECT": { "alias": "StrongREJECT", "acc,none": 0.9424920127795527, "acc_stderr,none": 0.013180310031262344, "acc_norm,none": 0.19808306709265175, "acc_norm_stderr,none": 0.022563739178544238 }, "WildGuardTest": { "alias": "WildGuardTest", "acc,none": 0.5710144927536231, "acc_stderr,none": 0.011919999713357268, "acc_norm,none": 0.5640579710144927, "acc_norm_stderr,none": 0.011942839896636795 }, "bbq": { "acc_norm,none": 0.9325377829446762, "acc_norm_stderr,none": 0.0010218971033224293, "acc,none": 0.9324864938795049, "acc_stderr,none": 0.0010226628876829763, "alias": "bbq" }, "bbq_age": { "alias": " - bbq_age", "acc,none": 0.8301630434782609, "acc_stderr,none": 0.006190602280652597, "acc_norm,none": 0.8301630434782609, "acc_norm_stderr,none": 0.006190602280652598 }, "bbq_disabilitystatus": { "alias": " - bbq_disabilitystatus", "acc,none": 0.9042416452442159, "acc_stderr,none": 0.007462174379038646, "acc_norm,none": 0.9048843187660668, "acc_norm_stderr,none": 0.007439733789161572 }, "bbq_genderidentity": { "alias": " - bbq_genderidentity", "acc,none": 0.9442877291960508, "acc_stderr,none": 0.00304577473375189, "acc_norm,none": 0.942524682651622, "acc_norm_stderr,none": 0.003090702699187024 }, "bbq_nationality": { "alias": " - bbq_nationality", "acc,none": 0.9168831168831169, "acc_stderr,none": 0.00497504218811962, "acc_norm,none": 0.9181818181818182, "acc_norm_stderr,none": 0.0049395161759693985 }, "bbq_physicalappearance": { "alias": " - bbq_physicalappearance", "acc,none": 0.8299492385786802, "acc_stderr,none": 0.009466180402043615, "acc_norm,none": 0.8280456852791879, "acc_norm_stderr,none": 0.009508092739152258 }, "bbq_raceethnicity": { "alias": " - bbq_raceethnicity", "acc,none": 0.9156976744186046, "acc_stderr,none": 0.0033499103029905916, "acc_norm,none": 0.915406976744186, "acc_norm_stderr,none": 0.0033551483498529358 }, "bbq_racexgender": { "alias": " - bbq_racexgender", "acc,none": 0.9584586466165413, "acc_stderr,none": 0.0015795165224040926, "acc_norm,none": 0.9586466165413534, "acc_norm_stderr,none": 0.0015760934304819407 }, "bbq_racexses": { "alias": " - bbq_racexses", "acc,none": 0.9760752688172043, "acc_stderr,none": 0.0014466136180125736, "acc_norm,none": 0.9779569892473118, "acc_norm_stderr,none": 0.0013898972475625852 }, "bbq_religion": { "alias": " - bbq_religion", "acc,none": 0.835, "acc_stderr,none": 0.010719526896310953, "acc_norm,none": 0.8341666666666666, "acc_norm_stderr,none": 0.01074119842135935 }, "bbq_ses": { "alias": " - bbq_ses", "acc,none": 0.921037296037296, "acc_stderr,none": 0.003255311967010792, "acc_norm,none": 0.9195804195804196, "acc_norm_stderr,none": 0.0032826060053440993 }, "bbq_sexualorientation": { "alias": " - bbq_sexualorientation", "acc,none": 0.9016203703703703, "acc_stderr,none": 0.010138157908353061, "acc_norm,none": 0.9016203703703703, "acc_norm_stderr,none": 0.01013815790835306 }, "leaderboard": { " ": " ", "alias": "leaderboard" }, "leaderboard_bbh": { " ": " ", "alias": " - leaderboard_bbh" }, "leaderboard_bbh_boolean_expressions": { "alias": " - leaderboard_bbh_boolean_expressions", "acc_norm,none": 0.8, "acc_norm_stderr,none": 0.025348970020979078 }, "leaderboard_bbh_causal_judgement": { "alias": " - leaderboard_bbh_causal_judgement", "acc_norm,none": 0.6096256684491979, "acc_norm_stderr,none": 0.03576973947986412 }, "leaderboard_bbh_date_understanding": { "alias": " - leaderboard_bbh_date_understanding", "acc_norm,none": 0.472, "acc_norm_stderr,none": 0.031636489531544396 }, "leaderboard_bbh_disambiguation_qa": { "alias": " - leaderboard_bbh_disambiguation_qa", "acc_norm,none": 0.668, "acc_norm_stderr,none": 0.02984403904746591 }, "leaderboard_bbh_formal_fallacies": { "alias": " - leaderboard_bbh_formal_fallacies", "acc_norm,none": 0.612, "acc_norm_stderr,none": 0.03088103874899391 }, "leaderboard_bbh_geometric_shapes": { "alias": " - leaderboard_bbh_geometric_shapes", "acc_norm,none": 0.376, "acc_norm_stderr,none": 0.030696336267394583 }, "leaderboard_bbh_hyperbaton": { "alias": " - leaderboard_bbh_hyperbaton", "acc_norm,none": 0.708, "acc_norm_stderr,none": 0.02881432040220564 }, "leaderboard_bbh_logical_deduction_five_objects": { "alias": " - leaderboard_bbh_logical_deduction_five_objects", "acc_norm,none": 0.472, "acc_norm_stderr,none": 0.031636489531544396 }, "leaderboard_bbh_logical_deduction_seven_objects": { "alias": " - leaderboard_bbh_logical_deduction_seven_objects", "acc_norm,none": 0.448, "acc_norm_stderr,none": 0.03151438761115355 }, "leaderboard_bbh_logical_deduction_three_objects": { "alias": " - leaderboard_bbh_logical_deduction_three_objects", "acc_norm,none": 0.724, "acc_norm_stderr,none": 0.028328537274211352 }, "leaderboard_bbh_movie_recommendation": { "alias": " - leaderboard_bbh_movie_recommendation", "acc_norm,none": 0.668, "acc_norm_stderr,none": 0.02984403904746591 }, "leaderboard_bbh_navigate": { "alias": " - leaderboard_bbh_navigate", "acc_norm,none": 0.58, "acc_norm_stderr,none": 0.03127799950463661 }, "leaderboard_bbh_object_counting": { "alias": " - leaderboard_bbh_object_counting", "acc_norm,none": 0.34, "acc_norm_stderr,none": 0.030020073605457907 }, "leaderboard_bbh_penguins_in_a_table": { "alias": " - leaderboard_bbh_penguins_in_a_table", "acc_norm,none": 0.4657534246575342, "acc_norm_stderr,none": 0.04142522736934775 }, "leaderboard_bbh_reasoning_about_colored_objects": { "alias": " - leaderboard_bbh_reasoning_about_colored_objects", "acc_norm,none": 0.56, "acc_norm_stderr,none": 0.03145724452223563 }, "leaderboard_bbh_ruin_names": { "alias": " - leaderboard_bbh_ruin_names", "acc_norm,none": 0.632, "acc_norm_stderr,none": 0.030562070620993163 }, "leaderboard_bbh_salient_translation_error_detection": { "alias": " - leaderboard_bbh_salient_translation_error_detection", "acc_norm,none": 0.476, "acc_norm_stderr,none": 0.03164968895968782 }, "leaderboard_bbh_snarks": { "alias": " - leaderboard_bbh_snarks", "acc_norm,none": 0.7078651685393258, "acc_norm_stderr,none": 0.03418064934010763 }, "leaderboard_bbh_sports_understanding": { "alias": " - leaderboard_bbh_sports_understanding", "acc_norm,none": 0.78, "acc_norm_stderr,none": 0.026251792824605824 }, "leaderboard_bbh_temporal_sequences": { "alias": " - leaderboard_bbh_temporal_sequences", "acc_norm,none": 0.256, "acc_norm_stderr,none": 0.02765710871820491 }, "leaderboard_bbh_tracking_shuffled_objects_five_objects": { "alias": " - leaderboard_bbh_tracking_shuffled_objects_five_objects", "acc_norm,none": 0.192, "acc_norm_stderr,none": 0.024960691989172015 }, "leaderboard_bbh_tracking_shuffled_objects_seven_objects": { "alias": " - leaderboard_bbh_tracking_shuffled_objects_seven_objects", "acc_norm,none": 0.14, "acc_norm_stderr,none": 0.021989409645240272 }, "leaderboard_bbh_tracking_shuffled_objects_three_objects": { "alias": " - leaderboard_bbh_tracking_shuffled_objects_three_objects", "acc_norm,none": 0.3, "acc_norm_stderr,none": 0.029040893477575845 }, "leaderboard_bbh_web_of_lies": { "alias": " - leaderboard_bbh_web_of_lies", "acc_norm,none": 0.48, "acc_norm_stderr,none": 0.03166085340849519 }, "leaderboard_gpqa": { " ": " ", "alias": " - leaderboard_gpqa" }, "leaderboard_gpqa_diamond": { "alias": " - leaderboard_gpqa_diamond", "acc_norm,none": 0.29292929292929293, "acc_norm_stderr,none": 0.032424979581788166 }, "leaderboard_gpqa_extended": { "alias": " - leaderboard_gpqa_extended", "acc_norm,none": 0.30952380952380953, "acc_norm_stderr,none": 0.019802641880170335 }, "leaderboard_gpqa_main": { "alias": " - leaderboard_gpqa_main", "acc_norm,none": 0.29017857142857145, "acc_norm_stderr,none": 0.02146611544057122 }, "leaderboard_ifeval": { "alias": " - leaderboard_ifeval", "prompt_level_strict_acc,none": 0.3826247689463956, "prompt_level_strict_acc_stderr,none": 0.020915307841436647, "inst_level_strict_acc,none": 0.5179856115107914, "inst_level_strict_acc_stderr,none": "N/A", "prompt_level_loose_acc,none": 0.4602587800369686, "prompt_level_loose_acc_stderr,none": 0.02144850143413504, "inst_level_loose_acc,none": 0.5935251798561151, "inst_level_loose_acc_stderr,none": "N/A" }, "leaderboard_math_hard": { " ": " ", "alias": " - leaderboard_math_hard" }, "leaderboard_math_algebra_hard": { "alias": " - leaderboard_math_algebra_hard", "exact_match,none": 0.11400651465798045, "exact_match_stderr,none": 0.018168513062629208 }, "leaderboard_math_counting_and_prob_hard": { "alias": " - leaderboard_math_counting_and_prob_hard", "exact_match,none": 0.04878048780487805, "exact_match_stderr,none": 0.019502196558588064 }, "leaderboard_math_geometry_hard": { "alias": " - leaderboard_math_geometry_hard", "exact_match,none": 0.022727272727272728, "exact_match_stderr,none": 0.01302104690906372 }, "leaderboard_math_intermediate_algebra_hard": { "alias": " - leaderboard_math_intermediate_algebra_hard", "exact_match,none": 0.02857142857142857, "exact_match_stderr,none": 0.009973998820736029 }, "leaderboard_math_num_theory_hard": { "alias": " - leaderboard_math_num_theory_hard", "exact_match,none": 0.07142857142857142, "exact_match_stderr,none": 0.020820824576076355 }, "leaderboard_math_prealgebra_hard": { "alias": " - leaderboard_math_prealgebra_hard", "exact_match,none": 0.19689119170984457, "exact_match_stderr,none": 0.02869787397186069 }, "leaderboard_math_precalculus_hard": { "alias": " - leaderboard_math_precalculus_hard", "exact_match,none": 0.007407407407407408, "exact_match_stderr,none": 0.007407407407407402 }, "leaderboard_mmlu_pro": { "alias": " - leaderboard_mmlu_pro", "acc,none": 0.31299867021276595, "acc_stderr,none": 0.004227651421740313 }, "leaderboard_musr": { " ": " ", "alias": " - leaderboard_musr" }, "leaderboard_musr_murder_mysteries": { "alias": " - leaderboard_musr_murder_mysteries", "acc_norm,none": 0.58, "acc_norm_stderr,none": 0.03127799950463661 }, "leaderboard_musr_object_placements": { "alias": " - leaderboard_musr_object_placements", "acc_norm,none": 0.3671875, "acc_norm_stderr,none": 0.030186403889489913 }, "leaderboard_musr_team_allocation": { "alias": " - leaderboard_musr_team_allocation", "acc_norm,none": 0.372, "acc_norm_stderr,none": 0.03063032594455831 }, "toxigen": { "alias": "toxigen", "acc,none": 0.6, "acc_stderr,none": 0.015987215339548645, "acc_norm,none": 0.5702127659574469, "acc_norm_stderr,none": 0.016155203301509467 }, "wmdp": { "acc,none": 0.531897491821156, "acc_stderr,none": 0.008068859365825751, "alias": "wmdp" }, "wmdp_bio": { "alias": " - wmdp_bio", "acc,none": 0.6684996072270227, "acc_stderr,none": 0.013199243664138346 }, "wmdp_chem": { "alias": " - wmdp_chem", "acc,none": 0.5171568627450981, "acc_stderr,none": 0.024769484776043047 }, "wmdp_cyber": { "alias": " - wmdp_cyber", "acc,none": 0.447408152994464, "acc_stderr,none": 0.011157440033040955 }, "xstest": { "alias": "xstest", "acc,none": 0.4666666666666667, "acc_stderr,none": 0.023543963941101545, "acc_norm,none": 0.4444444444444444, "acc_norm_stderr,none": 0.023450349399618212 } } ``` ## Dataset Details ### Dataset Description  - **Curated by:** [More Information Needed] - **Funded by [optional]:** [More Information Needed] - **Shared by [optional]:** [More Information Needed] - **Language(s) (NLP):** [More Information Needed] - **License:** [More Information Needed] ### Dataset Sources [optional]  - **Repository:** [More Information Needed] - **Paper [optional]:** [More Information Needed] - **Demo [optional]:** [More Information Needed] ## Uses  ### Direct Use  [More Information Needed] ### Out-of-Scope Use  [More Information Needed] ## Dataset Structure  [More Information Needed] ## Dataset Creation ### Curation Rationale  [More Information Needed] ### Source Data  #### Data Collection and Processing  [More Information Needed] #### Who are the source data producers?  [More Information Needed] ### Annotations [optional]  #### Annotation process  [More Information Needed] #### Who are the annotators?  [More Information Needed] #### Personal and Sensitive Information  [More Information Needed] ## Bias, Risks, and Limitations  [More Information Needed] ### Recommendations  Users should be made aware of the risks, biases and limitations of the dataset. More information needed for further recommendations. ## Citation [optional]  **BibTeX:** [More Information Needed] **APA:** [More Information Needed] ## Glossary [optional]  [More Information Needed] ## More Information [optional] [More Information Needed] ## Dataset Card Authors [optional] [More Information Needed] ## Dataset Card Contact [More Information Needed]

--- pretty_name: zhengr/MixTAO-7Bx2-MoE-v8.1 模型评估运行数据集 dataset_summary: "本数据集是在对模型 [zhengr/MixTAO-7Bx2-MoE-v8.1](https://huggingface.co/zhengr/MixTAO-7Bx2-MoE-v8.1) 执行评估运行时自动生成的。该数据集共包含62个配置项，每个配置项对应一个被评估的任务。本数据集基于2次评估运行构建，每一次运行的结果会以该运行的时间戳作为划分（split）名称，存储在每个配置项中。其中，`train`划分始终指向最新的评估结果。此外，额外提供了一个名为`results`的配置项，用于存储本次评估运行的所有聚合结果。若需加载某次评估运行的详细数据，可参考如下示例代码： python from datasets import load_dataset data = load_dataset( "nyu-dice-lab/lm-eval-results-zhengr-MixTAO-7Bx2-MoE-v8.1-private", name="zhengr__MixTAO-7Bx2-MoE-v8.1__BeaverTailsEval", split="latest" ) ## 最新评估结果以下为来自2024-12-05T00-28-46.710222运行的最新结果（注：若后续多次评估未覆盖全部任务，则仓库中可能存在其他任务的评估结果，您可在各评估的`results`和`latest`划分中找到对应内容）： python {完整的原始评估结果JSON内容，此处省略示意，实际翻译时保留完整原文} " repo_url: https://huggingface.co/zhengr/MixTAO-7Bx2-MoE-v8.1 leaderboard_url: 无 point_of_contact: 无 configs: - 配置名称：zhengr__MixTAO-7Bx2-MoE-v8.1__BeaverTailsEval 数据文件： - 划分：2024_12_05T00_28_46.710222 路径： - '**/samples_BeaverTailsEval_2024-12-05T00-28-46.710222.jsonl' - 划分：latest 路径： - '**/samples_BeaverTailsEval_2024-12-05T00-28-46.710222.jsonl' // 其余配置项按此格式翻译，保留原有任务名称与文件路径 # zhengr/MixTAO-7Bx2-MoE-v8.1 模型评估运行数据集卡片  本数据集是在对模型 [zhengr/MixTAO-7Bx2-MoE-v8.1](https://huggingface.co/zhengr/MixTAO-7Bx2-MoE-v8.1) 执行评估运行时自动生成的。该数据集共包含62个配置项，每个配置项对应一个被评估的任务。本数据集基于2次评估运行构建，每一次运行的结果会以该运行的时间戳作为划分（split）名称，存储在每个配置项中。其中，`train`划分始终指向最新的评估结果。此外，额外提供了一个名为`results`的配置项，用于存储本次评估运行的所有聚合结果。若需加载某次评估运行的详细数据，可参考如下示例代码： python from datasets import load_dataset data = load_dataset( "nyu-dice-lab/lm-eval-results-zhengr-MixTAO-7Bx2-MoE-v8.1-private", name="zhengr__MixTAO-7Bx2-MoE-v8.1__BeaverTailsEval", split="latest" ) ## 最新评估结果以下为来自2024-12-05T00-28-46.710222运行的最新结果（注：若后续多次评估未覆盖全部任务，则仓库中可能存在其他任务的评估结果，您可在各评估的`results`和`latest`划分中找到对应内容）： python {完整的原始评估结果JSON内容，此处省略示意，实际翻译时保留完整原文} ## 数据集详情 ### 数据集概述  - **整理方：** [需补充更多信息] - **资助方（可选）：** [需补充更多信息] - **共享方（可选）：** [需补充更多信息] - **自然语言：** [需补充更多信息] - **授权协议：** [需补充更多信息] ### 数据集来源（可选）  - **代码仓库：** [需补充更多信息] - **相关论文（可选）：** [需补充更多信息] - **演示示例（可选）：** [需补充更多信息] ## 数据集用途 ### 直接用途  [需补充更多信息] ### 超出范围的使用场景  [需补充更多信息] ## 数据集结构  [需补充更多信息] ## 数据集创建 ### 整理依据  [需补充更多信息] ### 源数据 #### 数据收集与处理流程  [需补充更多信息] #### 源数据生产者  [需补充更多信息] ### 标注信息（可选）  #### 标注流程  [需补充更多信息] #### 标注者  [需补充更多信息] #### 个人与敏感信息  [需补充更多信息] ## 偏见、风险与局限性  [需补充更多信息] ### 建议  用户应知晓本数据集存在的风险、偏见与局限性，相关建议需进一步补充完善。 ## 引用信息（可选）  **BibTeX格式：** [需补充更多信息] **APA格式：** [需补充更多信息] ## 术语表（可选）  [需补充更多信息] ## 更多信息（可选） [需补充更多信息] ## 数据集卡片作者（可选） [需补充更多信息] ## 数据集卡片联系人 [需补充更多信息]

应用场景：