open-llm-leaderboard/details_JosephusCheung__Qwen-LLaMAfied-7B-Chat

--- pretty_name: Evaluation run of JosephusCheung/Qwen-LLaMAfied-7B-Chat dataset_summary: "Dataset automatically created during the evaluation run of model\ \ [JosephusCheung/Qwen-LLaMAfied-7B-Chat](https://huggingface.co/JosephusCheung/Qwen-LLaMAfied-7B-Chat)\ \ on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).\n\ \nThe dataset is composed of 64 configuration, each one coresponding to one of the\ \ evaluated task.\n\nThe dataset has been created from 2 run(s). Each run can be\ \ found as a specific split in each configuration, the split being named using the\ \ timestamp of the run.The \"train\" split is always pointing to the latest results.\n\ \nAn additional configuration \"results\" store all the aggregated results of the\ \ run (and is used to compute and display the agregated metrics on the [Open LLM\ \ Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)).\n\ \nTo load the details from a run, you can for instance do the following:\n```python\n\ from datasets import load_dataset\ndata = load_dataset(\"open-llm-leaderboard/details_JosephusCheung__Qwen-LLaMAfied-7B-Chat\"\ ,\n\t\"harness_winogrande_5\",\n\tsplit=\"train\")\n```\n\n## Latest results\n\n\ These are the [latest results from run 2023-10-29T05:54:59.935248](https://huggingface.co/datasets/open-llm-leaderboard/details_JosephusCheung__Qwen-LLaMAfied-7B-Chat/blob/main/results_2023-10-29T05-54-59.935248.json)(note\ \ that their might be results for other tasks in the repos if successive evals didn't\ \ cover the same tasks. You find each in the results and the \"latest\" split for\ \ each eval):\n\n```python\n{\n \"all\": {\n \"em\": 0.29425335570469796,\n\ \ \"em_stderr\": 0.004666860017033486,\n \"f1\": 0.3722158137583904,\n\ \ \"f1_stderr\": 0.004557451176367578,\n \"acc\": 0.38970651153411406,\n\ \ \"acc_stderr\": 0.009163863947895253\n },\n \"harness|drop|3\": {\n\ \ \"em\": 0.29425335570469796,\n \"em_stderr\": 0.004666860017033486,\n\ \ \"f1\": 0.3722158137583904,\n \"f1_stderr\": 0.004557451176367578\n\ \ },\n \"harness|gsm8k|5\": {\n \"acc\": 0.047763457164518575,\n \ \ \"acc_stderr\": 0.00587438753622931\n },\n \"harness|winogrande|5\"\ : {\n \"acc\": 0.7316495659037096,\n \"acc_stderr\": 0.012453340359561195\n\ \ }\n}\n```" repo_url: https://huggingface.co/JosephusCheung/Qwen-LLaMAfied-7B-Chat leaderboard_url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard point_of_contact: [e-mail] configs: - config_name: harness_arc_challenge_25 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|arc:challenge|25_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|arc:challenge|25_2023-09-12T19-56-23.146408.parquet' - config_name: harness_drop_3 data_files: - split: 2023_10_29T05_54_59.935248 path: - '**/details_harness|drop|3_2023-10-29T05-54-59.935248.parquet' - split: latest path: - '**/details_harness|drop|3_2023-10-29T05-54-59.935248.parquet' - config_name: harness_gsm8k_5 data_files: - split: 2023_10_29T05_54_59.935248 path: - '**/details_harness|gsm8k|5_2023-10-29T05-54-59.935248.parquet' - split: latest path: - '**/details_harness|gsm8k|5_2023-10-29T05-54-59.935248.parquet' - config_name: harness_hellaswag_10 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hellaswag|10_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hellaswag|10_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-international_law|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-management|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-marketing|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-sociology|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-virology|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-international_law|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-management|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-marketing|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-sociology|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-virology|5_2023-09-12T19-56-23.146408.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_abstract_algebra_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_anatomy_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-anatomy|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-anatomy|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_astronomy_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-astronomy|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-astronomy|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_business_ethics_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-business_ethics|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-business_ethics|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_clinical_knowledge_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_college_biology_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-college_biology|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_biology|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_college_chemistry_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-college_chemistry|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_chemistry|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_college_computer_science_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-college_computer_science|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_computer_science|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_college_mathematics_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-college_mathematics|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_mathematics|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_college_medicine_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-college_medicine|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_medicine|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_college_physics_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-college_physics|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_physics|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_computer_security_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-computer_security|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-computer_security|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_conceptual_physics_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_econometrics_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-econometrics|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-econometrics|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_electrical_engineering_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_elementary_mathematics_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_formal_logic_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-formal_logic|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-formal_logic|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_global_facts_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-global_facts|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-global_facts|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_high_school_biology_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-high_school_biology|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_biology|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_high_school_chemistry_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_high_school_computer_science_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_high_school_european_history_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_high_school_geography_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-high_school_geography|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_geography|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_high_school_government_and_politics_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_high_school_macroeconomics_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_high_school_mathematics_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_high_school_microeconomics_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_high_school_physics_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-high_school_physics|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_physics|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_high_school_psychology_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_high_school_statistics_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_high_school_us_history_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_high_school_world_history_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_human_aging_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-human_aging|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-human_aging|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_human_sexuality_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-human_sexuality|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-human_sexuality|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_international_law_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-international_law|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-international_law|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_jurisprudence_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-jurisprudence|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-jurisprudence|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_logical_fallacies_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_machine_learning_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-machine_learning|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-machine_learning|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_management_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-management|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-management|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_marketing_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-marketing|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-marketing|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_medical_genetics_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-medical_genetics|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-medical_genetics|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_miscellaneous_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-miscellaneous|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-miscellaneous|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_moral_disputes_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-moral_disputes|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-moral_disputes|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_moral_scenarios_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_nutrition_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-nutrition|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-nutrition|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_philosophy_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-philosophy|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-philosophy|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_prehistory_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-prehistory|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-prehistory|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_professional_accounting_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-professional_accounting|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_accounting|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_professional_law_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-professional_law|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_law|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_professional_medicine_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-professional_medicine|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_medicine|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_professional_psychology_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-professional_psychology|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_psychology|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_public_relations_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-public_relations|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-public_relations|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_security_studies_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-security_studies|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-security_studies|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_sociology_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-sociology|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-sociology|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_us_foreign_policy_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_virology_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-virology|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-virology|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_hendrycksTest_world_religions_5 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|hendrycksTest-world_religions|5_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|hendrycksTest-world_religions|5_2023-09-12T19-56-23.146408.parquet' - config_name: harness_truthfulqa_mc_0 data_files: - split: 2023_09_12T19_56_23.146408 path: - '**/details_harness|truthfulqa:mc|0_2023-09-12T19-56-23.146408.parquet' - split: latest path: - '**/details_harness|truthfulqa:mc|0_2023-09-12T19-56-23.146408.parquet' - config_name: harness_winogrande_5 data_files: - split: 2023_10_29T05_54_59.935248 path: - '**/details_harness|winogrande|5_2023-10-29T05-54-59.935248.parquet' - split: latest path: - '**/details_harness|winogrande|5_2023-10-29T05-54-59.935248.parquet' - config_name: results data_files: - split: 2023_09_12T19_56_23.146408 path: - results_2023-09-12T19-56-23.146408.parquet - split: 2023_10_29T05_54_59.935248 path: - results_2023-10-29T05-54-59.935248.parquet - split: latest path: - results_2023-10-29T05-54-59.935248.parquet --- # Dataset Card for Evaluation run of JosephusCheung/Qwen-LLaMAfied-7B-Chat ## Dataset Description - **Homepage:** - **Repository:** https://huggingface.co/JosephusCheung/Qwen-LLaMAfied-7B-Chat - **Paper:** - **Leaderboard:** https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard - **Point of Contact:** [e-mail] ### Dataset Summary Dataset automatically created during the evaluation run of model [JosephusCheung/Qwen-LLaMAfied-7B-Chat](https://huggingface.co/JosephusCheung/Qwen-LLaMAfied-7B-Chat) on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard). The dataset is composed of 64 configuration, each one coresponding to one of the evaluated task. The dataset has been created from 2 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest results. An additional configuration "results" store all the aggregated results of the run (and is used to compute and display the agregated metrics on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)). To load the details from a run, you can for instance do the following: ```python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_JosephusCheung__Qwen-LLaMAfied-7B-Chat", "harness_winogrande_5", split="train") ``` ## Latest results These are the [latest results from run 2023-10-29T05:54:59.935248](https://huggingface.co/datasets/open-llm-leaderboard/details_JosephusCheung__Qwen-LLaMAfied-7B-Chat/blob/main/results_2023-10-29T05-54-59.935248.json)(note that their might be results for other tasks in the repos if successive evals didn't cover the same tasks. You find each in the results and the "latest" split for each eval): ```python { "all": { "em": 0.29425335570469796, "em_stderr": 0.004666860017033486, "f1": 0.3722158137583904, "f1_stderr": 0.004557451176367578, "acc": 0.38970651153411406, "acc_stderr": 0.009163863947895253 }, "harness|drop|3": { "em": 0.29425335570469796, "em_stderr": 0.004666860017033486, "f1": 0.3722158137583904, "f1_stderr": 0.004557451176367578 }, "harness|gsm8k|5": { "acc": 0.047763457164518575, "acc_stderr": 0.00587438753622931 }, "harness|winogrande|5": { "acc": 0.7316495659037096, "acc_stderr": 0.012453340359561195 } } ``` ### Supported Tasks and Leaderboards [More Information Needed] ### Languages [More Information Needed] ## Dataset Structure ### Data Instances [More Information Needed] ### Data Fields [More Information Needed] ### Data Splits [More Information Needed] ## Dataset Creation ### Curation Rationale [More Information Needed] ### Source Data #### Initial Data Collection and Normalization [More Information Needed] #### Who are the source language producers? [More Information Needed] ### Annotations #### Annotation process [More Information Needed] #### Who are the annotators? [More Information Needed] ### Personal and Sensitive Information [More Information Needed] ## Considerations for Using the Data ### Social Impact of Dataset [More Information Needed] ### Discussion of Biases [More Information Needed] ### Other Known Limitations [More Information Needed] ## Additional Information ### Dataset Curators [More Information Needed] ### Licensing Information [More Information Needed] ### Citation Information [More Information Needed] ### Contributions [More Information Needed]

原始信息汇总

数据集概述

数据集来源

该数据集是在评估模型 JosephusCheung/Qwen-LLaMAfied-7B-Chat 在 Open LLM Leaderboard 上的运行过程中自动创建的。

数据集结构

• 配置数量：64个配置，每个配置对应一个评估任务。
• 运行次数：数据集从2次运行中创建。每个运行在每个配置中作为一个特定的分片存在，分片名称使用运行的时间戳。
• 分片命名：每个配置的分片名称使用运行的时间戳，"train" 分片始终指向最新的结果。

额外配置

• 结果配置：一个额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示在 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_JosephusCheung__Qwen-LLaMAfied-7B-Chat", "harness_winogrande_5", split="train")

最新结果

以下是 2023-10-29T05:54:59.935248 运行的最新结果： python { "all": { "em": 0.29425335570469796, "em_stderr": 0.004666860017033486, "f1": 0.3722158137583904, "f1_stderr": 0.004557451176367578, "acc": 0.38970651153411406, "acc_stderr": 0.009163863947895253 }, "harness|drop|3": { "em": 0.29425335570469796, "em_stderr": 0.004666860017033486, "f1": 0.3722158137583904, "f1_stderr": 0.004557451176367578 }, "harness|gsm8k|5": { "acc": 0.047763457164518575, "acc_stderr": 0.00587438753622931 }, "harness|winogrande|5": { "acc": 0.7316495659037096, "acc_stderr": 0.012453340359561195 } }

配置详情

以下是部分配置的详细信息：

• harness_arc_challenge_25

  • 分片：2023_09_12T19_56_23.146408
  • 路径：**/details_harness|arc:challenge|25_2023-09-12T19-56-23.146408.parquet
  • 分片：latest
  • 路径：**/details_harness|arc:challenge|25_2023-09-12T19-56-23.146408.parquet

• harness_drop_3

  • 分片：2023_10_29T05_54_59.935248
  • 路径：**/details_harness|drop|3_2023-10-29T05-54-59.935248.parquet
  • 分片：latest
  • 路径：**/details_harness|drop|3_2023-10-29T05-54-59.935248.parquet

• harness_gsm8k_5

  • 分片：2023_10_29T05_54_59.935248
  • 路径：**/details_harness|gsm8k|5_2023-10-29T05-54-59.935248.parquet
  • 分片：latest
  • 路径：**/details_harness|gsm8k|5_2023-10-29T05-54-59.935248.parquet

• harness_hellaswag_10

  • 分片：2023_09_12T19_56_23.146408
  • 路径：**/details_harness|hellaswag|10_2023-09-12T19-56-23.146408.parquet
  • 分片：latest
  • 路径：**/details_harness|hellaswag|10_2023-09-12T19-56-23.146408.parquet

• harness_hendrycksTest_5

  • 分片：2023_09_12T19_56_23.146408
  • 路径：**/details_harness|hendrycksTest-abstract_algebra|5_2023-09-12T19-56-23.146408.parquet 等
  • 分片：latest
  • 路径：**/details_harness|hendrycksTest-abstract_algebra|5_2023-09-12T19-56-23.146408.parquet 等

搜集汇总

数据集介绍

构建方式

该数据集是在Open LLM Leaderboard框架下，对模型JosephusCheung/Qwen-LLaMAfied-7B-Chat进行自动评估过程中生成的副产品。它囊括了64个独立的配置，每个配置精准对应一项被评估的基准任务，诸如ARC挑战、DROP、GSM8K、HellaSwag以及涵盖广泛学科的MMLU测试等。数据集的构建源自两轮独立的评估运行，每轮运行的结果被存储为特定配置下的独立数据分割，分割名称以运行时间戳命名，而统一的'train'分割则始终指向最新的评估结果。另外，一个名为'results'的专门配置用于汇总所有运行的聚合指标，为排行榜上综合得分的计算与展示提供支撑。

特点

此数据集的核心特征在于其结构化的多任务评估记录体系。它并非单一任务的结果集，而是将大语言模型在数十个差异化基准上的表现细节有机整合，每个配置下的数据分割忠实记录了特定时间点的评估原始数据。这种设计使得研究者能够追溯模型性能的演化轨迹，对比不同运行间的波动。同时，数据集以Parquet格式高效存储，保证了大规模数值型评估结果（如准确率、F1分数及其标准误）的快速读写与处理能力，为深入分析模型在推理、常识理解、数学计算及专业知识等维度的优劣提供了坚实的数据基础。

使用方法

使用者可通过Hugging Face的datasets库便捷地加载该数据集。具体而言，调用load_dataset函数并指定数据集名称以及目标任务的配置名称（如'harness_winogrande_5'），即可获取对应的评估详情。通过设置split参数为'train'，可以访问该任务的最新评估结果；若需回溯历史记录，则可将split指定为对应运行的时间戳字符串。对于需要全局视角的分析，加载名为'results'的配置能够获取所有任务聚合后的综合指标，便于进行模型整体性能的横向对比与趋势研判。

背景与挑战

背景概述

在大语言模型迅猛发展的浪潮中，如何系统性地评估模型在多样化任务上的表现已成为核心议题。Open LLM Leaderboard由HuggingFace团队于2023年创建，旨在为开源社区提供一个标准化、可复现的模型评测基准。该数据集记录了JosephusCheung/Qwen-LLaMAfied-7B-Chat模型在多项任务上的详细评估结果，涵盖ARC Challenge、DROP、GSM8K、HellaSwag、WinoGrande以及涵盖57个学科的MMLU测试。通过跨多个时间点的运行记录，该数据集不仅揭示了模型在常识推理、数学计算和知识理解等维度的能力边界，更推动了开源大模型评测体系的透明化与可比性，成为社区衡量模型进展的重要参照。

当前挑战

该数据集所解决的领域挑战在于，大语言模型的性能评估长期缺乏统一标准，不同研究采用的评测任务、采样方式和指标各异，导致模型间难以公平比较。构建过程中面临的挑战包括：需确保各评测任务数据格式的兼容性与一致性，将来自不同来源的ARC、HellaSwag等基准转化为统一Parquet格式；需管理多轮评估运行的时间戳分裂，以保证结果的可追溯性与最新性；还需处理如DROP任务中的精确匹配与F1分数等多维度指标的计算与聚合，以提供全面的性能画像。此外，持续跟踪社区提交的模型并自动化生成评测数据集，也对基础设施的鲁棒性提出了较高要求。

常用场景

经典使用场景

在大型语言模型（LLM）评估领域，Open LLM Leaderboard 提供的评估结果数据集扮演着标杆角色。该数据集专门记录了 JosephusCheung/Qwen-LLaMAfied-7B-Chat 模型在多个标准化基准任务上的表现，涵盖 ARC-Challenge、HellaSwag、MMLU、Winogrande、GSM8K 和 DROP 等经典评测。研究者常利用此数据集快速获取模型在常识推理、数学求解、阅读理解及多学科知识等维度的量化指标，从而横向对比不同模型的性能优劣，为模型选型与迭代提供可靠依据。

衍生相关工作

该数据集衍生了一系列围绕模型评估与改进的经典工作。其作为 Open LLM Leaderboard 生态的一部分，启发了后续研究如多任务联合微调策略的提出，以及针对低分任务（如 GSM8K）的专项增强方法。此外，数据集中 MMLU 子任务的 57 个学科配置被广泛用于构建知识蒸馏基线，而 DROP 的离散推理指标则催生了数值推理增强模块的设计。这些工作共同推动了语言模型在鲁棒性与泛化性上的进步，并为社区提供了可复现的评估范式。

数据集最近研究

最新研究方向

围绕大语言模型评估基准的精细化与标准化，Open LLM Leaderboard 数据集为模型性能的横向对比提供了关键支撑。当前前沿方向聚焦于多任务泛化能力的量化分析，尤其是模型在复杂推理（如GSM8K数学推理）、常识理解（如WinoGrande）及长文本阅读理解（如DROP）等任务上的表现。该数据集通过记录Qwen-LLaMAfied-7B-Chat等模型的详细评估结果，揭示了不同架构与微调策略在知识密集型场景下的脆弱性——例如该模型在GSM8K上准确率不足5%，凸显了数学推理仍是当前大模型的显著短板。这一发现推动了研究者对链式思维提示、工具增强学习等优化路径的探索。此外，数据集的公开可复现性促进了社区对评估一致性的审计，成为大模型竞技场中不可或缺的公平性基石。