open-llm-leaderboard/details_migtissera__Synthia-7B

--- pretty_name: Evaluation run of migtissera/Synthia-7B dataset_summary: "Dataset automatically created during the evaluation run of model\ \ [migtissera/Synthia-7B](https://huggingface.co/migtissera/Synthia-7B) on the [Open\ \ LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).\n\ \nThe dataset is composed of 64 configuration, each one coresponding to one of the\ \ evaluated task.\n\nThe dataset has been created from 2 run(s). Each run can be\ \ found as a specific split in each configuration, the split being named using the\ \ timestamp of the run.The \"train\" split is always pointing to the latest results.\n\ \nAn additional configuration \"results\" store all the aggregated results of the\ \ run (and is used to compute and display the agregated metrics on the [Open LLM\ \ Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)).\n\ \nTo load the details from a run, you can for instance do the following:\n```python\n\ from datasets import load_dataset\ndata = load_dataset(\"open-llm-leaderboard/details_migtissera__Synthia-7B\"\ ,\n\t\"harness_winogrande_5\",\n\tsplit=\"train\")\n```\n\n## Latest results\n\n\ These are the [latest results from run 2023-10-15T06:07:54.738296](https://huggingface.co/datasets/open-llm-leaderboard/details_migtissera__Synthia-7B/blob/main/results_2023-10-15T06-07-54.738296.json)(note\ \ that their might be results for other tasks in the repos if successive evals didn't\ \ cover the same tasks. You find each in the results and the \"latest\" split for\ \ each eval):\n\n```python\n{\n \"all\": {\n \"em\": 0.07151845637583892,\n\ \ \"em_stderr\": 0.00263897548039012,\n \"f1\": 0.14513737416107345,\n\ \ \"f1_stderr\": 0.0029452435334875074,\n \"acc\": 0.4043291747772373,\n\ \ \"acc_stderr\": 0.009561470405449964\n },\n \"harness|drop|3\": {\n\ \ \"em\": 0.07151845637583892,\n \"em_stderr\": 0.00263897548039012,\n\ \ \"f1\": 0.14513737416107345,\n \"f1_stderr\": 0.0029452435334875074\n\ \ },\n \"harness|gsm8k|5\": {\n \"acc\": 0.06595905989385899,\n \ \ \"acc_stderr\": 0.006836951192034222\n },\n \"harness|winogrande|5\"\ : {\n \"acc\": 0.7426992896606156,\n \"acc_stderr\": 0.012285989618865708\n\ \ }\n}\n```" repo_url: https://huggingface.co/migtissera/Synthia-7B leaderboard_url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard point_of_contact: [e-mail] configs: - config_name: harness_arc_challenge_25 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|arc:challenge|25_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|arc:challenge|25_2023-08-17T17:21:07.158534.parquet' - config_name: harness_drop_3 data_files: - split: 2023_10_15T06_07_54.738296 path: - '**/details_harness|drop|3_2023-10-15T06-07-54.738296.parquet' - split: latest path: - '**/details_harness|drop|3_2023-10-15T06-07-54.738296.parquet' - config_name: harness_gsm8k_5 data_files: - split: 2023_10_15T06_07_54.738296 path: - '**/details_harness|gsm8k|5_2023-10-15T06-07-54.738296.parquet' - split: latest path: - '**/details_harness|gsm8k|5_2023-10-15T06-07-54.738296.parquet' - config_name: harness_hellaswag_10 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hellaswag|10_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hellaswag|10_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-international_law|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-management|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-marketing|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-sociology|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-virology|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-international_law|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-management|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-marketing|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-sociology|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-virology|5_2023-08-17T17:21:07.158534.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_abstract_algebra_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_anatomy_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-anatomy|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-anatomy|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_astronomy_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-astronomy|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-astronomy|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_business_ethics_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_clinical_knowledge_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_college_biology_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-college_biology|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_biology|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_college_chemistry_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_college_computer_science_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_college_mathematics_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_college_medicine_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_college_physics_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-college_physics|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_physics|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_computer_security_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-computer_security|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-computer_security|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_conceptual_physics_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_econometrics_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-econometrics|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-econometrics|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_electrical_engineering_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_elementary_mathematics_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_formal_logic_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_global_facts_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-global_facts|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-global_facts|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_high_school_biology_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_high_school_chemistry_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_high_school_computer_science_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_high_school_european_history_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_high_school_geography_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_high_school_government_and_politics_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_high_school_macroeconomics_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_high_school_mathematics_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_high_school_microeconomics_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_high_school_physics_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_high_school_psychology_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_high_school_statistics_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_high_school_us_history_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_high_school_world_history_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_human_aging_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-human_aging|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-human_aging|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_human_sexuality_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_international_law_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-international_law|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-international_law|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_jurisprudence_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_logical_fallacies_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_machine_learning_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_management_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-management|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-management|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_marketing_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-marketing|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-marketing|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_medical_genetics_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_miscellaneous_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_moral_disputes_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_moral_scenarios_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_nutrition_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-nutrition|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-nutrition|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_philosophy_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-philosophy|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-philosophy|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_prehistory_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-prehistory|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-prehistory|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_professional_accounting_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_professional_law_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-professional_law|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_law|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_professional_medicine_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_professional_psychology_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_public_relations_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-public_relations|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-public_relations|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_security_studies_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-security_studies|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-security_studies|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_sociology_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-sociology|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-sociology|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_us_foreign_policy_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_virology_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-virology|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-virology|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_hendrycksTest_world_religions_5 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|hendrycksTest-world_religions|5_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|hendrycksTest-world_religions|5_2023-08-17T17:21:07.158534.parquet' - config_name: harness_truthfulqa_mc_0 data_files: - split: 2023_08_17T17_21_07.158534 path: - '**/details_harness|truthfulqa:mc|0_2023-08-17T17:21:07.158534.parquet' - split: latest path: - '**/details_harness|truthfulqa:mc|0_2023-08-17T17:21:07.158534.parquet' - config_name: harness_winogrande_5 data_files: - split: 2023_10_15T06_07_54.738296 path: - '**/details_harness|winogrande|5_2023-10-15T06-07-54.738296.parquet' - split: latest path: - '**/details_harness|winogrande|5_2023-10-15T06-07-54.738296.parquet' - config_name: results data_files: - split: 2023_08_17T17_21_07.158534 path: - results_2023-08-17T17:21:07.158534.parquet - split: 2023_10_15T06_07_54.738296 path: - results_2023-10-15T06-07-54.738296.parquet - split: latest path: - results_2023-10-15T06-07-54.738296.parquet --- # Dataset Card for Evaluation run of migtissera/Synthia-7B ## Dataset Description - **Homepage:** - **Repository:** https://huggingface.co/migtissera/Synthia-7B - **Paper:** - **Leaderboard:** https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard - **Point of Contact:** [e-mail] ### Dataset Summary Dataset automatically created during the evaluation run of model [migtissera/Synthia-7B](https://huggingface.co/migtissera/Synthia-7B) on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard). The dataset is composed of 64 configuration, each one coresponding to one of the evaluated task. The dataset has been created from 2 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest results. An additional configuration "results" store all the aggregated results of the run (and is used to compute and display the agregated metrics on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)). To load the details from a run, you can for instance do the following: ```python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_migtissera__Synthia-7B", "harness_winogrande_5", split="train") ``` ## Latest results These are the [latest results from run 2023-10-15T06:07:54.738296](https://huggingface.co/datasets/open-llm-leaderboard/details_migtissera__Synthia-7B/blob/main/results_2023-10-15T06-07-54.738296.json)(note that their might be results for other tasks in the repos if successive evals didn't cover the same tasks. You find each in the results and the "latest" split for each eval): ```python { "all": { "em": 0.07151845637583892, "em_stderr": 0.00263897548039012, "f1": 0.14513737416107345, "f1_stderr": 0.0029452435334875074, "acc": 0.4043291747772373, "acc_stderr": 0.009561470405449964 }, "harness|drop|3": { "em": 0.07151845637583892, "em_stderr": 0.00263897548039012, "f1": 0.14513737416107345, "f1_stderr": 0.0029452435334875074 }, "harness|gsm8k|5": { "acc": 0.06595905989385899, "acc_stderr": 0.006836951192034222 }, "harness|winogrande|5": { "acc": 0.7426992896606156, "acc_stderr": 0.012285989618865708 } } ``` ### Supported Tasks and Leaderboards [More Information Needed] ### Languages [More Information Needed] ## Dataset Structure ### Data Instances [More Information Needed] ### Data Fields [More Information Needed] ### Data Splits [More Information Needed] ## Dataset Creation ### Curation Rationale [More Information Needed] ### Source Data #### Initial Data Collection and Normalization [More Information Needed] #### Who are the source language producers? [More Information Needed] ### Annotations #### Annotation process [More Information Needed] #### Who are the annotators? [More Information Needed] ### Personal and Sensitive Information [More Information Needed] ## Considerations for Using the Data ### Social Impact of Dataset [More Information Needed] ### Discussion of Biases [More Information Needed] ### Other Known Limitations [More Information Needed] ## Additional Information ### Dataset Curators [More Information Needed] ### Licensing Information [More Information Needed] ### Citation Information [More Information Needed] ### Contributions [More Information Needed]

原始信息汇总

数据集概述

数据集来源

该数据集是在对模型 migtissera/Synthia-7B 进行评估运行期间自动创建的，评估结果展示在 Open LLM Leaderboard 上。

数据集结构

• 数据集包含 64 个配置，每个配置对应一个评估任务。
• 数据集从 2 次运行中创建，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
• "train" 分割始终指向最新的结果。
• 一个额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示在 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_migtissera__Synthia-7B", "harness_winogrande_5", split="train")

最新结果

以下是 2023-10-15T06:07:54.738296 运行 的最新结果： python { "all": { "em": 0.07151845637583892, "em_stderr": 0.00263897548039012, "f1": 0.14513737416107345, "f1_stderr": 0.0029452435334875074, "acc": 0.4043291747772373, "acc_stderr": 0.009561470405449964 }, "harness|drop|3": { "em": 0.07151845637583892, "em_stderr": 0.00263897548039012, "f1": 0.14513737416107345, "f1_stderr": 0.0029452435334875074 }, "harness|gsm8k|5": { "acc": 0.06595905989385899, "acc_stderr": 0.006836951192034222 }, "harness|winogrande|5": { "acc": 0.7426992896606156, "acc_stderr": 0.012285989618865708 } }

配置详情

以下是部分配置及其数据文件路径：

• harness_arc_challenge_25

  • 分割: 2023_08_17T17_21_07.158534
    • 路径: **/details_harness|arc:challenge|25_2023-08-17T17:21:07.158534.parquet
  • 分割: latest
    • 路径: **/details_harness|arc:challenge|25_2023-08-17T17:21:07.158534.parquet

• harness_drop_3

  • 分割: 2023_10_15T06_07_54.738296
    • 路径: **/details_harness|drop|3_2023-10-15T06-07-54.738296.parquet
  • 分割: latest
    • 路径: **/details_harness|drop|3_2023-10-15T06-07-54.738296.parquet

• harness_gsm8k_5

  • 分割: 2023_10_15T06_07_54.738296
    • 路径: **/details_harness|gsm8k|5_2023-10-15T06-07-54.738296.parquet
  • 分割: latest
    • 路径: **/details_harness|gsm8k|5_2023-10-15T06-07-54.738296.parquet

• harness_hellaswag_10

  • 分割: 2023_08_17T17_21_07.158534
    • 路径: **/details_harness|hellaswag|10_2023-08-17T17:21:07.158534.parquet
  • 分割: latest
    • 路径: **/details_harness|hellaswag|10_2023-08-17T17:21:07.158534.parquet

• harness_hendrycksTest_5

  • 分割: 2023_08_17T17_21_07.158534
    • 路径:
      • **/details_harness|hendrycksTest-abstract_algebra|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-anatomy|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-astronomy|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-business_ethics|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-college_biology|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-college_chemistry|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-college_computer_science|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-college_mathematics|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-college_medicine|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-college_physics|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-computer_security|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-conceptual_physics|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-econometrics|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-electrical_engineering|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-formal_logic|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-global_facts|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-high_school_biology|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-high_school_european_history|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-high_school_geography|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-high_school_physics|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-high_school_psychology|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-high_school_statistics|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-high_school_us_history|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-high_school_world_history|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-human_aging|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-human_sexuality|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-international_law|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-jurisprudence|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-logical_fallacies|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-machine_learning|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-management|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-marketing|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-medical_genetics|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-miscellaneous|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-moral_disputes|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-moral_scenarios|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-nutrition|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-philosophy|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-prehistory|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-professional_accounting|5_2023-08-17T17:21:07.158534.parquet
      • **/details_harness|hendrycksTest-professional_law|5_2023-08-17T17:21:07.158534.parquet
      • **/details_har

搜集汇总

数据集介绍

构建方式

该数据集是在Open LLM Leaderboard框架下对migtissera/Synthia-7B模型进行自动化评估过程中生成的。数据集包含64个配置项，每个配置对应一个独立的评估任务，如ARC挑战、DROP、GSM8K、HellaSwag以及涵盖57个学科领域的HendrycksTest等。评估共执行了两次运行，每次运行的结果以时间戳命名的独立分割（split）形式存储于各配置中，其中“train”分割始终指向最新一次运行的结果。此外，数据集还设有专门的“results”配置，用于汇总并存储所有评估任务的聚合指标，这些指标直接用于Leaderboard上的可视化展示。数据以Parquet格式保存，确保高效存取与兼容性。

特点

该数据集的核心特色在于其结构化与动态更新机制。它完整记录了模型在数十个标准化基准任务上的细粒度表现，涵盖常识推理、数学计算、阅读理解及多学科知识等多个维度。每次评估运行均保留独立分割，便于研究者追溯模型性能的演变轨迹，而“train”分割自动指向最新结果则简化了持续追踪的流程。数据集通过“results”配置提供聚合指标，如准确率（acc）、F1分数和精确匹配率（em），并附带标准误差（stderr），为模型性能的统计显著性分析提供了可靠依据。这种设计兼顾了历史对比与实时更新的需求。

使用方法

研究者可通过HuggingFace Datasets库轻松加载该数据集。例如，调用load_dataset('open-llm-leaderboard/details_migtissera__Synthia-7B', 'harness_winogrande_5', split='train')即可获取Winogrande任务的最新评估详情。若要回溯特定历史运行，可将split参数替换为对应的时间戳标识，如'2023_10_15T06_07_54.738296'。对于聚合结果，可通过加载'results'配置来获取各任务的汇总指标。数据加载后，用户可利用其丰富的评估字段进行模型性能分析、任务难度比较或不同模型间的横向对比，为语言模型的研究与优化提供坚实的数据支撑。

背景与挑战

背景概述

在大型语言模型（LLM）性能评估领域，如何系统、可复现地度量模型在多样化任务上的表现，始终是推动技术迭代的核心议题。2023年，Hugging Face团队发起了Open LLM Leaderboard项目，旨在为开源社区提供一个标准化、透明的模型评测平台。该数据集正是围绕migtissera团队开发的Synthia-7B模型在Leaderboard上的评估过程而构建，记录了模型在ARC、DROP、GSM8K、HellaSwag、WinoGrande及MMLU等多项基准任务上的详细表现。由Clémentine Fourrier等人主导的这项工作，通过自动化流程生成包含64个配置的多任务评测结果，不仅为Synthia-7B模型的性能提供了量化基准，也推动了LLM评估方法的标准化与可复现性，对后续模型开发与比较研究产生了重要影响。

当前挑战

该数据集所应对的核心挑战源自LLM评测领域长期存在的两大难题：一是多任务评估的协调性与一致性，不同任务（如数学推理、常识理解、阅读问答）具有迥异的度量标准与难度层级，如何在统一框架下公平比较模型能力是一大挑战；二是评估过程的透明性与可复现性，传统评测常因随机种子、采样策略或提示模板的微小差异导致结果波动。在构建过程中，团队需解决数据版本管理与大规模并行运行的工程难题，例如确保每次评估的配置与时间戳准确对应，以及将来自不同运行（run）的异构结果整合为统一的“latest”快照，同时维护历史数据以供追溯分析。

常用场景

经典使用场景

在大型语言模型评估领域，open-llm-leaderboard/details_migtissera__Synthia-7B 数据集作为 Open LLM Leaderboard 的标准化评测产物，其核心用途在于系统性地记录与呈现 Synthia-7B 模型在多项基准任务上的细粒度表现。研究者可借助该数据集，通过加载不同配置（如 harness_winogrande_5、harness_gsm8k_5）对应的 Parquet 文件，复现模型在常识推理、数学问题求解、阅读理解等维度的精确得分与误差范围，从而实现对模型能力的多维度量化剖析。

衍生相关工作

基于该数据集所承载的评估记录，衍生出多项推动大模型评测领域发展的经典工作。例如，研究者利用其提供的细粒度结果，深入分析了 Synthia-7B 在 MMLU 子任务（如抽象代数、临床知识）上的表现差异，从而揭示了模型知识分布的不均衡性。同时，该数据集也为 Leaderboard 排名算法的改进提供了实证基础，促使社区开发出更鲁棒的聚合指标。此外，其分片存储的设计模式启发了后续评估数据集的构建规范，成为可复现评测流水线的参考范本。

数据集最近研究

最新研究方向

在大型语言模型迅猛发展的浪潮中，migtissera/Synthia-7B作为一款参数量为70亿的模型，其在Open LLM Leaderboard上的评估数据为理解中等规模开源模型的性能边界提供了关键参照。当前前沿研究方向聚焦于模型在多样化推理任务中的泛化能力与鲁棒性，该数据集通过涵盖ARC挑战、HellaSwag、GSM8K、Winogrande及MMLU等64项配置的细粒度评测，揭示了Synthia-7B在常识推理（Winogrande准确率74.27%）与数学解题（GSM8K准确率仅6.60%）之间的显著能力落差。这一发现与业界对模型推理深度与知识广度平衡的探讨紧密相连——尽管在DROP阅读理解任务上F1值仅14.51%，但其在57个学科组成的MMLU测试中展现的广博知识库，折射出当前开源模型在特定领域深耕与通识能力构建间的张力。该数据集通过标准化评估框架为社区提供了可复现的基准，其意义在于推动研究者关注模型在真实世界多步推理与事实性知识检索上的短板，进而引导数据增强策略与训练范式的革新，成为连接模型能力诊断与性能跃升的桥梁。