open-llm-leaderboard/details_CHIH-HUNG__llama-2-13b-dolphin_5w

--- pretty_name: Evaluation run of CHIH-HUNG/llama-2-13b-dolphin_5w dataset_summary: "Dataset automatically created during the evaluation run of model\ \ [CHIH-HUNG/llama-2-13b-dolphin_5w](https://huggingface.co/CHIH-HUNG/llama-2-13b-dolphin_5w)\ \ on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).\n\ \nThe dataset is composed of 64 configuration, each one coresponding to one of the\ \ evaluated task.\n\nThe dataset has been created from 2 run(s). Each run can be\ \ found as a specific split in each configuration, the split being named using the\ \ timestamp of the run.The \"train\" split is always pointing to the latest results.\n\ \nAn additional configuration \"results\" store all the aggregated results of the\ \ run (and is used to compute and display the agregated metrics on the [Open LLM\ \ Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)).\n\ \nTo load the details from a run, you can for instance do the following:\n```python\n\ from datasets import load_dataset\ndata = load_dataset(\"open-llm-leaderboard/details_CHIH-HUNG__llama-2-13b-dolphin_5w\"\ ,\n\t\"harness_winogrande_5\",\n\tsplit=\"train\")\n```\n\n## Latest results\n\n\ These are the [latest results from run 2023-10-28T08:56:45.584398](https://huggingface.co/datasets/open-llm-leaderboard/details_CHIH-HUNG__llama-2-13b-dolphin_5w/blob/main/results_2023-10-28T08-56-45.584398.json)(note\ \ that their might be results for other tasks in the repos if successive evals didn't\ \ cover the same tasks. You find each in the results and the \"latest\" split for\ \ each eval):\n\n```python\n{\n \"all\": {\n \"em\": 0.004614093959731544,\n\ \ \"em_stderr\": 0.0006940305886353427,\n \"f1\": 0.06955327181208026,\n\ \ \"f1_stderr\": 0.001514906421482751,\n \"acc\": 0.44587604036206907,\n\ \ \"acc_stderr\": 0.010329612453566641\n },\n \"harness|drop|3\": {\n\ \ \"em\": 0.004614093959731544,\n \"em_stderr\": 0.0006940305886353427,\n\ \ \"f1\": 0.06955327181208026,\n \"f1_stderr\": 0.001514906421482751\n\ \ },\n \"harness|gsm8k|5\": {\n \"acc\": 0.11827141774071266,\n \ \ \"acc_stderr\": 0.008895075852434951\n },\n \"harness|winogrande|5\"\ : {\n \"acc\": 0.7734806629834254,\n \"acc_stderr\": 0.011764149054698329\n\ \ }\n}\n```" repo_url: https://huggingface.co/CHIH-HUNG/llama-2-13b-dolphin_5w leaderboard_url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard point_of_contact: [e-mail] configs: - config_name: harness_arc_challenge_25 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|arc:challenge|25_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|arc:challenge|25_2023-08-29T20:17:39.109039.parquet' - config_name: harness_drop_3 data_files: - split: 2023_10_28T08_56_45.584398 path: - '**/details_harness|drop|3_2023-10-28T08-56-45.584398.parquet' - split: latest path: - '**/details_harness|drop|3_2023-10-28T08-56-45.584398.parquet' - config_name: harness_gsm8k_5 data_files: - split: 2023_10_28T08_56_45.584398 path: - '**/details_harness|gsm8k|5_2023-10-28T08-56-45.584398.parquet' - split: latest path: - '**/details_harness|gsm8k|5_2023-10-28T08-56-45.584398.parquet' - config_name: harness_hellaswag_10 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hellaswag|10_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hellaswag|10_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-international_law|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-management|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-marketing|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-sociology|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-virology|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-anatomy|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-astronomy|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-college_biology|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-college_physics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-computer_security|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-econometrics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-global_facts|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-human_aging|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-international_law|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-management|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-marketing|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-nutrition|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-philosophy|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-prehistory|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-professional_law|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-public_relations|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-security_studies|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-sociology|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-virology|5_2023-08-29T20:17:39.109039.parquet' - '**/details_harness|hendrycksTest-world_religions|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_abstract_algebra_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-abstract_algebra|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_anatomy_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-anatomy|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-anatomy|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_astronomy_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-astronomy|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-astronomy|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_business_ethics_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-business_ethics|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_clinical_knowledge_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_college_biology_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-college_biology|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_biology|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_college_chemistry_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_chemistry|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_college_computer_science_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_computer_science|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_college_mathematics_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_mathematics|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_college_medicine_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_medicine|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_college_physics_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-college_physics|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-college_physics|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_computer_security_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-computer_security|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-computer_security|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_conceptual_physics_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-conceptual_physics|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_econometrics_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-econometrics|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-econometrics|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_electrical_engineering_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-electrical_engineering|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_elementary_mathematics_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_formal_logic_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-formal_logic|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_global_facts_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-global_facts|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-global_facts|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_high_school_biology_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_biology|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_high_school_chemistry_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_high_school_computer_science_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_high_school_european_history_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_european_history|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_high_school_geography_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_geography|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_high_school_government_and_politics_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_high_school_macroeconomics_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_high_school_mathematics_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_high_school_microeconomics_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_high_school_physics_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_physics|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_high_school_psychology_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_psychology|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_high_school_statistics_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_statistics|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_high_school_us_history_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_us_history|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_high_school_world_history_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-high_school_world_history|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_human_aging_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-human_aging|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-human_aging|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_human_sexuality_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-human_sexuality|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_international_law_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-international_law|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-international_law|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_jurisprudence_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-jurisprudence|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_logical_fallacies_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-logical_fallacies|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_machine_learning_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-machine_learning|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_management_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-management|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-management|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_marketing_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-marketing|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-marketing|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_medical_genetics_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-medical_genetics|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_miscellaneous_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-miscellaneous|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_moral_disputes_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-moral_disputes|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_moral_scenarios_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-moral_scenarios|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_nutrition_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-nutrition|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-nutrition|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_philosophy_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-philosophy|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-philosophy|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_prehistory_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-prehistory|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-prehistory|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_professional_accounting_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_accounting|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_professional_law_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-professional_law|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_law|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_professional_medicine_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_medicine|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_professional_psychology_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-professional_psychology|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_public_relations_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-public_relations|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-public_relations|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_security_studies_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-security_studies|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-security_studies|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_sociology_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-sociology|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-sociology|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_us_foreign_policy_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-us_foreign_policy|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_virology_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-virology|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-virology|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_hendrycksTest_world_religions_5 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|hendrycksTest-world_religions|5_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|hendrycksTest-world_religions|5_2023-08-29T20:17:39.109039.parquet' - config_name: harness_truthfulqa_mc_0 data_files: - split: 2023_08_29T20_17_39.109039 path: - '**/details_harness|truthfulqa:mc|0_2023-08-29T20:17:39.109039.parquet' - split: latest path: - '**/details_harness|truthfulqa:mc|0_2023-08-29T20:17:39.109039.parquet' - config_name: harness_winogrande_5 data_files: - split: 2023_10_28T08_56_45.584398 path: - '**/details_harness|winogrande|5_2023-10-28T08-56-45.584398.parquet' - split: latest path: - '**/details_harness|winogrande|5_2023-10-28T08-56-45.584398.parquet' - config_name: results data_files: - split: 2023_08_29T20_17_39.109039 path: - results_2023-08-29T20:17:39.109039.parquet - split: 2023_10_28T08_56_45.584398 path: - results_2023-10-28T08-56-45.584398.parquet - split: latest path: - results_2023-10-28T08-56-45.584398.parquet --- # Dataset Card for Evaluation run of CHIH-HUNG/llama-2-13b-dolphin_5w ## Dataset Description - **Homepage:** - **Repository:** https://huggingface.co/CHIH-HUNG/llama-2-13b-dolphin_5w - **Paper:** - **Leaderboard:** https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard - **Point of Contact:** [e-mail] ### Dataset Summary Dataset automatically created during the evaluation run of model [CHIH-HUNG/llama-2-13b-dolphin_5w](https://huggingface.co/CHIH-HUNG/llama-2-13b-dolphin_5w) on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard). The dataset is composed of 64 configuration, each one coresponding to one of the evaluated task. The dataset has been created from 2 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest results. An additional configuration "results" store all the aggregated results of the run (and is used to compute and display the agregated metrics on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)). To load the details from a run, you can for instance do the following: ```python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_CHIH-HUNG__llama-2-13b-dolphin_5w", "harness_winogrande_5", split="train") ``` ## Latest results These are the [latest results from run 2023-10-28T08:56:45.584398](https://huggingface.co/datasets/open-llm-leaderboard/details_CHIH-HUNG__llama-2-13b-dolphin_5w/blob/main/results_2023-10-28T08-56-45.584398.json)(note that their might be results for other tasks in the repos if successive evals didn't cover the same tasks. You find each in the results and the "latest" split for each eval): ```python { "all": { "em": 0.004614093959731544, "em_stderr": 0.0006940305886353427, "f1": 0.06955327181208026, "f1_stderr": 0.001514906421482751, "acc": 0.44587604036206907, "acc_stderr": 0.010329612453566641 }, "harness|drop|3": { "em": 0.004614093959731544, "em_stderr": 0.0006940305886353427, "f1": 0.06955327181208026, "f1_stderr": 0.001514906421482751 }, "harness|gsm8k|5": { "acc": 0.11827141774071266, "acc_stderr": 0.008895075852434951 }, "harness|winogrande|5": { "acc": 0.7734806629834254, "acc_stderr": 0.011764149054698329 } } ``` ### Supported Tasks and Leaderboards [More Information Needed] ### Languages [More Information Needed] ## Dataset Structure ### Data Instances [More Information Needed] ### Data Fields [More Information Needed] ### Data Splits [More Information Needed] ## Dataset Creation ### Curation Rationale [More Information Needed] ### Source Data #### Initial Data Collection and Normalization [More Information Needed] #### Who are the source language producers? [More Information Needed] ### Annotations #### Annotation process [More Information Needed] #### Who are the annotators? [More Information Needed] ### Personal and Sensitive Information [More Information Needed] ## Considerations for Using the Data ### Social Impact of Dataset [More Information Needed] ### Discussion of Biases [More Information Needed] ### Other Known Limitations [More Information Needed] ## Additional Information ### Dataset Curators [More Information Needed] ### Licensing Information [More Information Needed] ### Citation Information [More Information Needed] ### Contributions [More Information Needed]

原始信息汇总

数据集概述

数据集简介

该数据集是在评估模型 CHIH-HUNG/llama-2-13b-dolphin_5w 在 Open LLM Leaderboard 上的自动创建的。

数据集结构

• 数据集包含 64 个配置，每个配置对应一个评估任务。
• 数据集从 2 次运行中创建，每次运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
• "train" 分割始终指向最新的结果。
• 额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示 Open LLM Leaderboard 上的聚合指标。

数据加载示例

python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_CHIH-HUNG__llama-2-13b-dolphin_5w", "harness_winogrande_5", split="train")

最新结果

以下是 2023-10-28T08:56:45.584398 运行的最新结果： python { "all": { "em": 0.004614093959731544, "em_stderr": 0.0006940305886353427, "f1": 0.06955327181208026, "f1_stderr": 0.001514906421482751, "acc": 0.44587604036206907, "acc_stderr": 0.010329612453566641 }, "harness|drop|3": { "em": 0.004614093959731544, "em_stderr": 0.0006940305886353427, "f1": 0.06955327181208026, "f1_stderr": 0.001514906421482751 }, "harness|gsm8k|5": { "acc": 0.11827141774071266, "acc_stderr": 0.008895075852434951 }, "harness|winogrande|5": { "acc": 0.7734806629834254, "acc_stderr": 0.011764149054698329 } }

配置详情

以下是数据集的配置详情：

• harness_arc_challenge_25

  • 分割：2023_08_29T20_17_39.109039
    • 路径：**/details_harness|arc:challenge|25_2023-08-29T20:17:39.109039.parquet
  • 分割：latest
    • 路径：**/details_harness|arc:challenge|25_2023-08-29T20:17:39.109039.parquet

• harness_drop_3

  • 分割：2023_10_28T08_56_45.584398
    • 路径：**/details_harness|drop|3_2023-10-28T08-56-45.584398.parquet
  • 分割：latest
    • 路径：**/details_harness|drop|3_2023-10-28T08-56-45.584398.parquet

• harness_gsm8k_5

  • 分割：2023_10_28T08_56_45.584398
    • 路径：**/details_harness|gsm8k|5_2023-10-28T08-56-45.584398.parquet
  • 分割：latest
    • 路径：**/details_harness|gsm8k|5_2023-10-28T08-56-45.584398.parquet

• harness_hellaswag_10

  • 分割：2023_08_29T20_17_39.109039
    • 路径：**/details_harness|hellaswag|10_2023-08-29T20:17:39.109039.parquet
  • 分割：latest
    • 路径：**/details_harness|hellaswag|10_2023-08-29T20:17:39.109039.parquet

• harness_hendrycksTest_5

  • 分割：2023_08_29T20_17_39.109039
    • 路径：
      • **/details_harness|hendrycksTest-abstract_algebra|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-anatomy|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-astronomy|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-business_ethics|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-clinical_knowledge|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-college_biology|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-college_chemistry|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-college_computer_science|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-college_mathematics|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-college_medicine|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-college_physics|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-computer_security|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-conceptual_physics|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-econometrics|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-electrical_engineering|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-elementary_mathematics|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-formal_logic|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-global_facts|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-high_school_biology|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-high_school_chemistry|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-high_school_computer_science|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-high_school_european_history|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-high_school_geography|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-high_school_government_and_politics|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-high_school_macroeconomics|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-high_school_mathematics|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-high_school_microeconomics|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-high_school_physics|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-high_school_psychology|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-high_school_statistics|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-high_school_us_history|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-high_school_world_history|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-human_aging|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-human_sexuality|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-international_law|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-jurisprudence|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-logical_fallacies|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-machine_learning|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-management|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-marketing|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-medical_genetics|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-miscellaneous|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-moral_disputes|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-moral_scenarios|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-nutrition|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-philosophy|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-prehistory|5_2023-08-29T20:17:39.109039.parquet
      • **/details_harness|hendrycksTest-professional_accounting|5_2023-08-29T20:17:39.109039.parquet
      • `**/details_harness|hendrycksTest-professional_law|5_2023-08-

搜集汇总

数据集介绍

构建方式

本数据集源于对CHIH-HUNG/llama-2-13b-dolphin_5w模型在Open LLM Leaderboard上的自动评测过程，旨在系统记录模型在多项基准任务中的表现细节。数据集构建基于两次独立的评测运行，每一次运行的结果都被归档为对应配置下的一个独立分割（split），分割名称以运行时间戳命名，而'train'分割则始终指向最新一次评测的结果。整个数据集包含64个配置，每个配置对应一项被评测的具体任务，例如harness_winogrande_5、harness_drop_3等，此外还设有专门的'results'配置，用于存储所有任务的聚合指标，以便在Leaderboard上计算和展示模型整体性能。

使用方法

使用该数据集时，研究者可通过HuggingFace的datasets库轻松加载所需任务的详细信息。具体而言，调用load_dataset函数并指定数据集名称、任务配置名称（如'harness_winogrande_5'）以及所需的分割（如'train'），即可获取该任务在最新评测中的完整结果。若需回溯历史表现，则可选择对应时间戳的分割名称进行加载。此外，'results'配置为研究者提供了直接访问所有任务聚合指标的便捷途径，便于快速进行模型间的横向比较与性能分析。

背景与挑战

背景概述

大规模语言模型的迅猛发展催生了对其性能进行系统化评估的迫切需求。在此背景下，Hugging Face团队于2023年推出了Open LLM Leaderboard，旨在为开源大语言模型提供公平、透明且可复现的评测基准。该数据集作为Leaderboard的一部分，专门记录了由CHIH-HUNG团队开发的llama-2-13b-dolphin_5w模型在一次完整评估运行中的详细结果。数据集由Clementine等人维护，涵盖ARC-Challenge、DROP、GSM8K、HellaSwag、WinoGrande及MMLU（Hendrycks Test）等多项经典任务，全面衡量模型在推理、常识、数学与知识理解等多维度的能力。通过结构化存储每次运行的时间戳与分片数据，该数据集不仅为模型开发者提供了精细化的性能反馈，也推动了社区对模型能力边界的深入理解，成为开源LLM评估领域的重要基础设施。

当前挑战

当前，该数据集所面临的核心挑战体现在两个层面。其一，在领域问题层面，大语言模型的评估仍缺乏对复杂推理与长文本理解的统一度量标准，现有任务如DROP和GSM8K虽能反映部分能力，但难以覆盖模型在真实场景中的鲁棒性与泛化表现，尤其面对对抗性样本或领域迁移时，评测指标的局限性愈发凸显。其二，在构建过程中，数据集依赖多次运行的时间戳分片管理，导致版本控制与结果复现的复杂性增加；同时，任务配置繁多（如MMLU涵盖57个子领域），数据文件的组织与同步需精细维护，稍有不慎便可能引入不一致性，为后续分析与横向比较带来挑战。

常用场景

经典使用场景

该数据集作为Open LLM Leaderboard评估流程的产物，其经典使用场景在于系统性地记录并复现大语言模型在多样化基准任务上的细粒度性能表现。研究者可借助该数据集，针对llama-2-13b-dolphin_5w模型在ARC挑战、DROP、GSM8K、HellaSwag、Winogrande及涵盖57个学科的MMLU等任务上的逐样本结果进行深入剖析，从而探究模型在不同推理难度与知识领域的泛化能力差异。

解决学术问题

在学术研究中，该数据集有效解决了大语言模型评估过程中结果碎片化与不可复现的痛点。通过标准化存储每次评估的原始响应与逐项得分，它使得研究者能够对模型在常识推理、数学计算、阅读理解和专业知识等维度的优劣进行精细归因分析，为理解模型能力边界、诊断失败案例以及对比不同训练策略（如微调与对齐）的边际收益提供了坚实的数据基础。

实际应用

实际应用中，该数据集为模型选型与迭代优化提供了量化依据。开发团队可依据其中记录的跨任务表现，快速判断llama-2-13b-dolphin_5w在需要逻辑推理（如Winogrande达77.3%准确率）或数学求解（如GSM8K为11.8%）等场景下的适用性。同时，其细粒度结果可用于构建自动化回归测试流水线，确保模型在迭代升级过程中不出现关键能力的退化。

数据集最近研究

最新研究方向

当前，大语言模型（LLM）的评估体系正经历从单一指标向多维度、细粒度任务泛化能力验证的深刻转型。该数据集作为Open LLM Leaderboard上对llama-2-13b-dolphin_5w模型进行自动化评估的产物，其核心价值在于为模型在ARC推理挑战、DROP文本理解、GSM8K数学推理、Winogrande常识推理及涵盖57个学科的MMLU等多样化基准上的表现提供了系统化的评测框架。前沿研究聚焦于利用此类高分辨率评估数据，深入剖析模型在零样本泛化与少样本学习场景下的能力边界，尤其关注其在复杂推理与知识密集型任务中的性能瓶颈。这一研究方向与当前业界对模型鲁棒性、公平性及实际部署效能的迫切需求紧密相连，该数据集通过标准化、可复现的评测流程，为社区甄别模型优劣、指导后续优化提供了关键依据，对推动大模型向更可靠、更通用方向演进具有里程碑式的意义。