open-llm-leaderboard-old/details_prince-canuma__Damysus-Coder-v0.1
收藏Hugging Face2024-04-15 更新2024-06-22 收录
下载链接：
https://hf-mirror.com/datasets/open-llm-leaderboard-old/details_prince-canuma__Damysus-Coder-v0.1
下载链接
链接失效反馈官方服务：
资源简介：
---
pretty_name: Evaluation run of prince-canuma/Damysus-Coder-v0.1
dataset_summary: "Dataset automatically created during the evaluation run of model\
\ [prince-canuma/Damysus-Coder-v0.1](https://huggingface.co/prince-canuma/Damysus-Coder-v0.1)\
\ on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).\n\
\nThe dataset is composed of 63 configuration, each one coresponding to one of the\
\ evaluated task.\n\nThe dataset has been created from 1 run(s). Each run can be\
\ found as a specific split in each configuration, the split being named using the\
\ timestamp of the run.The \"train\" split is always pointing to the latest results.\n\
\nAn additional configuration \"results\" store all the aggregated results of the\
\ run (and is used to compute and display the aggregated metrics on the [Open LLM\
\ Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)).\n\
\nTo load the details from a run, you can for instance do the following:\n```python\n\
from datasets import load_dataset\ndata = load_dataset(\"open-llm-leaderboard/details_prince-canuma__Damysus-Coder-v0.1\"\
,\n\t\"harness_winogrande_5\",\n\tsplit=\"train\")\n```\n\n## Latest results\n\n\
These are the [latest results from run 2024-04-15T21:07:57.637768](https://huggingface.co/datasets/open-llm-leaderboard/details_prince-canuma__Damysus-Coder-v0.1/blob/main/results_2024-04-15T21-07-57.637768.json)(note\
\ that their might be results for other tasks in the repos if successive evals didn't\
\ cover the same tasks. You find each in the results and the \"latest\" split for\
\ each eval):\n\n```python\n{\n \"all\": {\n \"acc\": 0.6043961155404135,\n\
\ \"acc_stderr\": 0.033186632585662346,\n \"acc_norm\": 0.6094482265591639,\n\
\ \"acc_norm_stderr\": 0.033859874632835685,\n \"mc1\": 0.4724602203182375,\n\
\ \"mc1_stderr\": 0.017476930190712187,\n \"mc2\": 0.6419919334749323,\n\
\ \"mc2_stderr\": 0.01518622081933932\n },\n \"harness|arc:challenge|25\"\
: {\n \"acc\": 0.5554607508532423,\n \"acc_stderr\": 0.01452122640562708,\n\
\ \"acc_norm\": 0.6092150170648464,\n \"acc_norm_stderr\": 0.014258563880513782\n\
\ },\n \"harness|hellaswag|10\": {\n \"acc\": 0.64070902210715,\n \
\ \"acc_stderr\": 0.004788120727316245,\n \"acc_norm\": 0.840071698864768,\n\
\ \"acc_norm_stderr\": 0.0036579044379436557\n },\n \"harness|hendrycksTest-abstract_algebra|5\"\
: {\n \"acc\": 0.3,\n \"acc_stderr\": 0.046056618647183814,\n \
\ \"acc_norm\": 0.3,\n \"acc_norm_stderr\": 0.046056618647183814\n \
\ },\n \"harness|hendrycksTest-anatomy|5\": {\n \"acc\": 0.5703703703703704,\n\
\ \"acc_stderr\": 0.042763494943765995,\n \"acc_norm\": 0.5703703703703704,\n\
\ \"acc_norm_stderr\": 0.042763494943765995\n },\n \"harness|hendrycksTest-astronomy|5\"\
: {\n \"acc\": 0.6118421052631579,\n \"acc_stderr\": 0.03965842097512744,\n\
\ \"acc_norm\": 0.6118421052631579,\n \"acc_norm_stderr\": 0.03965842097512744\n\
\ },\n \"harness|hendrycksTest-business_ethics|5\": {\n \"acc\": 0.57,\n\
\ \"acc_stderr\": 0.049756985195624284,\n \"acc_norm\": 0.57,\n \
\ \"acc_norm_stderr\": 0.049756985195624284\n },\n \"harness|hendrycksTest-clinical_knowledge|5\"\
: {\n \"acc\": 0.690566037735849,\n \"acc_stderr\": 0.028450154794118637,\n\
\ \"acc_norm\": 0.690566037735849,\n \"acc_norm_stderr\": 0.028450154794118637\n\
\ },\n \"harness|hendrycksTest-college_biology|5\": {\n \"acc\": 0.6527777777777778,\n\
\ \"acc_stderr\": 0.039812405437178615,\n \"acc_norm\": 0.6527777777777778,\n\
\ \"acc_norm_stderr\": 0.039812405437178615\n },\n \"harness|hendrycksTest-college_chemistry|5\"\
: {\n \"acc\": 0.37,\n \"acc_stderr\": 0.04852365870939099,\n \
\ \"acc_norm\": 0.37,\n \"acc_norm_stderr\": 0.04852365870939099\n \
\ },\n \"harness|hendrycksTest-college_computer_science|5\": {\n \"acc\"\
: 0.49,\n \"acc_stderr\": 0.05024183937956913,\n \"acc_norm\": 0.49,\n\
\ \"acc_norm_stderr\": 0.05024183937956913\n },\n \"harness|hendrycksTest-college_mathematics|5\"\
: {\n \"acc\": 0.36,\n \"acc_stderr\": 0.04824181513244218,\n \
\ \"acc_norm\": 0.36,\n \"acc_norm_stderr\": 0.04824181513244218\n \
\ },\n \"harness|hendrycksTest-college_medicine|5\": {\n \"acc\": 0.5780346820809249,\n\
\ \"acc_stderr\": 0.0376574669386515,\n \"acc_norm\": 0.5780346820809249,\n\
\ \"acc_norm_stderr\": 0.0376574669386515\n },\n \"harness|hendrycksTest-college_physics|5\"\
: {\n \"acc\": 0.46078431372549017,\n \"acc_stderr\": 0.049598599663841815,\n\
\ \"acc_norm\": 0.46078431372549017,\n \"acc_norm_stderr\": 0.049598599663841815\n\
\ },\n \"harness|hendrycksTest-computer_security|5\": {\n \"acc\":\
\ 0.74,\n \"acc_stderr\": 0.04408440022768078,\n \"acc_norm\": 0.74,\n\
\ \"acc_norm_stderr\": 0.04408440022768078\n },\n \"harness|hendrycksTest-conceptual_physics|5\"\
: {\n \"acc\": 0.5191489361702127,\n \"acc_stderr\": 0.03266204299064678,\n\
\ \"acc_norm\": 0.5191489361702127,\n \"acc_norm_stderr\": 0.03266204299064678\n\
\ },\n \"harness|hendrycksTest-econometrics|5\": {\n \"acc\": 0.40350877192982454,\n\
\ \"acc_stderr\": 0.046151869625837026,\n \"acc_norm\": 0.40350877192982454,\n\
\ \"acc_norm_stderr\": 0.046151869625837026\n },\n \"harness|hendrycksTest-electrical_engineering|5\"\
: {\n \"acc\": 0.5793103448275863,\n \"acc_stderr\": 0.0411391498118926,\n\
\ \"acc_norm\": 0.5793103448275863,\n \"acc_norm_stderr\": 0.0411391498118926\n\
\ },\n \"harness|hendrycksTest-elementary_mathematics|5\": {\n \"acc\"\
: 0.3835978835978836,\n \"acc_stderr\": 0.025043757318520193,\n \"\
acc_norm\": 0.3835978835978836,\n \"acc_norm_stderr\": 0.025043757318520193\n\
\ },\n \"harness|hendrycksTest-formal_logic|5\": {\n \"acc\": 0.36507936507936506,\n\
\ \"acc_stderr\": 0.04306241259127153,\n \"acc_norm\": 0.36507936507936506,\n\
\ \"acc_norm_stderr\": 0.04306241259127153\n },\n \"harness|hendrycksTest-global_facts|5\"\
: {\n \"acc\": 0.4,\n \"acc_stderr\": 0.049236596391733084,\n \
\ \"acc_norm\": 0.4,\n \"acc_norm_stderr\": 0.049236596391733084\n \
\ },\n \"harness|hendrycksTest-high_school_biology|5\": {\n \"acc\": 0.6967741935483871,\n\
\ \"acc_stderr\": 0.02614868593067175,\n \"acc_norm\": 0.6967741935483871,\n\
\ \"acc_norm_stderr\": 0.02614868593067175\n },\n \"harness|hendrycksTest-high_school_chemistry|5\"\
: {\n \"acc\": 0.5073891625615764,\n \"acc_stderr\": 0.0351760354036101,\n\
\ \"acc_norm\": 0.5073891625615764,\n \"acc_norm_stderr\": 0.0351760354036101\n\
\ },\n \"harness|hendrycksTest-high_school_computer_science|5\": {\n \
\ \"acc\": 0.63,\n \"acc_stderr\": 0.04852365870939099,\n \"acc_norm\"\
: 0.63,\n \"acc_norm_stderr\": 0.04852365870939099\n },\n \"harness|hendrycksTest-high_school_european_history|5\"\
: {\n \"acc\": 0.7333333333333333,\n \"acc_stderr\": 0.03453131801885417,\n\
\ \"acc_norm\": 0.7333333333333333,\n \"acc_norm_stderr\": 0.03453131801885417\n\
\ },\n \"harness|hendrycksTest-high_school_geography|5\": {\n \"acc\"\
: 0.7424242424242424,\n \"acc_stderr\": 0.03115626951964683,\n \"\
acc_norm\": 0.7424242424242424,\n \"acc_norm_stderr\": 0.03115626951964683\n\
\ },\n \"harness|hendrycksTest-high_school_government_and_politics|5\": {\n\
\ \"acc\": 0.8549222797927462,\n \"acc_stderr\": 0.02541634309630644,\n\
\ \"acc_norm\": 0.8549222797927462,\n \"acc_norm_stderr\": 0.02541634309630644\n\
\ },\n \"harness|hendrycksTest-high_school_macroeconomics|5\": {\n \
\ \"acc\": 0.5615384615384615,\n \"acc_stderr\": 0.02515826601686858,\n \
\ \"acc_norm\": 0.5615384615384615,\n \"acc_norm_stderr\": 0.02515826601686858\n\
\ },\n \"harness|hendrycksTest-high_school_mathematics|5\": {\n \"\
acc\": 0.34444444444444444,\n \"acc_stderr\": 0.028972648884844267,\n \
\ \"acc_norm\": 0.34444444444444444,\n \"acc_norm_stderr\": 0.028972648884844267\n\
\ },\n \"harness|hendrycksTest-high_school_microeconomics|5\": {\n \
\ \"acc\": 0.6302521008403361,\n \"acc_stderr\": 0.03135709599613591,\n \
\ \"acc_norm\": 0.6302521008403361,\n \"acc_norm_stderr\": 0.03135709599613591\n\
\ },\n \"harness|hendrycksTest-high_school_physics|5\": {\n \"acc\"\
: 0.3509933774834437,\n \"acc_stderr\": 0.038969819642573754,\n \"\
acc_norm\": 0.3509933774834437,\n \"acc_norm_stderr\": 0.038969819642573754\n\
\ },\n \"harness|hendrycksTest-high_school_psychology|5\": {\n \"acc\"\
: 0.8,\n \"acc_stderr\": 0.01714985851425095,\n \"acc_norm\": 0.8,\n\
\ \"acc_norm_stderr\": 0.01714985851425095\n },\n \"harness|hendrycksTest-high_school_statistics|5\"\
: {\n \"acc\": 0.4444444444444444,\n \"acc_stderr\": 0.03388857118502326,\n\
\ \"acc_norm\": 0.4444444444444444,\n \"acc_norm_stderr\": 0.03388857118502326\n\
\ },\n \"harness|hendrycksTest-high_school_us_history|5\": {\n \"acc\"\
: 0.7745098039215687,\n \"acc_stderr\": 0.02933116229425174,\n \"\
acc_norm\": 0.7745098039215687,\n \"acc_norm_stderr\": 0.02933116229425174\n\
\ },\n \"harness|hendrycksTest-high_school_world_history|5\": {\n \"\
acc\": 0.7510548523206751,\n \"acc_stderr\": 0.028146970599422644,\n \
\ \"acc_norm\": 0.7510548523206751,\n \"acc_norm_stderr\": 0.028146970599422644\n\
\ },\n \"harness|hendrycksTest-human_aging|5\": {\n \"acc\": 0.6367713004484304,\n\
\ \"acc_stderr\": 0.032277904428505,\n \"acc_norm\": 0.6367713004484304,\n\
\ \"acc_norm_stderr\": 0.032277904428505\n },\n \"harness|hendrycksTest-human_sexuality|5\"\
: {\n \"acc\": 0.7022900763358778,\n \"acc_stderr\": 0.040103589424622034,\n\
\ \"acc_norm\": 0.7022900763358778,\n \"acc_norm_stderr\": 0.040103589424622034\n\
\ },\n \"harness|hendrycksTest-international_law|5\": {\n \"acc\":\
\ 0.8099173553719008,\n \"acc_stderr\": 0.03581796951709282,\n \"\
acc_norm\": 0.8099173553719008,\n \"acc_norm_stderr\": 0.03581796951709282\n\
\ },\n \"harness|hendrycksTest-jurisprudence|5\": {\n \"acc\": 0.6851851851851852,\n\
\ \"acc_stderr\": 0.04489931073591312,\n \"acc_norm\": 0.6851851851851852,\n\
\ \"acc_norm_stderr\": 0.04489931073591312\n },\n \"harness|hendrycksTest-logical_fallacies|5\"\
: {\n \"acc\": 0.7300613496932515,\n \"acc_stderr\": 0.03487825168497892,\n\
\ \"acc_norm\": 0.7300613496932515,\n \"acc_norm_stderr\": 0.03487825168497892\n\
\ },\n \"harness|hendrycksTest-machine_learning|5\": {\n \"acc\": 0.48214285714285715,\n\
\ \"acc_stderr\": 0.047427623612430116,\n \"acc_norm\": 0.48214285714285715,\n\
\ \"acc_norm_stderr\": 0.047427623612430116\n },\n \"harness|hendrycksTest-management|5\"\
: {\n \"acc\": 0.7475728155339806,\n \"acc_stderr\": 0.04301250399690878,\n\
\ \"acc_norm\": 0.7475728155339806,\n \"acc_norm_stderr\": 0.04301250399690878\n\
\ },\n \"harness|hendrycksTest-marketing|5\": {\n \"acc\": 0.8717948717948718,\n\
\ \"acc_stderr\": 0.02190190511507332,\n \"acc_norm\": 0.8717948717948718,\n\
\ \"acc_norm_stderr\": 0.02190190511507332\n },\n \"harness|hendrycksTest-medical_genetics|5\"\
: {\n \"acc\": 0.68,\n \"acc_stderr\": 0.046882617226215034,\n \
\ \"acc_norm\": 0.68,\n \"acc_norm_stderr\": 0.046882617226215034\n \
\ },\n \"harness|hendrycksTest-miscellaneous|5\": {\n \"acc\": 0.7739463601532567,\n\
\ \"acc_stderr\": 0.014957458504335842,\n \"acc_norm\": 0.7739463601532567,\n\
\ \"acc_norm_stderr\": 0.014957458504335842\n },\n \"harness|hendrycksTest-moral_disputes|5\"\
: {\n \"acc\": 0.6791907514450867,\n \"acc_stderr\": 0.025131000233647886,\n\
\ \"acc_norm\": 0.6791907514450867,\n \"acc_norm_stderr\": 0.025131000233647886\n\
\ },\n \"harness|hendrycksTest-moral_scenarios|5\": {\n \"acc\": 0.3396648044692737,\n\
\ \"acc_stderr\": 0.015839400406212494,\n \"acc_norm\": 0.3396648044692737,\n\
\ \"acc_norm_stderr\": 0.015839400406212494\n },\n \"harness|hendrycksTest-nutrition|5\"\
: {\n \"acc\": 0.7058823529411765,\n \"acc_stderr\": 0.026090162504279056,\n\
\ \"acc_norm\": 0.7058823529411765,\n \"acc_norm_stderr\": 0.026090162504279056\n\
\ },\n \"harness|hendrycksTest-philosophy|5\": {\n \"acc\": 0.6881028938906752,\n\
\ \"acc_stderr\": 0.02631185807185416,\n \"acc_norm\": 0.6881028938906752,\n\
\ \"acc_norm_stderr\": 0.02631185807185416\n },\n \"harness|hendrycksTest-prehistory|5\"\
: {\n \"acc\": 0.6697530864197531,\n \"acc_stderr\": 0.026168298456732846,\n\
\ \"acc_norm\": 0.6697530864197531,\n \"acc_norm_stderr\": 0.026168298456732846\n\
\ },\n \"harness|hendrycksTest-professional_accounting|5\": {\n \"\
acc\": 0.4574468085106383,\n \"acc_stderr\": 0.029719281272236844,\n \
\ \"acc_norm\": 0.4574468085106383,\n \"acc_norm_stderr\": 0.029719281272236844\n\
\ },\n \"harness|hendrycksTest-professional_law|5\": {\n \"acc\": 0.42633637548891784,\n\
\ \"acc_stderr\": 0.012630884771599698,\n \"acc_norm\": 0.42633637548891784,\n\
\ \"acc_norm_stderr\": 0.012630884771599698\n },\n \"harness|hendrycksTest-professional_medicine|5\"\
: {\n \"acc\": 0.6029411764705882,\n \"acc_stderr\": 0.02972215209928006,\n\
\ \"acc_norm\": 0.6029411764705882,\n \"acc_norm_stderr\": 0.02972215209928006\n\
\ },\n \"harness|hendrycksTest-professional_psychology|5\": {\n \"\
acc\": 0.6143790849673203,\n \"acc_stderr\": 0.019691459052354025,\n \
\ \"acc_norm\": 0.6143790849673203,\n \"acc_norm_stderr\": 0.019691459052354025\n\
\ },\n \"harness|hendrycksTest-public_relations|5\": {\n \"acc\": 0.7,\n\
\ \"acc_stderr\": 0.04389311454644287,\n \"acc_norm\": 0.7,\n \
\ \"acc_norm_stderr\": 0.04389311454644287\n },\n \"harness|hendrycksTest-security_studies|5\"\
: {\n \"acc\": 0.7387755102040816,\n \"acc_stderr\": 0.028123429335142783,\n\
\ \"acc_norm\": 0.7387755102040816,\n \"acc_norm_stderr\": 0.028123429335142783\n\
\ },\n \"harness|hendrycksTest-sociology|5\": {\n \"acc\": 0.746268656716418,\n\
\ \"acc_stderr\": 0.030769444967296018,\n \"acc_norm\": 0.746268656716418,\n\
\ \"acc_norm_stderr\": 0.030769444967296018\n },\n \"harness|hendrycksTest-us_foreign_policy|5\"\
: {\n \"acc\": 0.81,\n \"acc_stderr\": 0.039427724440366255,\n \
\ \"acc_norm\": 0.81,\n \"acc_norm_stderr\": 0.039427724440366255\n \
\ },\n \"harness|hendrycksTest-virology|5\": {\n \"acc\": 0.5180722891566265,\n\
\ \"acc_stderr\": 0.03889951252827216,\n \"acc_norm\": 0.5180722891566265,\n\
\ \"acc_norm_stderr\": 0.03889951252827216\n },\n \"harness|hendrycksTest-world_religions|5\"\
: {\n \"acc\": 0.8245614035087719,\n \"acc_stderr\": 0.029170885500727665,\n\
\ \"acc_norm\": 0.8245614035087719,\n \"acc_norm_stderr\": 0.029170885500727665\n\
\ },\n \"harness|truthfulqa:mc|0\": {\n \"mc1\": 0.4724602203182375,\n\
\ \"mc1_stderr\": 0.017476930190712187,\n \"mc2\": 0.6419919334749323,\n\
\ \"mc2_stderr\": 0.01518622081933932\n },\n \"harness|winogrande|5\"\
: {\n \"acc\": 0.771112865035517,\n \"acc_stderr\": 0.011807360224025395\n\
\ },\n \"harness|gsm8k|5\": {\n \"acc\": 0.39272175890826383,\n \
\ \"acc_stderr\": 0.01345174534958657\n }\n}\n```"
repo_url: https://huggingface.co/prince-canuma/Damysus-Coder-v0.1
leaderboard_url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
point_of_contact: clementine@hf.co
configs:
- config_name: harness_arc_challenge_25
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|arc:challenge|25_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|arc:challenge|25_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_gsm8k_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|gsm8k|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|gsm8k|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hellaswag_10
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hellaswag|10_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hellaswag|10_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-abstract_algebra|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-anatomy|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-astronomy|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-business_ethics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-clinical_knowledge|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-college_biology|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-college_chemistry|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-college_computer_science|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-college_mathematics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-college_medicine|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-college_physics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-computer_security|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-conceptual_physics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-econometrics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-electrical_engineering|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-elementary_mathematics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-formal_logic|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-global_facts|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_biology|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_chemistry|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_computer_science|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_european_history|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_geography|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_mathematics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_microeconomics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_physics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_psychology|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_statistics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_us_history|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_world_history|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-human_aging|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-human_sexuality|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-international_law|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-jurisprudence|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-logical_fallacies|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-machine_learning|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-management|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-marketing|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-medical_genetics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-miscellaneous|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-moral_disputes|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-moral_scenarios|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-nutrition|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-philosophy|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-prehistory|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-professional_accounting|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-professional_law|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-professional_medicine|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-professional_psychology|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-public_relations|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-security_studies|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-sociology|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-us_foreign_policy|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-virology|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-world_religions|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-abstract_algebra|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-anatomy|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-astronomy|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-business_ethics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-clinical_knowledge|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-college_biology|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-college_chemistry|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-college_computer_science|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-college_mathematics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-college_medicine|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-college_physics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-computer_security|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-conceptual_physics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-econometrics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-electrical_engineering|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-elementary_mathematics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-formal_logic|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-global_facts|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_biology|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_chemistry|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_computer_science|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_european_history|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_geography|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_mathematics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_microeconomics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_physics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_psychology|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_statistics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_us_history|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-high_school_world_history|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-human_aging|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-human_sexuality|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-international_law|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-jurisprudence|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-logical_fallacies|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-machine_learning|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-management|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-marketing|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-medical_genetics|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-miscellaneous|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-moral_disputes|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-moral_scenarios|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-nutrition|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-philosophy|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-prehistory|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-professional_accounting|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-professional_law|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-professional_medicine|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-professional_psychology|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-public_relations|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-security_studies|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-sociology|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-us_foreign_policy|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-virology|5_2024-04-15T21-07-57.637768.parquet'
- '**/details_harness|hendrycksTest-world_religions|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_abstract_algebra_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-abstract_algebra|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-abstract_algebra|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_anatomy_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-anatomy|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-anatomy|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_astronomy_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-astronomy|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-astronomy|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_business_ethics_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-business_ethics|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-business_ethics|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_clinical_knowledge_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-clinical_knowledge|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-clinical_knowledge|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_college_biology_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-college_biology|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_biology|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_college_chemistry_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-college_chemistry|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_chemistry|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_college_computer_science_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-college_computer_science|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_computer_science|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_college_mathematics_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-college_mathematics|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_mathematics|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_college_medicine_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-college_medicine|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_medicine|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_college_physics_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-college_physics|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-college_physics|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_computer_security_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-computer_security|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-computer_security|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_conceptual_physics_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-conceptual_physics|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-conceptual_physics|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_econometrics_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-econometrics|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-econometrics|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_electrical_engineering_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-electrical_engineering|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-electrical_engineering|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_elementary_mathematics_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-elementary_mathematics|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-elementary_mathematics|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_formal_logic_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-formal_logic|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-formal_logic|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_global_facts_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-global_facts|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-global_facts|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_high_school_biology_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-high_school_biology|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_biology|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_high_school_chemistry_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-high_school_chemistry|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_chemistry|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_high_school_computer_science_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-high_school_computer_science|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_computer_science|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_high_school_european_history_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-high_school_european_history|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_european_history|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_high_school_geography_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-high_school_geography|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_geography|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_high_school_government_and_politics_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_government_and_politics|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_high_school_macroeconomics_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_macroeconomics|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_high_school_mathematics_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-high_school_mathematics|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_mathematics|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_high_school_microeconomics_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-high_school_microeconomics|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_microeconomics|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_high_school_physics_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-high_school_physics|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_physics|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_high_school_psychology_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-high_school_psychology|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_psychology|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_high_school_statistics_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-high_school_statistics|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_statistics|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_high_school_us_history_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-high_school_us_history|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_us_history|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_high_school_world_history_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-high_school_world_history|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-high_school_world_history|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_human_aging_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-human_aging|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-human_aging|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_human_sexuality_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-human_sexuality|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-human_sexuality|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_international_law_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-international_law|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-international_law|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_jurisprudence_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-jurisprudence|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-jurisprudence|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_logical_fallacies_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-logical_fallacies|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-logical_fallacies|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_machine_learning_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-machine_learning|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-machine_learning|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_management_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-management|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-management|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_marketing_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-marketing|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-marketing|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_medical_genetics_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-medical_genetics|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-medical_genetics|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_miscellaneous_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-miscellaneous|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-miscellaneous|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_moral_disputes_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-moral_disputes|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-moral_disputes|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_moral_scenarios_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-moral_scenarios|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-moral_scenarios|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_nutrition_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-nutrition|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-nutrition|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_philosophy_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-philosophy|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-philosophy|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_prehistory_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-prehistory|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-prehistory|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_professional_accounting_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-professional_accounting|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-professional_accounting|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_professional_law_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-professional_law|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-professional_law|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_professional_medicine_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-professional_medicine|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-professional_medicine|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_professional_psychology_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-professional_psychology|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-professional_psychology|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_public_relations_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-public_relations|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-public_relations|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_security_studies_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-security_studies|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-security_studies|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_sociology_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-sociology|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-sociology|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_us_foreign_policy_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-us_foreign_policy|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-us_foreign_policy|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_virology_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-virology|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-virology|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_hendrycksTest_world_religions_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|hendrycksTest-world_religions|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|hendrycksTest-world_religions|5_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_truthfulqa_mc_0
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|truthfulqa:mc|0_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|truthfulqa:mc|0_2024-04-15T21-07-57.637768.parquet'
- config_name: harness_winogrande_5
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- '**/details_harness|winogrande|5_2024-04-15T21-07-57.637768.parquet'
- split: latest
path:
- '**/details_harness|winogrande|5_2024-04-15T21-07-57.637768.parquet'
- config_name: results
data_files:
- split: 2024_04_15T21_07_57.637768
path:
- results_2024-04-15T21-07-57.637768.parquet
- split: latest
path:
- results_2024-04-15T21-07-57.637768.parquet
---
# Dataset Card for Evaluation run of prince-canuma/Damysus-Coder-v0.1
<!-- Provide a quick summary of the dataset. -->
Dataset automatically created during the evaluation run of model [prince-canuma/Damysus-Coder-v0.1](https://huggingface.co/prince-canuma/Damysus-Coder-v0.1) on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard).
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always pointing to the latest results.
An additional configuration "results" store all the aggregated results of the run (and is used to compute and display the aggregated metrics on the [Open LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)).
To load the details from a run, you can for instance do the following:
```python
from datasets import load_dataset
data = load_dataset("open-llm-leaderboard/details_prince-canuma__Damysus-Coder-v0.1",
"harness_winogrande_5",
split="train")
```
## Latest results
These are the [latest results from run 2024-04-15T21:07:57.637768](https://huggingface.co/datasets/open-llm-leaderboard/details_prince-canuma__Damysus-Coder-v0.1/blob/main/results_2024-04-15T21-07-57.637768.json)(note that their might be results for other tasks in the repos if successive evals didn't cover the same tasks. You find each in the results and the "latest" split for each eval):
```python
{
"all": {
"acc": 0.6043961155404135,
"acc_stderr": 0.033186632585662346,
"acc_norm": 0.6094482265591639,
"acc_norm_stderr": 0.033859874632835685,
"mc1": 0.4724602203182375,
"mc1_stderr": 0.017476930190712187,
"mc2": 0.6419919334749323,
"mc2_stderr": 0.01518622081933932
},
"harness|arc:challenge|25": {
"acc": 0.5554607508532423,
"acc_stderr": 0.01452122640562708,
"acc_norm": 0.6092150170648464,
"acc_norm_stderr": 0.014258563880513782
},
"harness|hellaswag|10": {
"acc": 0.64070902210715,
"acc_stderr": 0.004788120727316245,
"acc_norm": 0.840071698864768,
"acc_norm_stderr": 0.0036579044379436557
},
"harness|hendrycksTest-abstract_algebra|5": {
"acc": 0.3,
"acc_stderr": 0.046056618647183814,
"acc_norm": 0.3,
"acc_norm_stderr": 0.046056618647183814
},
"harness|hendrycksTest-anatomy|5": {
"acc": 0.5703703703703704,
"acc_stderr": 0.042763494943765995,
"acc_norm": 0.5703703703703704,
"acc_norm_stderr": 0.042763494943765995
},
"harness|hendrycksTest-astronomy|5": {
"acc": 0.6118421052631579,
"acc_stderr": 0.03965842097512744,
"acc_norm": 0.6118421052631579,
"acc_norm_stderr": 0.03965842097512744
},
"harness|hendrycksTest-business_ethics|5": {
"acc": 0.57,
"acc_stderr": 0.049756985195624284,
"acc_norm": 0.57,
"acc_norm_stderr": 0.049756985195624284
},
"harness|hendrycksTest-clinical_knowledge|5": {
"acc": 0.690566037735849,
"acc_stderr": 0.028450154794118637,
"acc_norm": 0.690566037735849,
"acc_norm_stderr": 0.028450154794118637
},
"harness|hendrycksTest-college_biology|5": {
"acc": 0.6527777777777778,
"acc_stderr": 0.039812405437178615,
"acc_norm": 0.6527777777777778,
"acc_norm_stderr": 0.039812405437178615
},
"harness|hendrycksTest-college_chemistry|5": {
"acc": 0.37,
"acc_stderr": 0.04852365870939099,
"acc_norm": 0.37,
"acc_norm_stderr": 0.04852365870939099
},
"harness|hendrycksTest-college_computer_science|5": {
"acc": 0.49,
"acc_stderr": 0.05024183937956913,
"acc_norm": 0.49,
"acc_norm_stderr": 0.05024183937956913
},
"harness|hendrycksTest-college_mathematics|5": {
"acc": 0.36,
"acc_stderr": 0.04824181513244218,
"acc_norm": 0.36,
"acc_norm_stderr": 0.04824181513244218
},
"harness|hendrycksTest-college_medicine|5": {
"acc": 0.5780346820809249,
"acc_stderr": 0.0376574669386515,
"acc_norm": 0.5780346820809249,
"acc_norm_stderr": 0.0376574669386515
},
"harness|hendrycksTest-college_physics|5": {
"acc": 0.46078431372549017,
"acc_stderr": 0.049598599663841815,
"acc_norm": 0.46078431372549017,
"acc_norm_stderr": 0.049598599663841815
},
"harness|hendrycksTest-computer_security|5": {
"acc": 0.74,
"acc_stderr": 0.04408440022768078,
"acc_norm": 0.74,
"acc_norm_stderr": 0.04408440022768078
},
"harness|hendrycksTest-conceptual_physics|5": {
"acc": 0.5191489361702127,
"acc_stderr": 0.03266204299064678,
"acc_norm": 0.5191489361702127,
"acc_norm_stderr": 0.03266204299064678
},
"harness|hendrycksTest-econometrics|5": {
"acc": 0.40350877192982454,
"acc_stderr": 0.046151869625837026,
"acc_norm": 0.40350877192982454,
"acc_norm_stderr": 0.046151869625837026
},
"harness|hendrycksTest-electrical_engineering|5": {
"acc": 0.5793103448275863,
"acc_stderr": 0.0411391498118926,
"acc_norm": 0.5793103448275863,
"acc_norm_stderr": 0.0411391498118926
},
"harness|hendrycksTest-elementary_mathematics|5": {
"acc": 0.3835978835978836,
"acc_stderr": 0.025043757318520193,
"acc_norm": 0.3835978835978836,
"acc_norm_stderr": 0.025043757318520193
},
"harness|hendrycksTest-formal_logic|5": {
"acc": 0.36507936507936506,
"acc_stderr": 0.04306241259127153,
"acc_norm": 0.36507936507936506,
"acc_norm_stderr": 0.04306241259127153
},
"harness|hendrycksTest-global_facts|5": {
"acc": 0.4,
"acc_stderr": 0.049236596391733084,
"acc_norm": 0.4,
"acc_norm_stderr": 0.049236596391733084
},
"harness|hendrycksTest-high_school_biology|5": {
"acc": 0.6967741935483871,
"acc_stderr": 0.02614868593067175,
"acc_norm": 0.6967741935483871,
"acc_norm_stderr": 0.02614868593067175
},
"harness|hendrycksTest-high_school_chemistry|5": {
"acc": 0.5073891625615764,
"acc_stderr": 0.0351760354036101,
"acc_norm": 0.5073891625615764,
"acc_norm_stderr": 0.0351760354036101
},
"harness|hendrycksTest-high_school_computer_science|5": {
"acc": 0.63,
"acc_stderr": 0.04852365870939099,
"acc_norm": 0.63,
"acc_norm_stderr": 0.04852365870939099
},
"harness|hendrycksTest-high_school_european_history|5": {
"acc": 0.7333333333333333,
"acc_stderr": 0.03453131801885417,
"acc_norm": 0.7333333333333333,
"acc_norm_stderr": 0.03453131801885417
},
"harness|hendrycksTest-high_school_geography|5": {
"acc": 0.7424242424242424,
"acc_stderr": 0.03115626951964683,
"acc_norm": 0.7424242424242424,
"acc_norm_stderr": 0.03115626951964683
},
"harness|hendrycksTest-high_school_government_and_politics|5": {
"acc": 0.8549222797927462,
"acc_stderr": 0.02541634309630644,
"acc_norm": 0.8549222797927462,
"acc_norm_stderr": 0.02541634309630644
},
"harness|hendrycksTest-high_school_macroeconomics|5": {
"acc": 0.5615384615384615,
"acc_stderr": 0.02515826601686858,
"acc_norm": 0.5615384615384615,
"acc_norm_stderr": 0.02515826601686858
},
"harness|hendrycksTest-high_school_mathematics|5": {
"acc": 0.34444444444444444,
"acc_stderr": 0.028972648884844267,
"acc_norm": 0.34444444444444444,
"acc_norm_stderr": 0.028972648884844267
},
"harness|hendrycksTest-high_school_microeconomics|5": {
"acc": 0.6302521008403361,
"acc_stderr": 0.03135709599613591,
"acc_norm": 0.6302521008403361,
"acc_norm_stderr": 0.03135709599613591
},
"harness|hendrycksTest-high_school_physics|5": {
"acc": 0.3509933774834437,
"acc_stderr": 0.038969819642573754,
"acc_norm": 0.3509933774834437,
"acc_norm_stderr": 0.038969819642573754
},
"harness|hendrycksTest-high_school_psychology|5": {
"acc": 0.8,
"acc_stderr": 0.01714985851425095,
"acc_norm": 0.8,
"acc_norm_stderr": 0.01714985851425095
},
"harness|hendrycksTest-high_school_statistics|5": {
"acc": 0.4444444444444444,
"acc_stderr": 0.03388857118502326,
"acc_norm": 0.4444444444444444,
"acc_norm_stderr": 0.03388857118502326
},
"harness|hendrycksTest-high_school_us_history|5": {
"acc": 0.7745098039215687,
"acc_stderr": 0.02933116229425174,
"acc_norm": 0.7745098039215687,
"acc_norm_stderr": 0.02933116229425174
},
"harness|hendrycksTest-high_school_world_history|5": {
"acc": 0.7510548523206751,
"acc_stderr": 0.028146970599422644,
"acc_norm": 0.7510548523206751,
"acc_norm_stderr": 0.028146970599422644
},
"harness|hendrycksTest-human_aging|5": {
"acc": 0.6367713004484304,
"acc_stderr": 0.032277904428505,
"acc_norm": 0.6367713004484304,
"acc_norm_stderr": 0.032277904428505
},
"harness|hendrycksTest-human_sexuality|5": {
"acc": 0.7022900763358778,
"acc_stderr": 0.040103589424622034,
"acc_norm": 0.7022900763358778,
"acc_norm_stderr": 0.040103589424622034
},
"harness|hendrycksTest-international_law|5": {
"acc": 0.8099173553719008,
"acc_stderr": 0.03581796951709282,
"acc_norm": 0.8099173553719008,
"acc_norm_stderr": 0.03581796951709282
},
"harness|hendrycksTest-jurisprudence|5": {
"acc": 0.6851851851851852,
"acc_stderr": 0.04489931073591312,
"acc_norm": 0.6851851851851852,
"acc_norm_stderr": 0.04489931073591312
},
"harness|hendrycksTest-logical_fallacies|5": {
"acc": 0.7300613496932515,
"acc_stderr": 0.03487825168497892,
"acc_norm": 0.7300613496932515,
"acc_norm_stderr": 0.03487825168497892
},
"harness|hendrycksTest-machine_learning|5": {
"acc": 0.48214285714285715,
"acc_stderr": 0.047427623612430116,
"acc_norm": 0.48214285714285715,
"acc_norm_stderr": 0.047427623612430116
},
"harness|hendrycksTest-management|5": {
"acc": 0.7475728155339806,
"acc_stderr": 0.04301250399690878,
"acc_norm": 0.7475728155339806,
"acc_norm_stderr": 0.04301250399690878
},
"harness|hendrycksTest-marketing|5": {
"acc": 0.8717948717948718,
"acc_stderr": 0.02190190511507332,
"acc_norm": 0.8717948717948718,
"acc_norm_stderr": 0.02190190511507332
},
"harness|hendrycksTest-medical_genetics|5": {
"acc": 0.68,
"acc_stderr": 0.046882617226215034,
"acc_norm": 0.68,
"acc_norm_stderr": 0.046882617226215034
},
"harness|hendrycksTest-miscellaneous|5": {
"acc": 0.7739463601532567,
"acc_stderr": 0.014957458504335842,
"acc_norm": 0.7739463601532567,
"acc_norm_stderr": 0.014957458504335842
},
"harness|hendrycksTest-moral_disputes|5": {
"acc": 0.6791907514450867,
"acc_stderr": 0.025131000233647886,
"acc_norm": 0.6791907514450867,
"acc_norm_stderr": 0.025131000233647886
},
"harness|hendrycksTest-moral_scenarios|5": {
"acc": 0.3396648044692737,
"acc_stderr": 0.015839400406212494,
"acc_norm": 0.3396648044692737,
"acc_norm_stderr": 0.015839400406212494
},
"harness|hendrycksTest-nutrition|5": {
"acc": 0.7058823529411765,
"acc_stderr": 0.026090162504279056,
"acc_norm": 0.7058823529411765,
"acc_norm_stderr": 0.026090162504279056
},
"harness|hendrycksTest-philosophy|5": {
"acc": 0.6881028938906752,
"acc_stderr": 0.02631185807185416,
"acc_norm": 0.6881028938906752,
"acc_norm_stderr": 0.02631185807185416
},
"harness|hendrycksTest-prehistory|5": {
"acc": 0.6697530864197531,
"acc_stderr": 0.026168298456732846,
"acc_norm": 0.6697530864197531,
"acc_norm_stderr": 0.026168298456732846
},
"harness|hendrycksTest-professional_accounting|5": {
"acc": 0.4574468085106383,
"acc_stderr": 0.029719281272236844,
"acc_norm": 0.4574468085106383,
"acc_norm_stderr": 0.029719281272236844
},
"harness|hendrycksTest-professional_law|5": {
"acc": 0.42633637548891784,
"acc_stderr": 0.012630884771599698,
"acc_norm": 0.42633637548891784,
"acc_norm_stderr": 0.012630884771599698
},
"harness|hendrycksTest-professional_medicine|5": {
"acc": 0.6029411764705882,
"acc_stderr": 0.02972215209928006,
"acc_norm": 0.6029411764705882,
"acc_norm_stderr": 0.02972215209928006
},
"harness|hendrycksTest-professional_psychology|5": {
"acc": 0.6143790849673203,
"acc_stderr": 0.019691459052354025,
"acc_norm": 0.6143790849673203,
"acc_norm_stderr": 0.019691459052354025
},
"harness|hendrycksTest-public_relations|5": {
"acc": 0.7,
"acc_stderr": 0.04389311454644287,
"acc_norm": 0.7,
"acc_norm_stderr": 0.04389311454644287
},
"harness|hendrycksTest-security_studies|5": {
"acc": 0.7387755102040816,
"acc_stderr": 0.028123429335142783,
"acc_norm": 0.7387755102040816,
"acc_norm_stderr": 0.028123429335142783
},
"harness|hendrycksTest-sociology|5": {
"acc": 0.746268656716418,
"acc_stderr": 0.030769444967296018,
"acc_norm": 0.746268656716418,
"acc_norm_stderr": 0.030769444967296018
},
"harness|hendrycksTest-us_foreign_policy|5": {
"acc": 0.81,
"acc_stderr": 0.039427724440366255,
"acc_norm": 0.81,
"acc_norm_stderr": 0.039427724440366255
},
"harness|hendrycksTest-virology|5": {
"acc": 0.5180722891566265,
"acc_stderr": 0.03889951252827216,
"acc_norm": 0.5180722891566265,
"acc_norm_stderr": 0.03889951252827216
},
"harness|hendrycksTest-world_religions|5": {
"acc": 0.8245614035087719,
"acc_stderr": 0.029170885500727665,
"acc_norm": 0.8245614035087719,
"acc_norm_stderr": 0.029170885500727665
},
"harness|truthfulqa:mc|0": {
"mc1": 0.4724602203182375,
"mc1_stderr": 0.017476930190712187,
"mc2": 0.6419919334749323,
"mc2_stderr": 0.01518622081933932
},
"harness|winogrande|5": {
"acc": 0.771112865035517,
"acc_stderr": 0.011807360224025395
},
"harness|gsm8k|5": {
"acc": 0.39272175890826383,
"acc_stderr": 0.01345174534958657
}
}
```
## Dataset Details
### Dataset Description
<!-- Provide a longer summary of what this dataset is. -->
- **Curated by:** [More Information Needed]
- **Funded by [optional]:** [More Information Needed]
- **Shared by [optional]:** [More Information Needed]
- **Language(s) (NLP):** [More Information Needed]
- **License:** [More Information Needed]
### Dataset Sources [optional]
<!-- Provide the basic links for the dataset. -->
- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]
## Uses
<!-- Address questions around how the dataset is intended to be used. -->
### Direct Use
<!-- This section describes suitable use cases for the dataset. -->
[More Information Needed]
### Out-of-Scope Use
<!-- This section addresses misuse, malicious use, and uses that the dataset will not work well for. -->
[More Information Needed]
## Dataset Structure
<!-- This section provides a description of the dataset fields, and additional information about the dataset structure such as criteria used to create the splits, relationships between data points, etc. -->
[More Information Needed]
## Dataset Creation
### Curation Rationale
<!-- Motivation for the creation of this dataset. -->
[More Information Needed]
### Source Data
<!-- This section describes the source data (e.g. news text and headlines, social media posts, translated sentences, ...). -->
#### Data Collection and Processing
<!-- This section describes the data collection and processing process such as data selection criteria, filtering and normalization methods, tools and libraries used, etc. -->
[More Information Needed]
#### Who are the source data producers?
<!-- This section describes the people or systems who originally created the data. It should also include self-reported demographic or identity information for the source data creators if this information is available. -->
[More Information Needed]
### Annotations [optional]
<!-- If the dataset contains annotations which are not part of the initial data collection, use this section to describe them. -->
#### Annotation process
<!-- This section describes the annotation process such as annotation tools used in the process, the amount of data annotated, annotation guidelines provided to the annotators, interannotator statistics, annotation validation, etc. -->
[More Information Needed]
#### Who are the annotators?
<!-- This section describes the people or systems who created the annotations. -->
[More Information Needed]
#### Personal and Sensitive Information
<!-- State whether the dataset contains data that might be considered personal, sensitive, or private (e.g., data that reveals addresses, uniquely identifiable names or aliases, racial or ethnic origins, sexual orientations, religious beliefs, political opinions, financial or health data, etc.). If efforts were made to anonymize the data, describe the anonymization process. -->
[More Information Needed]
## Bias, Risks, and Limitations
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
[More Information Needed]
### Recommendations
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
Users should be made aware of the risks, biases and limitations of the dataset. More information needed for further recommendations.
## Citation [optional]
<!-- If there is a paper or blog post introducing the dataset, the APA and Bibtex information for that should go in this section. -->
**BibTeX:**
[More Information Needed]
**APA:**
[More Information Needed]
## Glossary [optional]
<!-- If relevant, include terms and calculations in this section that can help readers understand the dataset or dataset card. -->
[More Information Needed]
## More Information [optional]
[More Information Needed]
## Dataset Card Authors [optional]
[More Information Needed]
## Dataset Card Contact
[More Information Needed]
原始信息汇总
数据集概述
该数据集是在模型 prince-canuma/Damysus-Coder-v0.1 的评估运行期间自动创建的，用于 Open LLM Leaderboard。
数据集组成
- 数据集包含 63 个配置，每个配置对应一个评估任务。
- 数据集从 1 次运行中创建。每个运行可以在每个配置中找到特定的分割，分割名称使用运行的时间戳。
- "train" 分割始终指向最新的结果。
- 一个额外的配置 "results" 存储所有运行的聚合结果，用于计算和显示 Open LLM Leaderboard 上的聚合指标。
数据加载示例
python from datasets import load_dataset data = load_dataset("open-llm-leaderboard/details_prince-canuma__Damysus-Coder-v0.1", "harness_winogrande_5", split="train")
最新结果
以下是 2024-04-15T21:07:57.637768 运行的最新结果：
python { "all": { "acc": 0.6043961155404135, "acc_stderr": 0.033186632585662346, "acc_norm": 0.6094482265591639, "acc_norm_stderr": 0.033859874632835685, "mc1": 0.4724602203182375, "mc1_stderr": 0.017476930190712187, "mc2": 0.6419919334749323, "mc2_stderr": 0.01518622081933932 }, "harness|arc:challenge|25": { "acc": 0.5554607508532423, "acc_stderr": 0.01452122640562708, "acc_norm": 0.6092150170648464, "acc_norm_stderr": 0.014258563880513782 }, "harness|hellaswag|10": { "acc": 0.64070902210715, "acc_stderr": 0.004788120727316245, "acc_norm": 0.840071698864768, "acc_norm_stderr": 0.0036579044379436557 }, "harness|hendrycksTest-abstract_algebra|5": { "acc": 0.3, "acc_stderr": 0.046056618647183814, "acc_norm": 0.3, "acc_norm_stderr": 0.046056618647183814 }, "harness|hendrycksTest-anatomy|5": { "acc": 0.5703703703703704, "acc_stderr": 0.042763494943765995, "acc_norm": 0.5703703703703704, "acc_norm_stderr": 0.042763494943765995 }, "harness|hendrycksTest-astronomy|5": { "acc": 0.6118421052631579, "acc_stderr": 0.03965842097512744, "acc_norm": 0.6118421052631579, "acc_norm_stderr": 0.03965842097512744 }, "harness|hendrycksTest-business_ethics|5": { "acc": 0.57, "acc_stderr": 0.049756985195624284, "acc_norm": 0.57, "acc_norm_stderr": 0.049756985195624284 }, "harness|hendrycksTest-clinical_knowledge|5": { "acc": 0.690566037735849, "acc_stderr": 0.028450154794118637, "acc_norm": 0.690566037735849, "acc_norm_stderr": 0.028450154794118637 }, "harness|hendrycksTest-college_biology|5": { "acc": 0.6527777777777778, "acc_stderr": 0.039812405437178615, "acc_norm": 0.6527777777777778, "acc_norm_stderr": 0.039812405437178615 }, "harness|hendrycksTest-college_chemistry|5": { "acc": 0.37, "acc_stderr": 0.04852365870939099, "acc_norm": 0.37, "acc_norm_stderr": 0.04852365870939099 }, "harness|hendrycksTest-college_computer_science|5": { "acc": 0.49, "acc_stderr": 0.05024183937956913, "acc_norm": 0.49, "acc_norm_stderr": 0.05024183937956913 }, "harness|hendrycksTest-college_mathematics|5": { "acc": 0.36, "acc_stderr": 0.04824181513244218, "acc_norm": 0.36, "acc_norm_stderr": 0.04824181513244218 }, "harness|hendrycksTest-college_medicine|5": { "acc": 0.5780346820809249, "acc_stderr": 0.0376574669386515, "acc_norm": 0.5780346820809249, "acc_norm_stderr": 0.0376574669386515 }, "harness|hendrycksTest-college_physics|5": { "acc": 0.46078431372549017, "acc_stderr": 0.049598599663841815, "acc_norm": 0.46078431372549017, "acc_norm_stderr": 0.049598599663841815 }, "harness|hendrycksTest-computer_security|5": { "acc": 0.74, "acc_stderr": 0.04408440022768078, "acc_norm": 0.74, "acc_norm_stderr": 0.04408440022768078 }, "harness|hendrycksTest-conceptual_physics|5": { "acc": 0.5191489361702127, "acc_stderr": 0.03266204299064678, "acc_norm": 0.5191489361702127, "acc_norm_stderr": 0.03266204299064678 }, "harness|hendrycksTest-econometrics|5": { "acc": 0.40350877192982454, "acc_stderr": 0.046151869625837026, "acc_norm": 0.40350877192982454, "acc_norm_stderr": 0.046151869625837026 }, "harness|hendrycksTest-electrical_engineering|5": { "acc": 0.5793103448275863, "acc_stderr": 0.0411391498118926, "acc_norm": 0.5793103448275863, "acc_norm_stderr": 0.0411391498118926 }, "harness|hendrycksTest-elementary_mathematics|5": { "acc": 0.3835978835978836, "acc_stderr": 0.025043757318520193, "acc_norm": 0.3835978835978836, "acc_norm_stderr": 0.025043757318520193 }, "harness|hendrycksTest-formal_logic|5": { "acc": 0.36507936507936506, "acc_stderr": 0.04306241259127153, "acc_norm": 0.36507936507936506, "acc_norm_stderr": 0.04306241259127153 }, "harness|hendrycksTest-global_facts|5": { "acc": 0.4, "acc_stderr": 0.049236596391733084, "acc_norm": 0.4, "acc_norm_stderr": 0.049236596391733084 }, "harness|hendrycksTest-high_school_biology|5": { "acc": 0.6967741935483871, "acc_stderr": 0.02614868593067175, "acc_norm": 0.6967741935483871, "acc_norm_stderr": 0.02614868593067175 }, "harness|hendrycksTest-high_school_chemistry|5": { "acc": 0.5073891625615764, "acc_stderr": 0.0351760354036101, "acc_norm": 0.5073891625615764, "acc_norm_stderr": 0.0351760354036101 }, "harness|hendrycksTest-high_school_computer_science|5": { "acc": 0.63, "acc_stderr": 0.04852365870939099, "acc_norm": 0.63, "acc_norm_stderr": 0.04852365870939099 }, "harness|hendrycksTest-high_school_european_history|5": { "acc": 0.7333333333333333, "acc_stderr": 0.03453131801885417, "acc_norm": 0.7333333333333333, "acc_norm_stderr": 0.03453131801885417 }, "harness|hendrycksTest-high_school_geography|5": { "acc": 0.7424242424242424, "acc_stderr": 0.03115626951964683, "acc_norm": 0.7424242424242424, "acc_norm_stderr": 0.03115626951964683 }, "harness|hendrycksTest-high_school_government_and_politics|5": { "acc": 0.8549222797927462, "acc_stderr": 0.02541634309630644, "acc_norm": 0.8549222797927462, "acc_norm_stderr": 0.02541634309630644 }, "harness|hendrycksTest-high_school_macroeconomics|5": { "acc": 0.5615384615384615, "acc_stderr": 0.02515826601686858, "acc_norm": 0.5615384615384615, "acc_norm_stderr": 0.02515826601686858 }, "harness|hendrycksTest-high_school_mathematics|5": { "acc": 0.34444444444444444, "acc_stderr": 0.028972648884844267, "acc_norm": 0.34444444444444444, "acc_norm_stderr": 0
搜集汇总
数据集介绍
构建方式
在大型语言模型评估领域，详细记录模型在标准化基准上的表现对于推动技术演进至关重要。该数据集正是在Open LLM Leaderboard框架下，针对prince-canuma/Damysus-Coder-v0.1模型的一次完整评估运行而自动生成的。数据集构建过程围绕63个评估任务配置展开，每个配置对应一个特定的基准测试，如ARC挑战赛、HellaSwag、GSM8K及涵盖57个学科的MMLU等。评估结果以Parquet文件形式存储，每一轮运行的数据被组织为独立的数据拆分，拆分名称采用运行时间戳命名，而'train'拆分则始终指向最新一轮的结果。此外，数据集还包含一个名为'results'的附加配置，用于汇总所有运行的聚合指标，这些指标直接服务于Open LLM Leaderboard上的综合评分与可视化展示。
特点
该数据集呈现出高度结构化且版本清晰的显著特征。它囊括了63个不同任务配置，覆盖从常识推理到专业学科知识等广泛领域，为模型能力的多维度剖析提供了丰富素材。每个配置下的数据均按运行轮次进行拆分，时间戳命名的拆分方式使得研究者能够追溯模型性能的演化轨迹，而'train'拆分自动指向最新结果的设计则确保了数据引用的时效性。'results'配置的设立进一步增强了数据集的实用性，它将分散于各任务的评估指标进行集中汇总，便于用户快速获取模型的整体表现概览。这种兼顾细粒度细节与宏观聚合的设计，使得该数据集既适合深入的单项任务分析，也适用于模型间的横向对比。
使用方法
研究人员可通过HuggingFace的datasets库便捷地加载与使用该数据集。加载时需指定数据集标识符'open-llm-leaderboard/details_prince-canuma__Damysus-Coder-v0.1'，并选择所需的任务配置名称，例如'harness_winogrande_5'，同时通过'split'参数指定具体的数据拆分，如'train'以获取最新结果。对于需要访问历史运行数据的场景，可使用对应的时间戳拆分名称。此外，'results'配置提供了所有任务的聚合结果，可直接用于模型性能的快速评估与报告生成。这种灵活的加载机制支持用户根据研究需求，精准提取特定任务或特定轮次的评估细节，极大地便利了后续的模型分析与比较工作。
背景与挑战
背景概述
随着大语言模型在自然语言处理领域的迅猛发展，如何系统性地评估其综合能力成为学术界与工业界共同关注的焦点。在此背景下，Hugging Face团队于2023年发起了Open LLM Leaderboard项目，旨在通过标准化评测框架为开源模型提供公平、透明的性能对比平台。该数据集作为项目的一部分，由Clémentine Fourrier等研究人员主导创建，专门记录了prince-canuma团队开发的Damysus-Coder-v0.1模型在2024年4月15日的评估运行细节。数据集涵盖了ARC挑战、HellaSwag、MMLU、TruthfulQA、Winogrande和GSM8K等63个多样化任务配置，每个配置均包含原始评分与归一化指标，为研究者深入分析代码生成模型在多维度推理、常识理解与数学问题求解方面的表现提供了详实依据。该数据集的发布不仅推动了开源模型评估的标准化进程，更成为衡量代码智能体领域进展的重要基准。
当前挑战
该数据集所承载的核心挑战在于如何精准衡量大语言模型在复杂推理任务中的泛化能力。在领域问题层面，现有基准如MMLU虽覆盖57个学科，但模型在抽象代数、大学数学等子任务上的表现仍显薄弱，揭示出当前模型在形式化推理与多步逻辑推导方面的系统性缺陷。构建过程中，评估框架需面对任务异构性带来的指标对齐难题——不同任务采用零样本或少样本设置，其评分标准（如准确率与归一化准确率）的统计显著性需通过严格的标准误差计算来验证。此外，模型输出对提示措辞高度敏感，单次运行结果可能无法完全反映真实能力，而数据集仅包含一次评估运行，这限制了结论的鲁棒性。数据存储格式上，63个独立配置与时间戳分割的管理方式虽便于细粒度分析，却增加了跨任务综合比较的复杂度，亟需更统一的聚合策略来提升结果的可解释性。
常用场景
经典使用场景
在大型语言模型（LLM）评估领域，Open LLM Leaderboard 的细节评测数据集扮演着关键角色。该数据集专为记录和复现模型在标准化基准上的细粒度表现而设计，涵盖从常识推理（如 HellaSwag、Winogrande）到数学解题（GSM8K）再到多学科知识（MMLU）的 63 个任务配置。研究者可借助其结构化存储的评测结果，精确分析模型在特定子任务上的优势与短板，从而指导模型优化方向。例如，通过加载特定任务的 Parquet 文件，能够获取模型在 ARC-Challenge 上的准确率与标准误差，为对比不同架构或训练策略提供可重复的量化依据。
衍生相关工作
该数据集衍生了一系列重要的评测工具与方法论工作。其设计理念直接启发了后续对 LLM 评测透明化的探索，例如研究者基于其细粒度结果开发出任务级能力雷达图，用于可视化模型在 57 个 MMLU 子领域上的强弱分布。同时，数据集中包含的多次运行分裂记录，为评测稳定性分析提供了基准，催生了关于随机性对模型排名影响的研究。此外，该数据集作为 Open LLM Leaderboard 的基础组件，支撑了诸如“模型能力解耦分析”、“评测任务难度分级”等前沿课题，推动了社区对 LLM 评测标准化的深入讨论，并成为众多对比实验的参考基线。
数据集最近研究
最新研究方向
当前，大语言模型的评估基准正从单一维度向多任务、多领域综合评测演进，以更全面地反映模型在复杂现实场景中的泛化能力。该数据集聚焦于Damysus-Coder-v0.1模型在Open LLM Leaderboard上的表现，覆盖了从常识推理（如ARC-Challenge、HellaSwag）到数学解题（GSM8K），再到涵盖57个学科的专业知识测试（MMLU）等广泛任务。这一评测体系与近期大模型在代码生成、逻辑推理及跨学科知识整合方面的前沿研究紧密相连，尤其是在推动模型在低资源场景下的鲁棒性提升方面具有重要价值。通过细粒度的准确率与标准差分析，该数据集为研究者提供了模型能力边界的定量刻画，促进了社区对开源模型性能的透明化比较与迭代优化。
以上内容由遇见数据集搜集并总结生成
