malhajar/mmlu_tr-v0.2

---
contributions:
- contributor: Mohamad Alhajar
profile: https://www.linkedin.com/in/muhammet-alhajar/
roles:
- translator
- data curator
configs:
- config_name: abstract_algebra
data_files:
- path: abstract_algebra/dev-*
split: dev
- path: abstract_algebra/test-*
split: test
- path: abstract_algebra/validation-*
split: validation
- config_name: anatomy
data_files:
- path: anatomy/dev-*
split: dev
- path: anatomy/test-*
split: test
- path: anatomy/validation-*
split: validation
- config_name: astronomy
data_files:
- path: astronomy/dev-*
split: dev
- path: astronomy/test-*
split: test
- path: astronomy/validation-*
split: validation
- config_name: business_ethics
data_files:
- path: business_ethics/dev-*
split: dev
- path: business_ethics/test-*
split: test
- path: business_ethics/validation-*
split: validation
- config_name: clinical_knowledge
data_files:
- path: clinical_knowledge/dev-*
split: dev
- path: clinical_knowledge/test-*
split: test
- path: clinical_knowledge/validation-*
split: validation
- config_name: college_biology
data_files:
- path: college_biology/dev-*
split: dev
- path: college_biology/test-*
split: test
- path: college_biology/validation-*
split: validation
- config_name: college_chemistry
data_files:
- path: college_chemistry/dev-*
split: dev
- path: college_chemistry/test-*
split: test
- path: college_chemistry/validation-*
split: validation
- config_name: college_computer_science
data_files:
- path: college_computer_science/dev-*
split: dev
- path: college_computer_science/test-*
split: test
- path: college_computer_science/validation-*
split: validation
- config_name: college_mathematics
data_files:
- path: college_mathematics/dev-*
split: dev
- path: college_mathematics/test-*
split: test
- path: college_mathematics/validation-*
split: validation
- config_name: college_medicine
data_files:
- path: college_medicine/dev-*
split: dev
- path: college_medicine/test-*
split: test
- path: college_medicine/validation-*
split: validation
- config_name: college_physics
data_files:
- path: college_physics/dev-*
split: dev
- path: college_physics/test-*
split: test
- path: college_physics/validation-*
split: validation
- config_name: computer_security
data_files:
- path: computer_security/dev-*
split: dev
- path: computer_security/test-*
split: test
- path: computer_security/validation-*
split: validation
- config_name: conceptual_physics
data_files:
- path: conceptual_physics/dev-*
split: dev
- path: conceptual_physics/test-*
split: test
- path: conceptual_physics/validation-*
split: validation
- config_name: econometrics
data_files:
- path: econometrics/dev-*
split: dev
- path: econometrics/test-*
split: test
- path: econometrics/validation-*
split: validation
- config_name: electrical_engineering
data_files:
- path: electrical_engineering/dev-*
split: dev
- path: electrical_engineering/test-*
split: test
- path: electrical_engineering/validation-*
split: validation
- config_name: elementary_mathematics
data_files:
- path: elementary_mathematics/dev-*
split: dev
- path: elementary_mathematics/test-*
split: test
- path: elementary_mathematics/validation-*
split: validation
- config_name: formal_logic
data_files:
- path: formal_logic/dev-*
split: dev
- path: formal_logic/test-*
split: test
- path: formal_logic/validation-*
split: validation
- config_name: global_facts
data_files:
- path: global_facts/dev-*
split: dev
- path: global_facts/test-*
split: test
- path: global_facts/validation-*
split: validation
- config_name: high_school_biology
data_files:
- path: high_school_biology/dev-*
split: dev
- path: high_school_biology/test-*
split: test
- path: high_school_biology/validation-*
split: validation
- config_name: high_school_chemistry
data_files:
- path: high_school_chemistry/dev-*
split: dev
- path: high_school_chemistry/test-*
split: test
- path: high_school_chemistry/validation-*
split: validation
- config_name: high_school_computer_science
data_files:
- path: high_school_computer_science/dev-*
split: dev
- path: high_school_computer_science/test-*
split: test
- path: high_school_computer_science/validation-*
split: validation
- config_name: high_school_european_history
data_files:
- path: high_school_european_history/dev-*
split: dev
- path: high_school_european_history/test-*
split: test
- path: high_school_european_history/validation-*
split: validation
- config_name: high_school_geography
data_files:
- path: high_school_geography/dev-*
split: dev
- path: high_school_geography/test-*
split: test
- path: high_school_geography/validation-*
split: validation
- config_name: high_school_government_and_politics
data_files:
- path: high_school_government_and_politics/dev-*
split: dev
- path: high_school_government_and_politics/test-*
split: test
- path: high_school_government_and_politics/validation-*
split: validation
- config_name: high_school_macroeconomics
data_files:
- path: high_school_macroeconomics/dev-*
split: dev
- path: high_school_macroeconomics/test-*
split: test
- path: high_school_macroeconomics/validation-*
split: validation
- config_name: high_school_mathematics
data_files:
- path: high_school_mathematics/dev-*
split: dev
- path: high_school_mathematics/test-*
split: test
- path: high_school_mathematics/validation-*
split: validation
- config_name: high_school_microeconomics
data_files:
- path: high_school_microeconomics/dev-*
split: dev
- path: high_school_microeconomics/test-*
split: test
- path: high_school_microeconomics/validation-*
split: validation
- config_name: high_school_physics
data_files:
- path: high_school_physics/dev-*
split: dev
- path: high_school_physics/test-*
split: test
- path: high_school_physics/validation-*
split: validation
- config_name: high_school_psychology
data_files:
- path: high_school_psychology/dev-*
split: dev
- path: high_school_psychology/test-*
split: test
- path: high_school_psychology/validation-*
split: validation
- config_name: high_school_statistics
data_files:
- path: high_school_statistics/dev-*
split: dev
- path: high_school_statistics/test-*
split: test
- path: high_school_statistics/validation-*
split: validation
- config_name: high_school_us_history
data_files:
- path: high_school_us_history/dev-*
split: dev
- path: high_school_us_history/test-*
split: test
- path: high_school_us_history/validation-*
split: validation
- config_name: high_school_world_history
data_files:
- path: high_school_world_history/dev-*
split: dev
- path: high_school_world_history/test-*
split: test
- path: high_school_world_history/validation-*
split: validation
- config_name: human_aging
data_files:
- path: human_aging/dev-*
split: dev
- path: human_aging/test-*
split: test
- path: human_aging/validation-*
split: validation
- config_name: human_sexuality
data_files:
- path: human_sexuality/dev-*
split: dev
- path: human_sexuality/test-*
split: test
- path: human_sexuality/validation-*
split: validation
- config_name: international_law
data_files:
- path: international_law/dev-*
split: dev
- path: international_law/test-*
split: test
- path: international_law/validation-*
split: validation
- config_name: jurisprudence
data_files:
- path: jurisprudence/dev-*
split: dev
- path: jurisprudence/test-*
split: test
- path: jurisprudence/validation-*
split: validation
- config_name: logical_fallacies
data_files:
- path: logical_fallacies/dev-*
split: dev
- path: logical_fallacies/test-*
split: test
- path: logical_fallacies/validation-*
split: validation
- config_name: machine_learning
data_files:
- path: machine_learning/dev-*
split: dev
- path: machine_learning/test-*
split: test
- path: machine_learning/validation-*
split: validation
- config_name: management
data_files:
- path: management/dev-*
split: dev
- path: management/test-*
split: test
- path: management/validation-*
split: validation
- config_name: marketing
data_files:
- path: marketing/dev-*
split: dev
- path: marketing/test-*
split: test
- path: marketing/validation-*
split: validation
- config_name: medical_genetics
data_files:
- path: medical_genetics/dev-*
split: dev
- path: medical_genetics/test-*
split: test
- path: medical_genetics/validation-*
split: validation
- config_name: miscellaneous
data_files:
- path: miscellaneous/dev-*
split: dev
- path: miscellaneous/test-*
split: test
- path: miscellaneous/validation-*
split: validation
- config_name: moral_disputes
data_files:
- path: moral_disputes/dev-*
split: dev
- path: moral_disputes/test-*
split: test
- path: moral_disputes/validation-*
split: validation
- config_name: moral_scenarios
data_files:
- path: moral_scenarios/dev-*
split: dev
- path: moral_scenarios/test-*
split: test
- path: moral_scenarios/validation-*
split: validation
- config_name: nutrition
data_files:
- path: nutrition/dev-*
split: dev
- path: nutrition/test-*
split: test
- path: nutrition/validation-*
split: validation
- config_name: philosophy
data_files:
- path: philosophy/dev-*
split: dev
- path: philosophy/test-*
split: test
- path: philosophy/validation-*
split: validation
- config_name: prehistory
data_files:
- path: prehistory/dev-*
split: dev
- path: prehistory/test-*
split: test
- path: prehistory/validation-*
split: validation
- config_name: professional_accounting
data_files:
- path: professional_accounting/dev-*
split: dev
- path: professional_accounting/test-*
split: test
- path: professional_accounting/validation-*
split: validation
- config_name: professional_law
data_files:
- path: professional_law/dev-*
split: dev
- path: professional_law/test-*
split: test
- path: professional_law/validation-*
split: validation
- config_name: professional_medicine
data_files:
- path: professional_medicine/dev-*
split: dev
- path: professional_medicine/test-*
split: test
- path: professional_medicine/validation-*
split: validation
- config_name: professional_psychology
data_files:
- path: professional_psychology/dev-*
split: dev
- path: professional_psychology/test-*
split: test
- path: professional_psychology/validation-*
split: validation
- config_name: public_relations
data_files:
- path: public_relations/dev-*
split: dev
- path: public_relations/test-*
split: test
- path: public_relations/validation-*
split: validation
- config_name: security_studies
data_files:
- path: security_studies/dev-*
split: dev
- path: security_studies/test-*
split: test
- path: security_studies/validation-*
split: validation
- config_name: sociology
data_files:
- path: sociology/dev-*
split: dev
- path: sociology/test-*
split: test
- path: sociology/validation-*
split: validation
- config_name: us_foreign_policy
data_files:
- path: us_foreign_policy/dev-*
split: dev
- path: us_foreign_policy/test-*
split: test
- path: us_foreign_policy/validation-*
split: validation
- config_name: virology
data_files:
- path: virology/dev-*
split: dev
- path: virology/test-*
split: test
- path: virology/validation-*
split: validation
- config_name: world_religions
data_files:
- path: world_religions/dev-*
split: dev
- path: world_religions/test-*
split: test
- path: world_religions/validation-*
split: validation
dataset_info:
- config_name: abstract_algebra
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2010
num_examples: 5
- name: test
num_bytes: 48110
num_examples: 100
- name: validation
num_bytes: 4900
num_examples: 11
- config_name: anatomy
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2170
num_examples: 5
- name: test
num_bytes: 70130
num_examples: 131
- name: validation
num_bytes: 7041
num_examples: 14
- config_name: astronomy
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 4823
num_examples: 5
- name: test
num_bytes: 107489
num_examples: 150
- name: validation
num_bytes: 10712
num_examples: 15
- config_name: business_ethics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 4960
num_examples: 5
- name: test
num_bytes: 72833
num_examples: 99
- name: validation
num_bytes: 6895
num_examples: 11
- config_name: clinical_knowledge
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2861
num_examples: 5
- name: test
num_bytes: 140864
num_examples: 256
- name: validation
num_bytes: 14851
num_examples: 28
- config_name: college_biology
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2450
num_examples: 4
- name: test
num_bytes: 108187
num_examples: 142
- name: validation
num_bytes: 11068
num_examples: 16
- config_name: college_chemistry
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2945
num_examples: 5
- name: test
num_bytes: 59254
num_examples: 99
- name: validation
num_bytes: 4551
num_examples: 7
- config_name: college_computer_science
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 5911
num_examples: 5
- name: test
num_bytes: 96162
num_examples: 99
- name: validation
num_bytes: 10339
num_examples: 11
- config_name: college_mathematics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3398
num_examples: 5
- name: test
num_bytes: 61015
num_examples: 100
- name: validation
num_bytes: 6527
num_examples: 11
- config_name: college_medicine
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3691
num_examples: 5
- name: test
num_bytes: 155366
num_examples: 168
- name: validation
num_bytes: 17607
num_examples: 22
- config_name: college_physics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2983
num_examples: 5
- name: test
num_bytes: 69270
num_examples: 101
- name: validation
num_bytes: 8330
num_examples: 11
- config_name: computer_security
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2845
num_examples: 5
- name: test
num_bytes: 65752
num_examples: 100
- name: validation
num_bytes: 10520
num_examples: 11
- config_name: conceptual_physics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2236
num_examples: 5
- name: test
num_bytes: 98238
num_examples: 233
- name: validation
num_bytes: 10674
num_examples: 26
- config_name: econometrics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3274
num_examples: 4
- name: test
num_bytes: 102293
num_examples: 114
- name: validation
num_bytes: 11241
num_examples: 12
- config_name: electrical_engineering
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2330
num_examples: 5
- name: test
num_bytes: 60117
num_examples: 144
- name: validation
num_bytes: 8397
num_examples: 16
- config_name: elementary_mathematics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3360
num_examples: 5
- name: test
num_bytes: 166175
num_examples: 373
- name: validation
num_bytes: 21462
num_examples: 40
- config_name: formal_logic
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 7700
num_examples: 5
- name: test
num_bytes: 141194
num_examples: 126
- name: validation
num_bytes: 15041
num_examples: 14
- config_name: global_facts
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2907
num_examples: 5
- name: test
num_bytes: 44241
num_examples: 98
- name: validation
num_bytes: 4412
num_examples: 10
- config_name: high_school_biology
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3613
num_examples: 5
- name: test
num_bytes: 235302
num_examples: 300
- name: validation
num_bytes: 24080
num_examples: 32
- config_name: high_school_chemistry
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2760
num_examples: 5
- name: test
num_bytes: 129230
num_examples: 197
- name: validation
num_bytes: 15225
num_examples: 21
- config_name: high_school_computer_science
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 7130
num_examples: 5
- name: test
num_bytes: 102104
num_examples: 100
- name: validation
num_bytes: 5303
num_examples: 8
- config_name: high_school_european_history
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 22311
num_examples: 5
- name: test
num_bytes: 481301
num_examples: 150
- name: validation
num_bytes: 53102
num_examples: 16
- config_name: high_school_geography
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3813
num_examples: 5
- name: test
num_bytes: 98025
num_examples: 197
- name: validation
num_bytes: 9695
num_examples: 20
- config_name: high_school_government_and_politics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 4037
num_examples: 5
- name: test
num_bytes: 145986
num_examples: 187
- name: validation
num_bytes: 15103
num_examples: 20
- config_name: high_school_macroeconomics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3173
num_examples: 5
- name: test
num_bytes: 262282
num_examples: 390
- name: validation
num_bytes: 28172
num_examples: 42
- config_name: high_school_mathematics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3929
num_examples: 5
- name: test
num_bytes: 146822
num_examples: 270
- name: validation
num_bytes: 14522
num_examples: 28
- config_name: high_school_microeconomics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2707
num_examples: 5
- name: test
num_bytes: 169374
num_examples: 237
- name: validation
num_bytes: 16409
num_examples: 25
- config_name: high_school_physics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3381
num_examples: 5
- name: test
num_bytes: 127963
num_examples: 147
- name: validation
num_bytes: 14540
num_examples: 17
- config_name: high_school_psychology
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 4065
num_examples: 5
- name: test
num_bytes: 350514
num_examples: 533
- name: validation
num_bytes: 38253
num_examples: 58
- config_name: high_school_statistics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 5108
num_examples: 5
- name: test
num_bytes: 240550
num_examples: 216
- name: validation
num_bytes: 21795
num_examples: 23
- config_name: high_school_us_history
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 17832
num_examples: 5
- name: test
num_bytes: 544251
num_examples: 179
- name: validation
num_bytes: 55762
num_examples: 18
- config_name: high_school_world_history
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 10288
num_examples: 5
- name: test
num_bytes: 676637
num_examples: 213
- name: validation
num_bytes: 87320
num_examples: 24
- config_name: human_aging
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2391
num_examples: 5
- name: test
num_bytes: 105704
num_examples: 212
- name: validation
num_bytes: 11471
num_examples: 23
- config_name: human_sexuality
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2480
num_examples: 4
- name: test
num_bytes: 62150
num_examples: 115
- name: validation
num_bytes: 5111
num_examples: 11
- config_name: international_law
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 5325
num_examples: 5
- name: test
num_bytes: 120577
num_examples: 121
- name: validation
num_bytes: 14361
num_examples: 13
- config_name: jurisprudence
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2829
num_examples: 5
- name: test
num_bytes: 74840
num_examples: 106
- name: validation
num_bytes: 8340
num_examples: 11
- config_name: logical_fallacies
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3425
num_examples: 5
- name: test
num_bytes: 113836
num_examples: 161
- name: validation
num_bytes: 11741
num_examples: 18
- config_name: machine_learning
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 5406
num_examples: 5
- name: test
num_bytes: 79515
num_examples: 112
- name: validation
num_bytes: 7467
num_examples: 11
- config_name: management
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2041
num_examples: 5
- name: test
num_bytes: 46345
num_examples: 99
- name: validation
num_bytes: 4231
num_examples: 11
- config_name: marketing
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3411
num_examples: 5
- name: test
num_bytes: 133427
num_examples: 217
- name: validation
num_bytes: 15743
num_examples: 23
- config_name: medical_genetics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2401
num_examples: 5
- name: test
num_bytes: 46863
num_examples: 95
- name: validation
num_bytes: 6735
num_examples: 11
- config_name: miscellaneous
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 1637
num_examples: 5
- name: test
num_bytes: 358140
num_examples: 766
- name: validation
num_bytes: 34548
num_examples: 86
- config_name: moral_disputes
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3858
num_examples: 5
- name: test
num_bytes: 221059
num_examples: 308
- name: validation
num_bytes: 26723
num_examples: 35
- config_name: moral_scenarios
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3844
num_examples: 4
- name: test
num_bytes: 850011
num_examples: 872
- name: validation
num_bytes: 97469
num_examples: 99
- config_name: nutrition
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 4609
num_examples: 5
- name: test
num_bytes: 210741
num_examples: 305
- name: validation
num_bytes: 19867
num_examples: 33
- config_name: philosophy
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2221
num_examples: 5
- name: test
num_bytes: 175629
num_examples: 299
- name: validation
num_bytes: 20743
num_examples: 34
- config_name: prehistory
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3442
num_examples: 4
- name: test
num_bytes: 191165
num_examples: 300
- name: validation
num_bytes: 20934
num_examples: 31
- config_name: professional_accounting
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 4859
num_examples: 5
- name: test
num_bytes: 274342
num_examples: 279
- name: validation
num_bytes: 31921
num_examples: 31
- config_name: professional_law
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 14021
num_examples: 5
- name: test
num_bytes: 3597412
num_examples: 1388
- name: validation
num_bytes: 363802
num_examples: 145
- config_name: professional_medicine
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 7991
num_examples: 5
- name: test
num_bytes: 435371
num_examples: 261
- name: validation
num_bytes: 48876
num_examples: 30
- config_name: professional_psychology
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 4180
num_examples: 4
- name: test
num_bytes: 492043
num_examples: 594
- name: validation
num_bytes: 61140
num_examples: 67
- config_name: public_relations
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3407
num_examples: 5
- name: test
num_bytes: 63332
num_examples: 108
- name: validation
num_bytes: 10088
num_examples: 12
- config_name: security_studies
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 8391
num_examples: 4
- name: test
num_bytes: 432951
num_examples: 234
- name: validation
num_bytes: 50120
num_examples: 27
- config_name: sociology
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3547
num_examples: 5
- name: test
num_bytes: 146354
num_examples: 195
- name: validation
num_bytes: 14561
num_examples: 19
- config_name: us_foreign_policy
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3439
num_examples: 4
- name: test
num_bytes: 65685
num_examples: 99
- name: validation
num_bytes: 7602
num_examples: 11
- config_name: virology
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2675
num_examples: 5
- name: test
num_bytes: 88457
num_examples: 159
- name: validation
num_bytes: 11911
num_examples: 16
- config_name: world_religions
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 1746
num_examples: 5
- name: test
num_bytes: 63092
num_examples: 168
- name: validation
num_bytes: 7307
num_examples: 18
language:
- tr
license: apache-2.0
tags:
- multi-task
- multitask
- mmlu
- hendrycks_test
task_categories:
- text-classification
- multiple-choice
- question-answering
task_ids:
- multiple-choice-qa
- open-domain-qa
- closed-domain-qa

---
# Dataset Card for mmlu_tr-v0.2

## Overview

**malhajar/mmlu_tr-v0.2** is an enhanced version of the original **mmlu-tr** dataset, specifically developed for use in the **[OpenLLMTurkishLeaderboard v0.2](https://huggingface.co/spaces/malhajar/OpenLLMTurkishLeaderboard)**. This iteration of the dataset has been translated into Turkish using advanced language models like GPT-4, with English text provided for cross-checking to ensure accuracy and reliability. The dataset is tailored to assist in evaluating the performance of Turkish language models (LLMs) and to establish robust benchmarks within the NLP community.

### Dataset Description

- **Source Dataset:** [mmlu](https://huggingface.co/datasets/tasksource/mmlu)
- **Leaderboard:** [OpenLLMTurkishLeaderboard v0.2](https://huggingface.co/spaces/malhajar/OpenLLMTurkishLeaderboard_v0.2)

### Languages

The text in the dataset is primarily in Turkish, with auxiliary English text for validation and cross-checking purposes.

## Dataset Structure

### Data Instances

A typical data instance comprises a question in Turkish, multiple choices, and an answer. English translations are provided for each instance to facilitate bilingual training and evaluation.

### Data Fields

- `question_tr`: the question text in Turkish.
- `choices_tr`: an array of multiple choice options in Turkish.
- `answer_tr`: the index of the correct answer in the choices array.
- `question_en`: the English translation of the question.
- `choices_en`: an array of multiple choice options in English.
- `answer_en`: the index of the correct answer in the English choices array, which should match `answer_tr`.

### Data Splits

The dataset is divided into three splits to support diverse training scenarios:
- **Development (dev)**: Used for model tuning and validation.
- **Test**: Used for final model evaluation to simulate performance on unseen data.
- **Validation**: Additional split for adjusting model hyperparameters without overfitting the test data.

## Additional Information

### Dataset Curator

The dataset was curated by [`Mohamad Alhajar`](https://www.linkedin.com/in/muhammet-alhajar/) , leveraging GPT-4 for translations to ensure high linguistic quality and fidelity.

### Licensing Information

The dataset is available under the Apache-2.0 license, allowing for wide distribution and use in both academic and commercial settings.

### Citation Information

If you use the **mmlu_tr-v0.2** dataset in your research or application, please cite it as follows:

```
@misc{mmlu_tr-v0.2,
author = {Mohamad Alhajar},
title = {mmlu_tr-v0.2},
year = {2024},
publisher = {Mohamad Alhajar},
howpublished = "{https://huggingface.co/datasets/malhajar/mmlu_tr-v0.2}"
}
```

贡献信息：
- 贡献者：穆罕默德·阿尔哈贾尔（Mohamad Alhajar）
个人主页：https://www.linkedin.com/in/muhammet-alhajar/
角色：
- 译员
- 数据管理员
配置项：
所有配置项均对应一个垂直学科领域，每个配置项包含开发集、测试集、验证集三个数据划分，文件路径格式为`{配置名称}/划分前缀-*`，配置名称包括抽象代数、解剖学、天文学、商业伦理、临床知识、大学生物学、大学化学、大学计算机科学、大学数学、大学医学、大学物理学、计算机安全、概念物理学、计量经济学、电气工程、初等数学、形式逻辑、全球事实、高中生物学、高中化学、高中计算机科学、高中欧洲历史、高中地理学、高中政府与政治、高中宏观经济学、高中数学、高中微观经济学、高中物理学、高中心理学、高中统计学、高中美国历史、高中世界历史、人类衰老、人类性学、国际法、法理学、逻辑谬误、机器学习、管理学、市场营销、医学遗传学、综合类目、道德争议、道德场景、营养学、哲学、史前史、专业会计学、专业法学、专业医学、专业心理学、公共关系、安全研究、社会学、美国外交政策、病毒学、世界宗教。
数据集信息：
每个配置项的数据字段均包含：
- 数据类型：字符串，字段名：问题
- 字段名：选项，类型：字符串序列
- 数据类型：64位整数，字段名：答案
- 数据类型：字符串，字段名：英文问题
- 字段名：英文选项，类型：字符串序列
各配置项对应的数据划分包含开发集、测试集、验证集，各划分的字节数与样本数量详见对应元数据。
语言：
- tr（土耳其语）
许可证：apache-2.0
标签：
- 多任务（multi-task）
- 多任务（multitask）
- MMLU（大规模多任务语言理解基准）
- hendrycks_test（亨德里克斯测试）
任务类别：
- 文本分类
- 多项选择
- 问答任务
任务类型：
- 多项选择问答
- 开放域问答
- 封闭域问答

# mmlu_tr-v0.2 数据集卡片

## 概述
**malhajar/mmlu_tr-v0.2** 是原始**mmlu-tr**数据集的增强版本，专为**[OpenLLMTurkishLeaderboard v0.2](https://huggingface.co/spaces/malhajar/OpenLLMTurkishLeaderboard)** 开发。本数据集通过GPT-4等先进大语言模型（LLM）将原始文本翻译为土耳其语，并附带英文原文用于交叉校验，以确保翻译的准确性与可靠性。该数据集旨在助力土耳其语大语言模型的性能评估，并为自然语言处理（NLP）社区构建可靠的基准测试集。

### 数据集描述
- **源数据集**：[MMLU（大规模多任务语言理解基准）](https://huggingface.co/datasets/tasksource/mmlu)
- **基准测试平台**：[OpenLLMTurkishLeaderboard v0.2](https://huggingface.co/spaces/malhajar/OpenLLMTurkishLeaderboard_v0.2)

### 语言
本数据集文本主体为土耳其语，附带英文原文用于校验与辅助训练评估。

## 数据集结构
### 数据实例
典型的数据实例包含土耳其语问题、多项选择选项与正确答案索引，同时提供每个实例的英文翻译，以支持双语训练与评估。

### 数据字段
- `question_tr`：土耳其语问题文本
- `choices_tr`：土耳其语多项选择选项数组
- `answer_tr`：正确答案在选项数组中的索引
- `question_en`：问题的英文翻译
- `choices_en`：英文多项选择选项数组
- `answer_en`：英文选项数组中正确答案的索引，该索引应与`answer_tr`保持一致

### 数据划分
本数据集划分为三个划分集以适配多样化的训练场景：
- **开发集（dev）**：用于模型调优与初步验证
- **测试集（test）**：用于最终模型评估，模拟模型在未见数据上的表现
- **验证集（validation）**：用于调整模型超参数，避免对测试集过拟合

## 附加信息
### 数据集管理员
本数据集由[穆罕默德·阿尔哈贾尔（Mohamad Alhajar）](https://www.linkedin.com/in/muhammet-alhajar/) 整理，采用GPT-4进行翻译以保障较高的语言质量与内容保真度。

### 授权信息
本数据集采用Apache-2.0许可证发布，可广泛应用于学术研究与商业场景。

### 引用信息
若在研究或应用中使用**mmlu_tr-v0.2**数据集，请按以下格式引用：

@misc{mmlu_tr-v0.2,
author = {Mohamad Alhajar},
title = {mmlu_tr-v0.2},
year = {2024},
publisher = {Mohamad Alhajar},
howpublished = "{https://huggingface.co/datasets/malhajar/mmlu_tr-v0.2}"
}

原始信息汇总

数据集概述

贡献者信息

• 贡献者: Mohamad Alhajar
• 角色:
  • 翻译者
  • 数据管理员

数据集配置

数据集包含多个子类别，每个子类别都有相应的数据文件，分为开发集（dev）、测试集（test）和验证集（validation）。以下是部分子类别的配置示例：

• abstract_algebra
  • 数据文件路径: abstract_algebra/dev-*, abstract_algebra/test-*, abstract_algebra/validation-*
• anatomy
  • 数据文件路径: anatomy/dev-*, anatomy/test-*, anatomy/validation-*
• astronomy
  • 数据文件路径: astronomy/dev-*, astronomy/test-*, astronomy/validation-*
• business_ethics
  • 数据文件路径: business_ethics/dev-*, business_ethics/test-*, business_ethics/validation-*
• clinical_knowledge
  • 数据文件路径: clinical_knowledge/dev-*, clinical_knowledge/test-*, clinical_knowledge/validation-*

数据集特征

每个子类别的数据集特征包括：

• question: 数据类型为字符串
• choices: 数据类型为字符串序列
• answer: 数据类型为int64
• question_eng: 数据类型为字符串
• choices-eng: 数据类型为字符串序列

数据集分割

每个子类别的数据集根据不同的分割（dev, test, validation）有不同的数据量和示例数量。以下是部分子类别的分割信息示例：

• abstract_algebra

  • dev:
    • 字节数: 2010
    • 示例数: 5
  • test:
    • 字节数: 48110
    • 示例数: 100
  • validation:
    • 字节数: 4900
    • 示例数: 11

• anatomy

  • dev:
    • 字节数: 2170
    • 示例数: 5
  • test:
    • 字节数: 70130
    • 示例数: 131
  • validation:
    • 字节数: 7041
    • 示例数: 14

• astronomy

  • dev:
    • 字节数: 4823
    • 示例数: 5
  • test:
    • 字节数: 107489
    • 示例数: 150
  • validation:
    • 字节数: 10712
    • 示例数: 15