malhajar/mmlu_tr-v0.2

---
contributions:
- contributor: Mohamad Alhajar
profile: https://www.linkedin.com/in/muhammet-alhajar/
roles:
- translator
- data curator
configs:
- config_name: abstract_algebra
data_files:
- path: abstract_algebra/dev-*
split: dev
- path: abstract_algebra/test-*
split: test
- path: abstract_algebra/validation-*
split: validation
- config_name: anatomy
data_files:
- path: anatomy/dev-*
split: dev
- path: anatomy/test-*
split: test
- path: anatomy/validation-*
split: validation
- config_name: astronomy
data_files:
- path: astronomy/dev-*
split: dev
- path: astronomy/test-*
split: test
- path: astronomy/validation-*
split: validation
- config_name: business_ethics
data_files:
- path: business_ethics/dev-*
split: dev
- path: business_ethics/test-*
split: test
- path: business_ethics/validation-*
split: validation
- config_name: clinical_knowledge
data_files:
- path: clinical_knowledge/dev-*
split: dev
- path: clinical_knowledge/test-*
split: test
- path: clinical_knowledge/validation-*
split: validation
- config_name: college_biology
data_files:
- path: college_biology/dev-*
split: dev
- path: college_biology/test-*
split: test
- path: college_biology/validation-*
split: validation
- config_name: college_chemistry
data_files:
- path: college_chemistry/dev-*
split: dev
- path: college_chemistry/test-*
split: test
- path: college_chemistry/validation-*
split: validation
- config_name: college_computer_science
data_files:
- path: college_computer_science/dev-*
split: dev
- path: college_computer_science/test-*
split: test
- path: college_computer_science/validation-*
split: validation
- config_name: college_mathematics
data_files:
- path: college_mathematics/dev-*
split: dev
- path: college_mathematics/test-*
split: test
- path: college_mathematics/validation-*
split: validation
- config_name: college_medicine
data_files:
- path: college_medicine/dev-*
split: dev
- path: college_medicine/test-*
split: test
- path: college_medicine/validation-*
split: validation
- config_name: college_physics
data_files:
- path: college_physics/dev-*
split: dev
- path: college_physics/test-*
split: test
- path: college_physics/validation-*
split: validation
- config_name: computer_security
data_files:
- path: computer_security/dev-*
split: dev
- path: computer_security/test-*
split: test
- path: computer_security/validation-*
split: validation
- config_name: conceptual_physics
data_files:
- path: conceptual_physics/dev-*
split: dev
- path: conceptual_physics/test-*
split: test
- path: conceptual_physics/validation-*
split: validation
- config_name: econometrics
data_files:
- path: econometrics/dev-*
split: dev
- path: econometrics/test-*
split: test
- path: econometrics/validation-*
split: validation
- config_name: electrical_engineering
data_files:
- path: electrical_engineering/dev-*
split: dev
- path: electrical_engineering/test-*
split: test
- path: electrical_engineering/validation-*
split: validation
- config_name: elementary_mathematics
data_files:
- path: elementary_mathematics/dev-*
split: dev
- path: elementary_mathematics/test-*
split: test
- path: elementary_mathematics/validation-*
split: validation
- config_name: formal_logic
data_files:
- path: formal_logic/dev-*
split: dev
- path: formal_logic/test-*
split: test
- path: formal_logic/validation-*
split: validation
- config_name: global_facts
data_files:
- path: global_facts/dev-*
split: dev
- path: global_facts/test-*
split: test
- path: global_facts/validation-*
split: validation
- config_name: high_school_biology
data_files:
- path: high_school_biology/dev-*
split: dev
- path: high_school_biology/test-*
split: test
- path: high_school_biology/validation-*
split: validation
- config_name: high_school_chemistry
data_files:
- path: high_school_chemistry/dev-*
split: dev
- path: high_school_chemistry/test-*
split: test
- path: high_school_chemistry/validation-*
split: validation
- config_name: high_school_computer_science
data_files:
- path: high_school_computer_science/dev-*
split: dev
- path: high_school_computer_science/test-*
split: test
- path: high_school_computer_science/validation-*
split: validation
- config_name: high_school_european_history
data_files:
- path: high_school_european_history/dev-*
split: dev
- path: high_school_european_history/test-*
split: test
- path: high_school_european_history/validation-*
split: validation
- config_name: high_school_geography
data_files:
- path: high_school_geography/dev-*
split: dev
- path: high_school_geography/test-*
split: test
- path: high_school_geography/validation-*
split: validation
- config_name: high_school_government_and_politics
data_files:
- path: high_school_government_and_politics/dev-*
split: dev
- path: high_school_government_and_politics/test-*
split: test
- path: high_school_government_and_politics/validation-*
split: validation
- config_name: high_school_macroeconomics
data_files:
- path: high_school_macroeconomics/dev-*
split: dev
- path: high_school_macroeconomics/test-*
split: test
- path: high_school_macroeconomics/validation-*
split: validation
- config_name: high_school_mathematics
data_files:
- path: high_school_mathematics/dev-*
split: dev
- path: high_school_mathematics/test-*
split: test
- path: high_school_mathematics/validation-*
split: validation
- config_name: high_school_microeconomics
data_files:
- path: high_school_microeconomics/dev-*
split: dev
- path: high_school_microeconomics/test-*
split: test
- path: high_school_microeconomics/validation-*
split: validation
- config_name: high_school_physics
data_files:
- path: high_school_physics/dev-*
split: dev
- path: high_school_physics/test-*
split: test
- path: high_school_physics/validation-*
split: validation
- config_name: high_school_psychology
data_files:
- path: high_school_psychology/dev-*
split: dev
- path: high_school_psychology/test-*
split: test
- path: high_school_psychology/validation-*
split: validation
- config_name: high_school_statistics
data_files:
- path: high_school_statistics/dev-*
split: dev
- path: high_school_statistics/test-*
split: test
- path: high_school_statistics/validation-*
split: validation
- config_name: high_school_us_history
data_files:
- path: high_school_us_history/dev-*
split: dev
- path: high_school_us_history/test-*
split: test
- path: high_school_us_history/validation-*
split: validation
- config_name: high_school_world_history
data_files:
- path: high_school_world_history/dev-*
split: dev
- path: high_school_world_history/test-*
split: test
- path: high_school_world_history/validation-*
split: validation
- config_name: human_aging
data_files:
- path: human_aging/dev-*
split: dev
- path: human_aging/test-*
split: test
- path: human_aging/validation-*
split: validation
- config_name: human_sexuality
data_files:
- path: human_sexuality/dev-*
split: dev
- path: human_sexuality/test-*
split: test
- path: human_sexuality/validation-*
split: validation
- config_name: international_law
data_files:
- path: international_law/dev-*
split: dev
- path: international_law/test-*
split: test
- path: international_law/validation-*
split: validation
- config_name: jurisprudence
data_files:
- path: jurisprudence/dev-*
split: dev
- path: jurisprudence/test-*
split: test
- path: jurisprudence/validation-*
split: validation
- config_name: logical_fallacies
data_files:
- path: logical_fallacies/dev-*
split: dev
- path: logical_fallacies/test-*
split: test
- path: logical_fallacies/validation-*
split: validation
- config_name: machine_learning
data_files:
- path: machine_learning/dev-*
split: dev
- path: machine_learning/test-*
split: test
- path: machine_learning/validation-*
split: validation
- config_name: management
data_files:
- path: management/dev-*
split: dev
- path: management/test-*
split: test
- path: management/validation-*
split: validation
- config_name: marketing
data_files:
- path: marketing/dev-*
split: dev
- path: marketing/test-*
split: test
- path: marketing/validation-*
split: validation
- config_name: medical_genetics
data_files:
- path: medical_genetics/dev-*
split: dev
- path: medical_genetics/test-*
split: test
- path: medical_genetics/validation-*
split: validation
- config_name: miscellaneous
data_files:
- path: miscellaneous/dev-*
split: dev
- path: miscellaneous/test-*
split: test
- path: miscellaneous/validation-*
split: validation
- config_name: moral_disputes
data_files:
- path: moral_disputes/dev-*
split: dev
- path: moral_disputes/test-*
split: test
- path: moral_disputes/validation-*
split: validation
- config_name: moral_scenarios
data_files:
- path: moral_scenarios/dev-*
split: dev
- path: moral_scenarios/test-*
split: test
- path: moral_scenarios/validation-*
split: validation
- config_name: nutrition
data_files:
- path: nutrition/dev-*
split: dev
- path: nutrition/test-*
split: test
- path: nutrition/validation-*
split: validation
- config_name: philosophy
data_files:
- path: philosophy/dev-*
split: dev
- path: philosophy/test-*
split: test
- path: philosophy/validation-*
split: validation
- config_name: prehistory
data_files:
- path: prehistory/dev-*
split: dev
- path: prehistory/test-*
split: test
- path: prehistory/validation-*
split: validation
- config_name: professional_accounting
data_files:
- path: professional_accounting/dev-*
split: dev
- path: professional_accounting/test-*
split: test
- path: professional_accounting/validation-*
split: validation
- config_name: professional_law
data_files:
- path: professional_law/dev-*
split: dev
- path: professional_law/test-*
split: test
- path: professional_law/validation-*
split: validation
- config_name: professional_medicine
data_files:
- path: professional_medicine/dev-*
split: dev
- path: professional_medicine/test-*
split: test
- path: professional_medicine/validation-*
split: validation
- config_name: professional_psychology
data_files:
- path: professional_psychology/dev-*
split: dev
- path: professional_psychology/test-*
split: test
- path: professional_psychology/validation-*
split: validation
- config_name: public_relations
data_files:
- path: public_relations/dev-*
split: dev
- path: public_relations/test-*
split: test
- path: public_relations/validation-*
split: validation
- config_name: security_studies
data_files:
- path: security_studies/dev-*
split: dev
- path: security_studies/test-*
split: test
- path: security_studies/validation-*
split: validation
- config_name: sociology
data_files:
- path: sociology/dev-*
split: dev
- path: sociology/test-*
split: test
- path: sociology/validation-*
split: validation
- config_name: us_foreign_policy
data_files:
- path: us_foreign_policy/dev-*
split: dev
- path: us_foreign_policy/test-*
split: test
- path: us_foreign_policy/validation-*
split: validation
- config_name: virology
data_files:
- path: virology/dev-*
split: dev
- path: virology/test-*
split: test
- path: virology/validation-*
split: validation
- config_name: world_religions
data_files:
- path: world_religions/dev-*
split: dev
- path: world_religions/test-*
split: test
- path: world_religions/validation-*
split: validation
dataset_info:
- config_name: abstract_algebra
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2010
num_examples: 5
- name: test
num_bytes: 48110
num_examples: 100
- name: validation
num_bytes: 4900
num_examples: 11
- config_name: anatomy
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2170
num_examples: 5
- name: test
num_bytes: 70130
num_examples: 131
- name: validation
num_bytes: 7041
num_examples: 14
- config_name: astronomy
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 4823
num_examples: 5
- name: test
num_bytes: 107489
num_examples: 150
- name: validation
num_bytes: 10712
num_examples: 15
- config_name: business_ethics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 4960
num_examples: 5
- name: test
num_bytes: 72833
num_examples: 99
- name: validation
num_bytes: 6895
num_examples: 11
- config_name: clinical_knowledge
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2861
num_examples: 5
- name: test
num_bytes: 140864
num_examples: 256
- name: validation
num_bytes: 14851
num_examples: 28
- config_name: college_biology
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2450
num_examples: 4
- name: test
num_bytes: 108187
num_examples: 142
- name: validation
num_bytes: 11068
num_examples: 16
- config_name: college_chemistry
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2945
num_examples: 5
- name: test
num_bytes: 59254
num_examples: 99
- name: validation
num_bytes: 4551
num_examples: 7
- config_name: college_computer_science
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 5911
num_examples: 5
- name: test
num_bytes: 96162
num_examples: 99
- name: validation
num_bytes: 10339
num_examples: 11
- config_name: college_mathematics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3398
num_examples: 5
- name: test
num_bytes: 61015
num_examples: 100
- name: validation
num_bytes: 6527
num_examples: 11
- config_name: college_medicine
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3691
num_examples: 5
- name: test
num_bytes: 155366
num_examples: 168
- name: validation
num_bytes: 17607
num_examples: 22
- config_name: college_physics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2983
num_examples: 5
- name: test
num_bytes: 69270
num_examples: 101
- name: validation
num_bytes: 8330
num_examples: 11
- config_name: computer_security
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2845
num_examples: 5
- name: test
num_bytes: 65752
num_examples: 100
- name: validation
num_bytes: 10520
num_examples: 11
- config_name: conceptual_physics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2236
num_examples: 5
- name: test
num_bytes: 98238
num_examples: 233
- name: validation
num_bytes: 10674
num_examples: 26
- config_name: econometrics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3274
num_examples: 4
- name: test
num_bytes: 102293
num_examples: 114
- name: validation
num_bytes: 11241
num_examples: 12
- config_name: electrical_engineering
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2330
num_examples: 5
- name: test
num_bytes: 60117
num_examples: 144
- name: validation
num_bytes: 8397
num_examples: 16
- config_name: elementary_mathematics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3360
num_examples: 5
- name: test
num_bytes: 166175
num_examples: 373
- name: validation
num_bytes: 21462
num_examples: 40
- config_name: formal_logic
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 7700
num_examples: 5
- name: test
num_bytes: 141194
num_examples: 126
- name: validation
num_bytes: 15041
num_examples: 14
- config_name: global_facts
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2907
num_examples: 5
- name: test
num_bytes: 44241
num_examples: 98
- name: validation
num_bytes: 4412
num_examples: 10
- config_name: high_school_biology
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3613
num_examples: 5
- name: test
num_bytes: 235302
num_examples: 300
- name: validation
num_bytes: 24080
num_examples: 32
- config_name: high_school_chemistry
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2760
num_examples: 5
- name: test
num_bytes: 129230
num_examples: 197
- name: validation
num_bytes: 15225
num_examples: 21
- config_name: high_school_computer_science
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 7130
num_examples: 5
- name: test
num_bytes: 102104
num_examples: 100
- name: validation
num_bytes: 5303
num_examples: 8
- config_name: high_school_european_history
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 22311
num_examples: 5
- name: test
num_bytes: 481301
num_examples: 150
- name: validation
num_bytes: 53102
num_examples: 16
- config_name: high_school_geography
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3813
num_examples: 5
- name: test
num_bytes: 98025
num_examples: 197
- name: validation
num_bytes: 9695
num_examples: 20
- config_name: high_school_government_and_politics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 4037
num_examples: 5
- name: test
num_bytes: 145986
num_examples: 187
- name: validation
num_bytes: 15103
num_examples: 20
- config_name: high_school_macroeconomics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3173
num_examples: 5
- name: test
num_bytes: 262282
num_examples: 390
- name: validation
num_bytes: 28172
num_examples: 42
- config_name: high_school_mathematics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3929
num_examples: 5
- name: test
num_bytes: 146822
num_examples: 270
- name: validation
num_bytes: 14522
num_examples: 28
- config_name: high_school_microeconomics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2707
num_examples: 5
- name: test
num_bytes: 169374
num_examples: 237
- name: validation
num_bytes: 16409
num_examples: 25
- config_name: high_school_physics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3381
num_examples: 5
- name: test
num_bytes: 127963
num_examples: 147
- name: validation
num_bytes: 14540
num_examples: 17
- config_name: high_school_psychology
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 4065
num_examples: 5
- name: test
num_bytes: 350514
num_examples: 533
- name: validation
num_bytes: 38253
num_examples: 58
- config_name: high_school_statistics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 5108
num_examples: 5
- name: test
num_bytes: 240550
num_examples: 216
- name: validation
num_bytes: 21795
num_examples: 23
- config_name: high_school_us_history
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 17832
num_examples: 5
- name: test
num_bytes: 544251
num_examples: 179
- name: validation
num_bytes: 55762
num_examples: 18
- config_name: high_school_world_history
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 10288
num_examples: 5
- name: test
num_bytes: 676637
num_examples: 213
- name: validation
num_bytes: 87320
num_examples: 24
- config_name: human_aging
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2391
num_examples: 5
- name: test
num_bytes: 105704
num_examples: 212
- name: validation
num_bytes: 11471
num_examples: 23
- config_name: human_sexuality
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2480
num_examples: 4
- name: test
num_bytes: 62150
num_examples: 115
- name: validation
num_bytes: 5111
num_examples: 11
- config_name: international_law
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 5325
num_examples: 5
- name: test
num_bytes: 120577
num_examples: 121
- name: validation
num_bytes: 14361
num_examples: 13
- config_name: jurisprudence
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2829
num_examples: 5
- name: test
num_bytes: 74840
num_examples: 106
- name: validation
num_bytes: 8340
num_examples: 11
- config_name: logical_fallacies
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3425
num_examples: 5
- name: test
num_bytes: 113836
num_examples: 161
- name: validation
num_bytes: 11741
num_examples: 18
- config_name: machine_learning
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 5406
num_examples: 5
- name: test
num_bytes: 79515
num_examples: 112
- name: validation
num_bytes: 7467
num_examples: 11
- config_name: management
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2041
num_examples: 5
- name: test
num_bytes: 46345
num_examples: 99
- name: validation
num_bytes: 4231
num_examples: 11
- config_name: marketing
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3411
num_examples: 5
- name: test
num_bytes: 133427
num_examples: 217
- name: validation
num_bytes: 15743
num_examples: 23
- config_name: medical_genetics
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2401
num_examples: 5
- name: test
num_bytes: 46863
num_examples: 95
- name: validation
num_bytes: 6735
num_examples: 11
- config_name: miscellaneous
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 1637
num_examples: 5
- name: test
num_bytes: 358140
num_examples: 766
- name: validation
num_bytes: 34548
num_examples: 86
- config_name: moral_disputes
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3858
num_examples: 5
- name: test
num_bytes: 221059
num_examples: 308
- name: validation
num_bytes: 26723
num_examples: 35
- config_name: moral_scenarios
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3844
num_examples: 4
- name: test
num_bytes: 850011
num_examples: 872
- name: validation
num_bytes: 97469
num_examples: 99
- config_name: nutrition
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 4609
num_examples: 5
- name: test
num_bytes: 210741
num_examples: 305
- name: validation
num_bytes: 19867
num_examples: 33
- config_name: philosophy
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2221
num_examples: 5
- name: test
num_bytes: 175629
num_examples: 299
- name: validation
num_bytes: 20743
num_examples: 34
- config_name: prehistory
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3442
num_examples: 4
- name: test
num_bytes: 191165
num_examples: 300
- name: validation
num_bytes: 20934
num_examples: 31
- config_name: professional_accounting
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 4859
num_examples: 5
- name: test
num_bytes: 274342
num_examples: 279
- name: validation
num_bytes: 31921
num_examples: 31
- config_name: professional_law
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 14021
num_examples: 5
- name: test
num_bytes: 3597412
num_examples: 1388
- name: validation
num_bytes: 363802
num_examples: 145
- config_name: professional_medicine
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 7991
num_examples: 5
- name: test
num_bytes: 435371
num_examples: 261
- name: validation
num_bytes: 48876
num_examples: 30
- config_name: professional_psychology
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 4180
num_examples: 4
- name: test
num_bytes: 492043
num_examples: 594
- name: validation
num_bytes: 61140
num_examples: 67
- config_name: public_relations
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3407
num_examples: 5
- name: test
num_bytes: 63332
num_examples: 108
- name: validation
num_bytes: 10088
num_examples: 12
- config_name: security_studies
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 8391
num_examples: 4
- name: test
num_bytes: 432951
num_examples: 234
- name: validation
num_bytes: 50120
num_examples: 27
- config_name: sociology
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3547
num_examples: 5
- name: test
num_bytes: 146354
num_examples: 195
- name: validation
num_bytes: 14561
num_examples: 19
- config_name: us_foreign_policy
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 3439
num_examples: 4
- name: test
num_bytes: 65685
num_examples: 99
- name: validation
num_bytes: 7602
num_examples: 11
- config_name: virology
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 2675
num_examples: 5
- name: test
num_bytes: 88457
num_examples: 159
- name: validation
num_bytes: 11911
num_examples: 16
- config_name: world_religions
features:
- dtype: string
name: question
- name: choices
sequence: string
- dtype: int64
name: answer
- dtype: string
name: question_eng
- name: choices-eng
sequence: string
splits:
- name: dev
num_bytes: 1746
num_examples: 5
- name: test
num_bytes: 63092
num_examples: 168
- name: validation
num_bytes: 7307
num_examples: 18
language:
- tr
license: apache-2.0
tags:
- multi-task
- multitask
- mmlu
- hendrycks_test
task_categories:
- text-classification
- multiple-choice
- question-answering
task_ids:
- multiple-choice-qa
- open-domain-qa
- closed-domain-qa

---
# Dataset Card for mmlu_tr-v0.2

## Overview

**malhajar/mmlu_tr-v0.2** is an enhanced version of the original **mmlu-tr** dataset, specifically developed for use in the **[OpenLLMTurkishLeaderboard v0.2](https://huggingface.co/spaces/malhajar/OpenLLMTurkishLeaderboard)**. This iteration of the dataset has been translated into Turkish using advanced language models like GPT-4, with English text provided for cross-checking to ensure accuracy and reliability. The dataset is tailored to assist in evaluating the performance of Turkish language models (LLMs) and to establish robust benchmarks within the NLP community.

### Dataset Description

- **Source Dataset:** [mmlu](https://huggingface.co/datasets/tasksource/mmlu)
- **Leaderboard:** [OpenLLMTurkishLeaderboard v0.2](https://huggingface.co/spaces/malhajar/OpenLLMTurkishLeaderboard_v0.2)

### Languages

The text in the dataset is primarily in Turkish, with auxiliary English text for validation and cross-checking purposes.

## Dataset Structure

### Data Instances

A typical data instance comprises a question in Turkish, multiple choices, and an answer. English translations are provided for each instance to facilitate bilingual training and evaluation.

### Data Fields

- `question_tr`: the question text in Turkish.
- `choices_tr`: an array of multiple choice options in Turkish.
- `answer_tr`: the index of the correct answer in the choices array.
- `question_en`: the English translation of the question.
- `choices_en`: an array of multiple choice options in English.
- `answer_en`: the index of the correct answer in the English choices array, which should match `answer_tr`.

### Data Splits

The dataset is divided into three splits to support diverse training scenarios:
- **Development (dev)**: Used for model tuning and validation.
- **Test**: Used for final model evaluation to simulate performance on unseen data.
- **Validation**: Additional split for adjusting model hyperparameters without overfitting the test data.

## Additional Information

### Dataset Curator

The dataset was curated by [`Mohamad Alhajar`](https://www.linkedin.com/in/muhammet-alhajar/) , leveraging GPT-4 for translations to ensure high linguistic quality and fidelity.

### Licensing Information

The dataset is available under the Apache-2.0 license, allowing for wide distribution and use in both academic and commercial settings.

### Citation Information

If you use the **mmlu_tr-v0.2** dataset in your research or application, please cite it as follows:

```
@misc{mmlu_tr-v0.2,
author = {Mohamad Alhajar},
title = {mmlu_tr-v0.2},
year = {2024},
publisher = {Mohamad Alhajar},
howpublished = "{https://huggingface.co/datasets/malhajar/mmlu_tr-v0.2}"
}
```

贡献信息：
- 贡献者：穆罕默德·阿尔哈贾尔（Mohamad Alhajar）
个人主页：https://www.linkedin.com/in/muhammet-alhajar/
角色：
- 译员
- 数据管理员
配置项：
所有配置项均对应一个垂直学科领域，每个配置项包含开发集、测试集、验证集三个数据划分，文件路径格式为`{配置名称}/划分前缀-*`，配置名称包括抽象代数、解剖学、天文学、商业伦理、临床知识、大学生物学、大学化学、大学计算机科学、大学数学、大学医学、大学物理学、计算机安全、概念物理学、计量经济学、电气工程、初等数学、形式逻辑、全球事实、高中生物学、高中化学、高中计算机科学、高中欧洲历史、高中地理学、高中政府与政治、高中宏观经济学、高中数学、高中微观经济学、高中物理学、高中心理学、高中统计学、高中美国历史、高中世界历史、人类衰老、人类性学、国际法、法理学、逻辑谬误、机器学习、管理学、市场营销、医学遗传学、综合类目、道德争议、道德场景、营养学、哲学、史前史、专业会计学、专业法学、专业医学、专业心理学、公共关系、安全研究、社会学、美国外交政策、病毒学、世界宗教。
数据集信息：
每个配置项的数据字段均包含：
- 数据类型：字符串，字段名：问题
- 字段名：选项，类型：字符串序列
- 数据类型：64位整数，字段名：答案
- 数据类型：字符串，字段名：英文问题
- 字段名：英文选项，类型：字符串序列
各配置项对应的数据划分包含开发集、测试集、验证集，各划分的字节数与样本数量详见对应元数据。
语言：
- tr（土耳其语）
许可证：apache-2.0
标签：
- 多任务（multi-task）
- 多任务（multitask）
- MMLU（大规模多任务语言理解基准）
- hendrycks_test（亨德里克斯测试）
任务类别：
- 文本分类
- 多项选择
- 问答任务
任务类型：
- 多项选择问答
- 开放域问答
- 封闭域问答

# mmlu_tr-v0.2 数据集卡片

## 概述
**malhajar/mmlu_tr-v0.2** 是原始**mmlu-tr**数据集的增强版本，专为**[OpenLLMTurkishLeaderboard v0.2](https://huggingface.co/spaces/malhajar/OpenLLMTurkishLeaderboard)** 开发。本数据集通过GPT-4等先进大语言模型（LLM）将原始文本翻译为土耳其语，并附带英文原文用于交叉校验，以确保翻译的准确性与可靠性。该数据集旨在助力土耳其语大语言模型的性能评估，并为自然语言处理（NLP）社区构建可靠的基准测试集。

### 数据集描述
- **源数据集**：[MMLU（大规模多任务语言理解基准）](https://huggingface.co/datasets/tasksource/mmlu)
- **基准测试平台**：[OpenLLMTurkishLeaderboard v0.2](https://huggingface.co/spaces/malhajar/OpenLLMTurkishLeaderboard_v0.2)

### 语言
本数据集文本主体为土耳其语，附带英文原文用于校验与辅助训练评估。

## 数据集结构
### 数据实例
典型的数据实例包含土耳其语问题、多项选择选项与正确答案索引，同时提供每个实例的英文翻译，以支持双语训练与评估。

### 数据字段
- `question_tr`：土耳其语问题文本
- `choices_tr`：土耳其语多项选择选项数组
- `answer_tr`：正确答案在选项数组中的索引
- `question_en`：问题的英文翻译
- `choices_en`：英文多项选择选项数组
- `answer_en`：英文选项数组中正确答案的索引，该索引应与`answer_tr`保持一致

### 数据划分
本数据集划分为三个划分集以适配多样化的训练场景：
- **开发集（dev）**：用于模型调优与初步验证
- **测试集（test）**：用于最终模型评估，模拟模型在未见数据上的表现
- **验证集（validation）**：用于调整模型超参数，避免对测试集过拟合

## 附加信息
### 数据集管理员
本数据集由[穆罕默德·阿尔哈贾尔（Mohamad Alhajar）](https://www.linkedin.com/in/muhammet-alhajar/) 整理，采用GPT-4进行翻译以保障较高的语言质量与内容保真度。

### 授权信息
本数据集采用Apache-2.0许可证发布，可广泛应用于学术研究与商业场景。

### 引用信息
若在研究或应用中使用**mmlu_tr-v0.2**数据集，请按以下格式引用：

@misc{mmlu_tr-v0.2,
author = {Mohamad Alhajar},
title = {mmlu_tr-v0.2},
year = {2024},
publisher = {Mohamad Alhajar},
howpublished = "{https://huggingface.co/datasets/malhajar/mmlu_tr-v0.2}"
}

原始信息汇总

数据集概述

贡献者信息

• 贡献者: Mohamad Alhajar
• 角色:
  • 翻译者
  • 数据管理员

数据集配置

数据集包含多个子类别，每个子类别都有相应的数据文件，分为开发集（dev）、测试集（test）和验证集（validation）。以下是部分子类别的配置示例：

• abstract_algebra
  • 数据文件路径: abstract_algebra/dev-*, abstract_algebra/test-*, abstract_algebra/validation-*
• anatomy
  • 数据文件路径: anatomy/dev-*, anatomy/test-*, anatomy/validation-*
• astronomy
  • 数据文件路径: astronomy/dev-*, astronomy/test-*, astronomy/validation-*
• business_ethics
  • 数据文件路径: business_ethics/dev-*, business_ethics/test-*, business_ethics/validation-*
• clinical_knowledge
  • 数据文件路径: clinical_knowledge/dev-*, clinical_knowledge/test-*, clinical_knowledge/validation-*

数据集特征

每个子类别的数据集特征包括：

• question: 数据类型为字符串
• choices: 数据类型为字符串序列
• answer: 数据类型为int64
• question_eng: 数据类型为字符串
• choices-eng: 数据类型为字符串序列

数据集分割

每个子类别的数据集根据不同的分割（dev, test, validation）有不同的数据量和示例数量。以下是部分子类别的分割信息示例：

• abstract_algebra

  • dev:
    • 字节数: 2010
    • 示例数: 5
  • test:
    • 字节数: 48110
    • 示例数: 100
  • validation:
    • 字节数: 4900
    • 示例数: 11

• anatomy

  • dev:
    • 字节数: 2170
    • 示例数: 5
  • test:
    • 字节数: 70130
    • 示例数: 131
  • validation:
    • 字节数: 7041
    • 示例数: 14

• astronomy

  • dev:
    • 字节数: 4823
    • 示例数: 5
  • test:
    • 字节数: 107489
    • 示例数: 150
  • validation:
    • 字节数: 10712
    • 示例数: 15

搜集汇总

数据集介绍

构建方式

在自然语言处理与多语言知识评估的交叉领域中，数据集的多语种适配是拓展模型泛化能力的关键。malhajar/mmlu_tr-v0.2 数据集正是基于广泛使用的 MMLU（Massive Multitask Language Understanding）基准，通过系统性的翻译与数据整理流程构建而成。该数据集涵盖了从抽象代数、解剖学到世界宗教等 57 个学科领域，每个学科均被组织为独立的配置项。在构建过程中，原始英文问题及选项被精准翻译为土耳其语，同时保留了英文原版内容作为对照，形成了包含问题、选项、答案及对应英文版本的完整结构化特征。数据划分严格遵循 MMLU 的原始分割策略，每个学科均设有 dev、test 和 validation 三个子集，其中 dev 集通常包含 4-5 个样本用于示例，test 集规模较大，validation 集则提供适中的验证样本，从而确保了评估流程的规范性与一致性。

特点

该数据集最显著的特点在于其多学科覆盖的广度与双语对齐的深度。数据集横跨自然科学、社会科学、人文学科及专业领域，从基础的高中生物学到高深的大学数学，从临床知识到国际法，构建了一个全面的知识评估体系。每个样本均包含土耳其语问题、土耳其语选项、正确答案索引，以及对应的英文问题与选项，这种结构使得数据集天然支持跨语言迁移学习与对比分析。此外，数据集的规模设计富有策略性：dev 集规模极小，适用于快速调试与少样本学习；test 集包含大量样本，能够提供统计显著的评估结果；validation 集则平衡了开发与测试的需求。这种三分割设计结合双语特性，使得该数据集不仅适用于土耳其语模型的评估，也为多语言理解模型的泛化能力研究提供了宝贵的资源。

使用方法

使用该数据集进行模型评估时，研究人员可通过 Hugging Face Datasets 库便捷地加载任意学科的子集。加载时需指定具体的配置名称，例如 'abstract_algebra' 或 'clinical_knowledge'，并通过 split 参数选择 'dev'、'test' 或 'validation' 分割。每个样本的结构清晰，包含 question（土耳其语问题）、choices（土耳其语选项列表）、answer（正确答案索引）以及对应的英文字段。在评估流程中，模型需根据给定的问题与选项输出预测答案索引，并与 ground truth 对比计算准确率。由于数据集覆盖学科广泛，建议针对不同学科分别评估，或采用加权平均的方式获得综合性能指标。双语字段的存在还允许研究者设计跨语言提示策略，例如使用英文问题引导模型生成土耳其语答案，从而深入探索模型的语言对齐能力。

背景与挑战

背景概述

大规模多任务语言理解（MMLU）基准测试自2020年由Hendrycks等人提出以来，已成为评估大语言模型知识与推理能力的核心标杆。malhajar/mmlu_tr-v0.2数据集由独立研究者Mohamad Alhajar于2023年主导创建，其核心使命是将原版MMLU的57个学科领域（涵盖从抽象代数、医学遗传学到美国外交政策等广泛主题）翻译为土耳其语，旨在填补非英语语言环境下多学科知识评估的空白。该数据集保留了原始的三段式划分（开发集、测试集、验证集），每个子集均包含问题原文、土耳其语翻译、选项及其对应的英文版本，为跨语言模型能力对比提供了关键资源。通过将经典基准测试拓展至土耳其语，该数据集不仅推动了多语言NLP研究的发展，更对评估模型在低资源语言上的泛化能力产生了深远影响。

当前挑战

该数据集面临的核心挑战在于多学科知识翻译的准确性与一致性。一方面，57个学科涵盖大量专业术语（如临床知识中的医学术语、形式逻辑中的符号表达），如何确保土耳其语译文既忠实于原意又符合本地学术规范，是构建过程中需要克服的难题。另一方面，原始MMLU题目本身包含文化特定概念（如美国历史与外交政策），这些内容在跨语言迁移时可能因文化差异导致理解偏差，从而影响评估的公平性。此外，数据集构建者仅凭个人之力完成如此庞大的翻译与整理工作，缺乏多轮专家校对与大规模众包验证，可能引入噪声或错误，对后续模型评测结果的可靠性构成潜在威胁。

常用场景

经典使用场景

MMLU-TR（土耳其语版大规模多任务语言理解数据集）作为MMLU的多语言扩展，其核心应用场景在于评估和比较预训练语言模型在土耳其语环境下的知识广度与推理能力。该数据集涵盖了从抽象代数、解剖学到世界宗教等57个学科领域，通过选择题形式检验模型在零样本或少样本条件下的学术知识掌握程度。研究者通常利用该数据集对模型进行跨语言泛化能力测试，尤其关注非英语语种中模型的知识迁移效果，为多语言自然语言处理研究提供了标准化的评估基准。

衍生相关工作

MMLU-TR的出现催生了多项相关研究工作，包括土耳其语大语言模型的专项预训练与微调策略探索，如基于该数据集开发的TurkuNLP系列模型在土耳其语理解任务上的性能优化。研究者还借鉴其构建思路，推出了阿拉伯语、俄语等版本的MMLU多语言扩展数据集，形成了跨语言知识评估体系。同时，该数据集被用于分析模型在翻译质量与知识保留之间的权衡关系，衍生出针对土耳其语提示工程和上下文学习能力的系统研究，为多语言模型的可解释性分析提供了实证基础。

数据集最近研究

最新研究方向

在大型语言模型多语言评估的浪潮中，malhajar/mmlu_tr-v0.2 数据集应运而生，它聚焦于将广受欢迎的 MMLU 基准测试进行土耳其语翻译与本地化，旨在为非英语语境下的模型知识推理能力提供精准度量。该数据集覆盖了从抽象代数到病毒学等 57 个学科领域，其核心研究前沿在于探索语言模型在多语言、多学科环境中的泛化表现与知识迁移效率。当前，围绕此数据集的热点事件包括对 GPT-4、Claude 等前沿模型在土耳其语知识问答上的鲁棒性测试，以及对比评估中英文模型在专业领域上的性能差异。这一工作不仅推动了多语言 NLP 评估体系的完善，更对构建包容性更强的全球人工智能基础设施具有深远影响，为跨语言知识库的构建与教育科技的落地提供了关键数据支撑。