Name: marlosb/mmlu-pt
Creator: marlosb
Published: 2026-03-06 16:59:29
License: 暂无描述

下载链接：

https://hf-mirror.com/datasets/marlosb/mmlu-pt

下载链接

链接失效反馈

官方服务：

资源简介：

--- annotations_creators: - no-annotation language_creators: - expert-generated language: - pt license: - mit multilinguality: - monolingual size_categories: - 10K<n<100K source_datasets: - cais/mmlu task_categories: - question-answering task_ids: - multiple-choice-qa paperswithcode_id: mmlu pretty_name: Measuring Massive Multitask Language Understanding dataset_info: - config_name: abstract_algebra features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 23297 num_examples: 100 - name: validation num_bytes: 2442 num_examples: 11 - name: dev num_bytes: 1023 num_examples: 5 download_size: 16469 dataset_size: 26762 - config_name: all features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 7258819 num_examples: 13932 - name: validation num_bytes: 790536 num_examples: 1515 - name: dev num_bytes: 133502 num_examples: 285 - name: auxiliary_train num_bytes: 165324901 num_examples: 99382 download_size: 69046842 dataset_size: 173507758 - config_name: anatomy features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 35991 num_examples: 135 - name: validation num_bytes: 3375 num_examples: 14 - name: dev num_bytes: 1081 num_examples: 5 download_size: 28952 dataset_size: 40447 - config_name: astronomy features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 52810 num_examples: 152 - name: validation num_bytes: 5620 num_examples: 16 - name: dev num_bytes: 2404 num_examples: 5 download_size: 40858 dataset_size: 60834 - config_name: auxiliary_train features: - name: train struct: - name: answer dtype: int64 - name: choices list: string - name: question dtype: string - name: subject dtype: string splits: - name: train num_bytes: 166278434 num_examples: 99765 download_size: 65038561 dataset_size: 166278434 - config_name: business_ethics features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 38455 num_examples: 100 - name: validation num_bytes: 3477 num_examples: 11 - name: dev num_bytes: 2448 num_examples: 5 download_size: 31480 dataset_size: 44380 - config_name: clinical_knowledge features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 75192 num_examples: 265 - name: validation num_bytes: 7973 num_examples: 29 - name: dev num_bytes: 1408 num_examples: 5 download_size: 53810 dataset_size: 84573 - config_name: college_biology features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 55720 num_examples: 144 - name: validation num_bytes: 5635 num_examples: 16 - name: dev num_bytes: 1761 num_examples: 5 download_size: 45426 dataset_size: 63116 - config_name: college_chemistry features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 28587 num_examples: 100 - name: validation num_bytes: 2605 num_examples: 8 - name: dev num_bytes: 1530 num_examples: 5 download_size: 27296 dataset_size: 32722 - config_name: college_computer_science features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 49652 num_examples: 100 - name: validation num_bytes: 5475 num_examples: 11 - name: dev num_bytes: 3133 num_examples: 5 download_size: 42529 dataset_size: 58260 - config_name: college_mathematics features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 28329 num_examples: 100 - name: validation num_bytes: 3070 num_examples: 11 - name: dev num_bytes: 1654 num_examples: 5 download_size: 27272 dataset_size: 33053 - config_name: college_medicine features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 92218 num_examples: 173 - name: validation num_bytes: 9006 num_examples: 22 - name: dev num_bytes: 1989 num_examples: 5 download_size: 60326 dataset_size: 103213 - config_name: college_physics features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 33976 num_examples: 102 - name: validation num_bytes: 3880 num_examples: 11 - name: dev num_bytes: 1559 num_examples: 5 download_size: 29089 dataset_size: 39415 - config_name: computer_security features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 31631 num_examples: 99 - name: validation num_bytes: 5197 num_examples: 11 - name: dev num_bytes: 1331 num_examples: 5 download_size: 31003 dataset_size: 38159 - config_name: conceptual_physics features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 49376 num_examples: 235 - name: validation num_bytes: 5435 num_examples: 26 - name: dev num_bytes: 1107 num_examples: 5 download_size: 35684 dataset_size: 55918 - config_name: econometrics features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 50052 num_examples: 112 - name: validation num_bytes: 5587 num_examples: 12 - name: dev num_bytes: 1967 num_examples: 5 download_size: 35813 dataset_size: 57606 - config_name: electrical_engineering features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 31464 num_examples: 144 - name: validation num_bytes: 3557 num_examples: 16 - name: dev num_bytes: 1199 num_examples: 5 download_size: 26786 dataset_size: 36220 - config_name: elementary_mathematics features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 83604 num_examples: 378 - name: validation num_bytes: 10463 num_examples: 41 - name: dev num_bytes: 1616 num_examples: 5 download_size: 55396 dataset_size: 95683 - config_name: formal_logic features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 53606 num_examples: 124 - name: validation num_bytes: 6995 num_examples: 14 download_size: 28491 dataset_size: 60601 - config_name: global_facts features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 21446 num_examples: 100 - name: validation num_bytes: 2101 num_examples: 10 - name: dev num_bytes: 1337 num_examples: 5 download_size: 19470 dataset_size: 24884 - config_name: high_school_biology features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 125015 num_examples: 309 - name: validation num_bytes: 12468 num_examples: 32 - name: dev num_bytes: 1951 num_examples: 5 download_size: 81991 dataset_size: 139434 - config_name: high_school_chemistry features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 66519 num_examples: 202 - name: validation num_bytes: 8099 num_examples: 22 - name: dev num_bytes: 1429 num_examples: 5 download_size: 47007 dataset_size: 76047 - config_name: high_school_computer_science features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 51167 num_examples: 100 - name: validation num_bytes: 3905 num_examples: 9 - name: dev num_bytes: 3248 num_examples: 5 download_size: 41105 dataset_size: 58320 - config_name: high_school_european_history features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 280734 num_examples: 165 - name: validation num_bytes: 30981 num_examples: 18 download_size: 185271 dataset_size: 311715 - config_name: high_school_geography features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 50889 num_examples: 198 - name: validation num_bytes: 5238 num_examples: 22 - name: dev num_bytes: 1753 num_examples: 5 download_size: 40048 dataset_size: 57880 - config_name: high_school_government_and_politics features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 79703 num_examples: 193 - name: validation num_bytes: 8407 num_examples: 21 - name: dev num_bytes: 2085 num_examples: 5 download_size: 55114 dataset_size: 90195 - config_name: high_school_macroeconomics features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 140820 num_examples: 390 - name: validation num_bytes: 15405 num_examples: 43 - name: dev num_bytes: 1631 num_examples: 5 download_size: 74367 dataset_size: 157856 - config_name: high_school_mathematics features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 64552 num_examples: 270 - name: validation num_bytes: 6838 num_examples: 29 - name: dev num_bytes: 1422 num_examples: 5 download_size: 45516 dataset_size: 72812 - config_name: high_school_microeconomics features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 91019 num_examples: 238 - name: validation num_bytes: 9189 num_examples: 26 - name: dev num_bytes: 1504 num_examples: 5 download_size: 52634 dataset_size: 101712 - config_name: high_school_physics features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 65623 num_examples: 151 - name: validation num_bytes: 7596 num_examples: 17 - name: dev num_bytes: 1696 num_examples: 5 download_size: 45862 dataset_size: 74915 - config_name: high_school_psychology features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 187613 num_examples: 545 - name: validation num_bytes: 20350 num_examples: 60 - name: dev num_bytes: 2143 num_examples: 5 download_size: 120414 dataset_size: 210106 - config_name: high_school_statistics features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 125681 num_examples: 216 - name: validation num_bytes: 11254 num_examples: 23 - name: dev num_bytes: 2667 num_examples: 5 download_size: 78239 dataset_size: 139602 - config_name: high_school_us_history features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 307878 num_examples: 201 - name: validation num_bytes: 33518 num_examples: 22 - name: dev num_bytes: 9442 num_examples: 5 download_size: 211283 dataset_size: 350838 - config_name: high_school_world_history features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 396109 num_examples: 236 - name: validation num_bytes: 47997 num_examples: 26 - name: dev num_bytes: 5111 num_examples: 5 download_size: 264511 dataset_size: 449217 - config_name: human_aging features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 54044 num_examples: 223 - name: validation num_bytes: 5613 num_examples: 23 - name: dev num_bytes: 1195 num_examples: 5 download_size: 43144 dataset_size: 60852 - config_name: human_sexuality features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 36795 num_examples: 129 - name: validation num_bytes: 2795 num_examples: 12 - name: dev num_bytes: 1231 num_examples: 5 download_size: 31847 dataset_size: 40821 - config_name: international_law features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 60757 num_examples: 121 - name: validation num_bytes: 7087 num_examples: 13 - name: dev num_bytes: 2676 num_examples: 5 download_size: 42528 dataset_size: 70520 - config_name: jurisprudence features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 38421 num_examples: 108 - name: validation num_bytes: 4167 num_examples: 11 - name: dev num_bytes: 1407 num_examples: 5 download_size: 34210 dataset_size: 43995 - config_name: logical_fallacies features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 56518 num_examples: 163 - name: validation num_bytes: 5768 num_examples: 18 - name: dev num_bytes: 1748 num_examples: 5 download_size: 35399 dataset_size: 64034 - config_name: machine_learning features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 40286 num_examples: 112 - name: validation num_bytes: 3769 num_examples: 11 - name: dev num_bytes: 2742 num_examples: 5 download_size: 31141 dataset_size: 46797 - config_name: management features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 23345 num_examples: 103 - name: validation num_bytes: 2114 num_examples: 11 - name: dev num_bytes: 1004 num_examples: 5 download_size: 22110 dataset_size: 26463 - config_name: marketing features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 72401 num_examples: 234 - name: validation num_bytes: 8467 num_examples: 25 - name: dev num_bytes: 1808 num_examples: 5 download_size: 51926 dataset_size: 82676 - config_name: medical_genetics features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 24611 num_examples: 100 - name: validation num_bytes: 3479 num_examples: 11 - name: dev num_bytes: 1279 num_examples: 5 download_size: 24904 dataset_size: 29369 - config_name: miscellaneous features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 171574 num_examples: 783 - name: validation num_bytes: 16761 num_examples: 86 - name: dev num_bytes: 805 num_examples: 5 download_size: 122202 dataset_size: 189140 - config_name: moral_disputes features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 121209 num_examples: 345 - name: validation num_bytes: 13777 num_examples: 38 - name: dev num_bytes: 1884 num_examples: 5 download_size: 79314 dataset_size: 136870 - config_name: moral_scenarios features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 415594 num_examples: 893 - name: validation num_bytes: 46926 num_examples: 100 - name: dev num_bytes: 2265 num_examples: 5 download_size: 123961 dataset_size: 464785 - config_name: nutrition features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 106429 num_examples: 305 - name: validation num_bytes: 9712 num_examples: 33 - name: dev num_bytes: 2343 num_examples: 5 download_size: 71396 dataset_size: 118484 - config_name: philosophy features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 89202 num_examples: 310 - name: validation num_bytes: 10124 num_examples: 34 - name: dev num_bytes: 1112 num_examples: 5 download_size: 62225 dataset_size: 100438 - config_name: prehistory features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 101001 num_examples: 323 - name: validation num_bytes: 11552 num_examples: 35 - name: dev num_bytes: 2126 num_examples: 5 download_size: 72297 dataset_size: 114679 - config_name: professional_accounting features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 145398 num_examples: 282 - name: validation num_bytes: 16607 num_examples: 31 - name: dev num_bytes: 2510 num_examples: 5 download_size: 93278 dataset_size: 164515 - config_name: professional_law features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 1980353 num_examples: 1512 - name: validation num_bytes: 210730 num_examples: 166 - name: dev num_bytes: 7026 num_examples: 5 download_size: 1198069 dataset_size: 2198109 - config_name: professional_medicine features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 233333 num_examples: 271 - name: validation num_bytes: 25866 num_examples: 31 - name: dev num_bytes: 4107 num_examples: 5 download_size: 159350 dataset_size: 263306 - config_name: professional_psychology features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 259846 num_examples: 612 - name: validation num_bytes: 33049 num_examples: 69 - name: dev num_bytes: 2478 num_examples: 5 download_size: 169706 dataset_size: 295373 - config_name: public_relations features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 33842 num_examples: 110 - name: validation num_bytes: 5188 num_examples: 12 - name: dev num_bytes: 1751 num_examples: 5 download_size: 31556 dataset_size: 40781 - config_name: security_studies features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 222531 num_examples: 241 - name: validation num_bytes: 24852 num_examples: 27 - name: dev num_bytes: 5779 num_examples: 5 download_size: 143477 dataset_size: 253162 - config_name: sociology features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 74937 num_examples: 201 - name: validation num_bytes: 8147 num_examples: 22 - name: dev num_bytes: 1848 num_examples: 5 download_size: 58745 dataset_size: 84932 - config_name: us_foreign_policy features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 32963 num_examples: 100 - name: validation num_bytes: 3753 num_examples: 11 - name: dev num_bytes: 1887 num_examples: 5 download_size: 29694 dataset_size: 38603 - config_name: virology features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 44552 num_examples: 166 - name: validation num_bytes: 6244 num_examples: 18 - name: dev num_bytes: 1263 num_examples: 5 download_size: 38365 dataset_size: 52059 - config_name: world_religions features: - name: question dtype: string - name: subject dtype: string - name: choices list: string - name: answer dtype: class_label: names: '0': A '1': B '2': C '3': D splits: - name: test num_bytes: 29091 num_examples: 171 - name: validation num_bytes: 3177 num_examples: 19 - name: dev num_bytes: 766 num_examples: 5 download_size: 26667 dataset_size: 33034 configs: - config_name: abstract_algebra data_files: - split: test path: abstract_algebra/test-* - split: validation path: abstract_algebra/validation-* - split: dev path: abstract_algebra/dev-* - config_name: all data_files: - split: test path: all/test-* - split: validation path: all/validation-* - split: dev path: all/dev-* - split: auxiliary_train path: all/auxiliary_train-* - config_name: anatomy data_files: - split: test path: anatomy/test-* - split: validation path: anatomy/validation-* - split: dev path: anatomy/dev-* - config_name: astronomy data_files: - split: test path: astronomy/test-* - split: validation path: astronomy/validation-* - split: dev path: astronomy/dev-* - config_name: auxiliary_train data_files: - split: train path: auxiliary_train/train-* - config_name: business_ethics data_files: - split: test path: business_ethics/test-* - split: validation path: business_ethics/validation-* - split: dev path: business_ethics/dev-* - config_name: clinical_knowledge data_files: - split: test path: clinical_knowledge/test-* - split: validation path: clinical_knowledge/validation-* - split: dev path: clinical_knowledge/dev-* - config_name: college_biology data_files: - split: test path: college_biology/test-* - split: validation path: college_biology/validation-* - split: dev path: college_biology/dev-* - config_name: college_chemistry data_files: - split: test path: college_chemistry/test-* - split: validation path: college_chemistry/validation-* - split: dev path: college_chemistry/dev-* - config_name: college_computer_science data_files: - split: test path: college_computer_science/test-* - split: validation path: college_computer_science/validation-* - split: dev path: college_computer_science/dev-* - config_name: college_mathematics data_files: - split: test path: college_mathematics/test-* - split: validation path: college_mathematics/validation-* - split: dev path: college_mathematics/dev-* - config_name: college_medicine data_files: - split: test path: college_medicine/test-* - split: validation path: college_medicine/validation-* - split: dev path: college_medicine/dev-* - config_name: college_physics data_files: - split: test path: college_physics/test-* - split: validation path: college_physics/validation-* - split: dev path: college_physics/dev-* - config_name: computer_security data_files: - split: test path: computer_security/test-* - split: validation path: computer_security/validation-* - split: dev path: computer_security/dev-* - config_name: conceptual_physics data_files: - split: test path: conceptual_physics/test-* - split: validation path: conceptual_physics/validation-* - split: dev path: conceptual_physics/dev-* - config_name: econometrics data_files: - split: test path: econometrics/test-* - split: validation path: econometrics/validation-* - split: dev path: econometrics/dev-* - config_name: electrical_engineering data_files: - split: test path: electrical_engineering/test-* - split: validation path: electrical_engineering/validation-* - split: dev path: electrical_engineering/dev-* - config_name: elementary_mathematics data_files: - split: test path: elementary_mathematics/test-* - split: validation path: elementary_mathematics/validation-* - split: dev path: elementary_mathematics/dev-* - config_name: formal_logic data_files: - split: test path: formal_logic/test-* - split: validation path: formal_logic/validation-* - config_name: global_facts data_files: - split: test path: global_facts/test-* - split: validation path: global_facts/validation-* - split: dev path: global_facts/dev-* - config_name: high_school_biology data_files: - split: test path: high_school_biology/test-* - split: validation path: high_school_biology/validation-* - split: dev path: high_school_biology/dev-* - config_name: high_school_chemistry data_files: - split: test path: high_school_chemistry/test-* - split: validation path: high_school_chemistry/validation-* - split: dev path: high_school_chemistry/dev-* - config_name: high_school_computer_science data_files: - split: test path: high_school_computer_science/test-* - split: validation path: high_school_computer_science/validation-* - split: dev path: high_school_computer_science/dev-* - config_name: high_school_european_history data_files: - split: test path: high_school_european_history/test-* - split: validation path: high_school_european_history/validation-* - config_name: high_school_geography data_files: - split: test path: high_school_geography/test-* - split: validation path: high_school_geography/validation-* - split: dev path: high_school_geography/dev-* - config_name: high_school_government_and_politics data_files: - split: test path: high_school_government_and_politics/test-* - split: validation path: high_school_government_and_politics/validation-* - split: dev path: high_school_government_and_politics/dev-* - config_name: high_school_macroeconomics data_files: - split: test path: high_school_macroeconomics/test-* - split: validation path: high_school_macroeconomics/validation-* - split: dev path: high_school_macroeconomics/dev-* - config_name: high_school_mathematics data_files: - split: test path: high_school_mathematics/test-* - split: validation path: high_school_mathematics/validation-* - split: dev path: high_school_mathematics/dev-* - config_name: high_school_microeconomics data_files: - split: test path: high_school_microeconomics/test-* - split: validation path: high_school_microeconomics/validation-* - split: dev path: high_school_microeconomics/dev-* - config_name: high_school_physics data_files: - split: test path: high_school_physics/test-* - split: validation path: high_school_physics/validation-* - split: dev path: high_school_physics/dev-* - config_name: high_school_psychology data_files: - split: test path: high_school_psychology/test-* - split: validation path: high_school_psychology/validation-* - split: dev path: high_school_psychology/dev-* - config_name: high_school_statistics data_files: - split: test path: high_school_statistics/test-* - split: validation path: high_school_statistics/validation-* - split: dev path: high_school_statistics/dev-* - config_name: high_school_us_history data_files: - split: test path: high_school_us_history/test-* - split: validation path: high_school_us_history/validation-* - split: dev path: high_school_us_history/dev-* - config_name: high_school_world_history data_files: - split: test path: high_school_world_history/test-* - split: validation path: high_school_world_history/validation-* - split: dev path: high_school_world_history/dev-* - config_name: human_aging data_files: - split: test path: human_aging/test-* - split: validation path: human_aging/validation-* - split: dev path: human_aging/dev-* - config_name: human_sexuality data_files: - split: test path: human_sexuality/test-* - split: validation path: human_sexuality/validation-* - split: dev path: human_sexuality/dev-* - config_name: international_law data_files: - split: test path: international_law/test-* - split: validation path: international_law/validation-* - split: dev path: international_law/dev-* - config_name: jurisprudence data_files: - split: test path: jurisprudence/test-* - split: validation path: jurisprudence/validation-* - split: dev path: jurisprudence/dev-* - config_name: logical_fallacies data_files: - split: test path: logical_fallacies/test-* - split: validation path: logical_fallacies/validation-* - split: dev path: logical_fallacies/dev-* - config_name: machine_learning data_files: - split: test path: machine_learning/test-* - split: validation path: machine_learning/validation-* - split: dev path: machine_learning/dev-* - config_name: management data_files: - split: test path: management/test-* - split: validation path: management/validation-* - split: dev path: management/dev-* - config_name: marketing data_files: - split: test path: marketing/test-* - split: validation path: marketing/validation-* - split: dev path: marketing/dev-* - config_name: medical_genetics data_files: - split: test path: medical_genetics/test-* - split: validation path: medical_genetics/validation-* - split: dev path: medical_genetics/dev-* - config_name: miscellaneous data_files: - split: test path: miscellaneous/test-* - split: validation path: miscellaneous/validation-* - split: dev path: miscellaneous/dev-* - config_name: moral_disputes data_files: - split: test path: moral_disputes/test-* - split: validation path: moral_disputes/validation-* - split: dev path: moral_disputes/dev-* - config_name: moral_scenarios data_files: - split: test path: moral_scenarios/test-* - split: validation path: moral_scenarios/validation-* - split: dev path: moral_scenarios/dev-* - config_name: nutrition data_files: - split: test path: nutrition/test-* - split: validation path: nutrition/validation-* - split: dev path: nutrition/dev-* - config_name: philosophy data_files: - split: test path: philosophy/test-* - split: validation path: philosophy/validation-* - split: dev path: philosophy/dev-* - config_name: prehistory data_files: - split: test path: prehistory/test-* - split: validation path: prehistory/validation-* - split: dev path: prehistory/dev-* - config_name: professional_accounting data_files: - split: test path: professional_accounting/test-* - split: validation path: professional_accounting/validation-* - split: dev path: professional_accounting/dev-* - config_name: professional_law data_files: - split: test path: professional_law/test-* - split: validation path: professional_law/validation-* - split: dev path: professional_law/dev-* - config_name: professional_medicine data_files: - split: test path: professional_medicine/test-* - split: validation path: professional_medicine/validation-* - split: dev path: professional_medicine/dev-* - config_name: professional_psychology data_files: - split: test path: professional_psychology/test-* - split: validation path: professional_psychology/validation-* - split: dev path: professional_psychology/dev-* - config_name: public_relations data_files: - split: test path: public_relations/test-* - split: validation path: public_relations/validation-* - split: dev path: public_relations/dev-* - config_name: security_studies data_files: - split: test path: security_studies/test-* - split: validation path: security_studies/validation-* - split: dev path: security_studies/dev-* - config_name: sociology data_files: - split: test path: sociology/test-* - split: validation path: sociology/validation-* - split: dev path: sociology/dev-* - config_name: us_foreign_policy data_files: - split: test path: us_foreign_policy/test-* - split: validation path: us_foreign_policy/validation-* - split: dev path: us_foreign_policy/dev-* - config_name: virology data_files: - split: test path: virology/test-* - split: validation path: virology/validation-* - split: dev path: virology/dev-* - config_name: world_religions data_files: - split: test path: world_religions/test-* - split: validation path: world_religions/validation-* - split: dev path: world_religions/dev-* --- # marlosb/mmlu-pt This dataset is a Portuguese translation of the original **MMLU (Measuring Massive Multitask Language Understanding)** dataset. ## Original Dataset - **Hugging Face**: [cais/mmlu](https://huggingface.co/datasets/cais/mmlu) - **Repository**: [https://github.com/hendrycks/test](https://github.com/hendrycks/test) - **Paper**: [Measuring Massive Multitask Language Understanding](https://arxiv.org/abs/2009.03300) ## Dataset Summary MMLU contains ~15,000 multiple-choice questions across 57 subjects (elementary mathematics, history, computer science, law, medicine, etc.), designed to evaluate broad world knowledge and problem-solving ability in large language models. This translated version preserves the **exact structure**, all configs (57 subject splits + `auxiliary_train`, `dev`, `test`), and fields (`question`, `choices`, `answer`). Questions and choices were translated to Portuguese; answer keys (A/B/C/D) remain unchanged. Translation was performed by gpt-4.1-mini. Some questions triggered errors during the LLM call, mostly related to Safety Filter. All this errors were recorded in a file starting with "exceptions_". ## License **MIT License** (same as the original) ### Citation ```bibtex @article{hendryckstest2021, title={Measuring Massive Multitask Language Understanding}, author={Dan Hendrycks and Collin Burns and Steven Basart and Andy Zou and Mantas Mazeika and Dawn Song and Jacob Steinhardt}, journal={Proceedings of the International Conference on Learning Representations (ICLR)}, year={2021} }

应用场景：