five

aslessor/MMMU

收藏
Hugging Face2024-01-06 更新2024-03-04 收录
下载链接：
https://hf-mirror.com/datasets/aslessor/MMMU
下载链接
链接失效反馈
官方服务：
资源简介：
--- language: - en license: apache-2.0 size_categories: - 10K<n<100K task_categories: - question-answering - visual-question-answering - multiple-choice pretty_name: mmmu dataset_info: - config_name: Accounting features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 262599.0 num_examples: 5 - name: validation num_bytes: 1598285.0 num_examples: 30 - name: test num_bytes: 22135625.0 num_examples: 380 download_size: 37363379 dataset_size: 23996509.0 - config_name: Agriculture features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 22082656.0 num_examples: 5 - name: validation num_bytes: 119217558.0 num_examples: 30 - name: test num_bytes: 993664077.0 num_examples: 287 download_size: 1158036990 dataset_size: 1134964291.0 - config_name: Architecture_and_Engineering features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 137750.0 num_examples: 5 - name: validation num_bytes: 721378.0 num_examples: 30 - name: test num_bytes: 16054607.0 num_examples: 551 download_size: 48763955 dataset_size: 16913735.0 - config_name: Art features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 6241184.0 num_examples: 5 - name: validation num_bytes: 29934534.0 num_examples: 30 - name: test num_bytes: 237801390.0 num_examples: 231 download_size: 585798641 dataset_size: 273977108.0 - config_name: Art_Theory features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 7435106.0 num_examples: 5 - name: validation num_bytes: 33481558.0 num_examples: 30 - name: test num_bytes: 553174647.0 num_examples: 429 download_size: 930525695 dataset_size: 594091311.0 - config_name: Basic_Medical_Science features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 814310.0 num_examples: 5 - name: validation num_bytes: 4125930.0 num_examples: 30 - name: test num_bytes: 48125891.0 num_examples: 326 download_size: 84666454 dataset_size: 53066131.0 - config_name: Biology features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 574342.0 num_examples: 5 - name: validation num_bytes: 8491863.0 num_examples: 30 - name: test num_bytes: 132966151.0 num_examples: 345 download_size: 410242502 dataset_size: 142032356.0 - config_name: Chemistry features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 262397.0 num_examples: 5 - name: validation num_bytes: 1518573.0 num_examples: 30 - name: test num_bytes: 37219529.0 num_examples: 603 download_size: 108345562 dataset_size: 39000499.0 - config_name: Clinical_Medicine features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 1467945.0 num_examples: 5 - name: validation num_bytes: 10882484.0 num_examples: 30 - name: test num_bytes: 98201863.0 num_examples: 325 download_size: 160611488 dataset_size: 110552292.0 - config_name: Computer_Science features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 440523.0 num_examples: 5 - name: validation num_bytes: 2072018.0 num_examples: 30 - name: test num_bytes: 32047381.0 num_examples: 371 download_size: 55640991 dataset_size: 34559922.0 - config_name: Design features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 2259873.0 num_examples: 5 - name: validation num_bytes: 17923120.0 num_examples: 30 - name: test num_bytes: 77676331.0 num_examples: 169 download_size: 142866617 dataset_size: 97859324.0 - config_name: Diagnostics_and_Laboratory_Medicine features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 2056117.0 num_examples: 5 - name: validation num_bytes: 37106233.0 num_examples: 30 - name: test num_bytes: 157003069.0 num_examples: 162 download_size: 603957093 dataset_size: 196165419.0 - config_name: Economics features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 171434.0 num_examples: 5 - name: validation num_bytes: 1487048.0 num_examples: 30 - name: test num_bytes: 11852300.0 num_examples: 267 download_size: 20777635 dataset_size: 13510782.0 - config_name: Electronics features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 123632.0 num_examples: 5 - name: validation num_bytes: 641377.0 num_examples: 30 - name: test num_bytes: 5717686.0 num_examples: 256 download_size: 11602832 dataset_size: 6482695.0 - config_name: Energy_and_Power features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 105006.0 num_examples: 5 - name: validation num_bytes: 1641935.0 num_examples: 30 - name: test num_bytes: 14748428.0 num_examples: 432 download_size: 35246567 dataset_size: 16495369.0 - config_name: Finance features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 296124.0 num_examples: 5 - name: validation num_bytes: 1071060.0 num_examples: 30 - name: test num_bytes: 12065803.0 num_examples: 355 download_size: 29551521 dataset_size: 13432987.0 - config_name: Geography features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 1494060.0 num_examples: 5 - name: validation num_bytes: 6671316.0 num_examples: 30 - name: test num_bytes: 137218400.0 num_examples: 565 download_size: 374766631 dataset_size: 145383776.0 - config_name: History features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 1444231.0 num_examples: 5 - name: validation num_bytes: 8819857.0 num_examples: 30 - name: test num_bytes: 115228815.0 num_examples: 278 download_size: 232549641 dataset_size: 125492903.0 - config_name: Literature features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 2451201.0 num_examples: 5 - name: validation num_bytes: 14241046.0 num_examples: 30 - name: test num_bytes: 50301541.0 num_examples: 112 download_size: 132145895 dataset_size: 66993788.0 - config_name: Manage features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 449514.0 num_examples: 5 - name: validation num_bytes: 3277436.0 num_examples: 30 - name: test num_bytes: 29963963.0 num_examples: 245 download_size: 51186888 dataset_size: 33690913.0 - config_name: Marketing features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 116960.0 num_examples: 5 - name: validation num_bytes: 1472981.0 num_examples: 30 - name: test num_bytes: 7732976.0 num_examples: 181 download_size: 13146078 dataset_size: 9322917.0 - config_name: Materials features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 239632.0 num_examples: 5 - name: validation num_bytes: 2305211.0 num_examples: 30 - name: test num_bytes: 25256854.0 num_examples: 458 download_size: 78365794 dataset_size: 27801697.0 - config_name: Math features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 175861.0 num_examples: 5 - name: validation num_bytes: 1444458.0 num_examples: 30 - name: test num_bytes: 27701878.0 num_examples: 505 download_size: 89368153 dataset_size: 29322197.0 - config_name: Mechanical_Engineering features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 152542.0 num_examples: 5 - name: validation num_bytes: 874988.0 num_examples: 30 - name: test num_bytes: 15093746.0 num_examples: 429 download_size: 30450114 dataset_size: 16121276.0 - config_name: Music features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 1417615.0 num_examples: 5 - name: validation num_bytes: 9359372.0 num_examples: 30 - name: test num_bytes: 134096770.0 num_examples: 334 download_size: 174725052 dataset_size: 144873757.0 - config_name: Pharmacy features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 207924.0 num_examples: 5 - name: validation num_bytes: 1656342.0 num_examples: 30 - name: test num_bytes: 31866248.0 num_examples: 430 download_size: 62721263 dataset_size: 33730514.0 - config_name: Physics features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 233734.0 num_examples: 5 - name: validation num_bytes: 1114130.0 num_examples: 30 - name: test num_bytes: 15905705.0 num_examples: 408 download_size: 35238571 dataset_size: 17253569.0 - config_name: Psychology features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 600864.0 num_examples: 5 - name: validation num_bytes: 4403886.0 num_examples: 30 - name: test num_bytes: 53813915.0 num_examples: 305 download_size: 102466671 dataset_size: 58818665.0 - config_name: Public_Health features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 234781.0 num_examples: 5 - name: validation num_bytes: 1508761.0 num_examples: 30 - name: test num_bytes: 32150088.0 num_examples: 509 download_size: 48231609 dataset_size: 33893630.0 - config_name: Sociology features: - name: id dtype: string - name: question dtype: string - name: options dtype: string - name: explanation dtype: string - name: image_1 dtype: image - name: image_2 dtype: image - name: image_3 dtype: image - name: image_4 dtype: image - name: image_5 dtype: image - name: image_6 dtype: image - name: image_7 dtype: image - name: img_type dtype: string - name: answer dtype: string - name: topic_difficulty dtype: string - name: question_type dtype: string - name: subfield dtype: string splits: - name: dev num_bytes: 3769220.0 num_examples: 5 - name: validation num_bytes: 18455336.0 num_examples: 30 - name: test num_bytes: 144301123.0 num_examples: 252 download_size: 310313826 dataset_size: 166525679.0 configs: - config_name: Accounting data_files: - split: dev path: Accounting/dev-* - split: validation path: Accounting/validation-* - split: test path: Accounting/test-* - config_name: Agriculture data_files: - split: dev path: Agriculture/dev-* - split: validation path: Agriculture/validation-* - split: test path: Agriculture/test-* - config_name: Architecture_and_Engineering data_files: - split: dev path: Architecture_and_Engineering/dev-* - split: validation path: Architecture_and_Engineering/validation-* - split: test path: Architecture_and_Engineering/test-* - config_name: Art data_files: - split: dev path: Art/dev-* - split: validation path: Art/validation-* - split: test path: Art/test-* - config_name: Art_Theory data_files: - split: dev path: Art_Theory/dev-* - split: validation path: Art_Theory/validation-* - split: test path: Art_Theory/test-* - config_name: Basic_Medical_Science data_files: - split: dev path: Basic_Medical_Science/dev-* - split: validation path: Basic_Medical_Science/validation-* - split: test path: Basic_Medical_Science/test-* - config_name: Biology data_files: - split: dev path: Biology/dev-* - split: validation path: Biology/validation-* - split: test path: Biology/test-* - config_name: Chemistry data_files: - split: dev path: Chemistry/dev-* - split: validation path: Chemistry/validation-* - split: test path: Chemistry/test-* - config_name: Clinical_Medicine data_files: - split: dev path: Clinical_Medicine/dev-* - split: validation path: Clinical_Medicine/validation-* - split: test path: Clinical_Medicine/test-* - config_name: Computer_Science data_files: - split: dev path: Computer_Science/dev-* - split: validation path: Computer_Science/validation-* - split: test path: Computer_Science/test-* - config_name: Design data_files: - split: dev path: Design/dev-* - split: validation path: Design/validation-* - split: test path: Design/test-* - config_name: Diagnostics_and_Laboratory_Medicine data_files: - split: dev path: Diagnostics_and_Laboratory_Medicine/dev-* - split: validation path: Diagnostics_and_Laboratory_Medicine/validation-* - split: test path: Diagnostics_and_Laboratory_Medicine/test-* - config_name: Economics data_files: - split: dev path: Economics/dev-* - split: validation path: Economics/validation-* - split: test path: Economics/test-* - config_name: Electronics data_files: - split: dev path: Electronics/dev-* - split: validation path: Electronics/validation-* - split: test path: Electronics/test-* - config_name: Energy_and_Power data_files: - split: dev path: Energy_and_Power/dev-* - split: validation path: Energy_and_Power/validation-* - split: test path: Energy_and_Power/test-* - config_name: Finance data_files: - split: dev path: Finance/dev-* - split: validation path: Finance/validation-* - split: test path: Finance/test-* - config_name: Geography data_files: - split: dev path: Geography/dev-* - split: validation path: Geography/validation-* - split: test path: Geography/test-* - config_name: History data_files: - split: dev path: History/dev-* - split: validation path: History/validation-* - split: test path: History/test-* - config_name: Literature data_files: - split: dev path: Literature/dev-* - split: validation path: Literature/validation-* - split: test path: Literature/test-* - config_name: Manage data_files: - split: dev path: Manage/dev-* - split: validation path: Manage/validation-* - split: test path: Manage/test-* - config_name: Marketing data_files: - split: dev path: Marketing/dev-* - split: validation path: Marketing/validation-* - split: test path: Marketing/test-* - config_name: Materials data_files: - split: dev path: Materials/dev-* - split: validation path: Materials/validation-* - split: test path: Materials/test-* - config_name: Math data_files: - split: dev path: Math/dev-* - split: validation path: Math/validation-* - split: test path: Math/test-* - config_name: Mechanical_Engineering data_files: - split: dev path: Mechanical_Engineering/dev-* - split: validation path: Mechanical_Engineering/validation-* - split: test path: Mechanical_Engineering/test-* - config_name: Music data_files: - split: dev path: Music/dev-* - split: validation path: Music/validation-* - split: test path: Music/test-* - config_name: Pharmacy data_files: - split: dev path: Pharmacy/dev-* - split: validation path: Pharmacy/validation-* - split: test path: Pharmacy/test-* - config_name: Physics data_files: - split: dev path: Physics/dev-* - split: validation path: Physics/validation-* - split: test path: Physics/test-* - config_name: Psychology data_files: - split: dev path: Psychology/dev-* - split: validation path: Psychology/validation-* - split: test path: Psychology/test-* - config_name: Public_Health data_files: - split: dev path: Public_Health/dev-* - split: validation path: Public_Health/validation-* - split: test path: Public_Health/test-* - config_name: Sociology data_files: - split: dev path: Sociology/dev-* - split: validation path: Sociology/validation-* - split: test path: Sociology/test-* tags: - biology - medical - finance - chemistry - music - art - art_theory - design - music - business - accounting - economics - finance - manage - marketing - health - medicine - basic_medical_science - clinical - pharmacy - public_health - humanities - social_science - history - literature - sociology - psychology - science - biology - chemistry - geography - math - physics - engineering - agriculture - architecture - computer_science - electronics - energy_and_power - materials - mechanical_engineering --- # MMMU (A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI) [**🌐 Homepage**](https://mmmu-benchmark.github.io/) | [**🤗 Dataset**](https://huggingface.co/datasets/MMMU/MMMU/) | [**🤗 Paper**](https://huggingface.co/papers/2311.16502) | [**📖 arXiv**](https://arxiv.org/abs/2311.16502) | [**GitHub**](https://github.com/MMMU-Benchmark/MMMU) ## 🔔News - **🔥[2023-12-04]: Our evaluation server for test set is now availble on [EvalAI](https://eval.ai/web/challenges/challenge-page/2179/overview). We welcome all submissions and look forward to your participation! 😆** ## Dataset Details ### Dataset Description We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning. MMMU includes **11.5K meticulously collected multimodal questions** from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering. These questions span **30 subjects** and **183 subfields**, comprising **30 highly heterogeneous image types**, such as charts, diagrams, maps, tables, music sheets, and chemical structures. We believe MMMU will stimulate the community to build next-generation multimodal foundation models towards expert artificial general intelligence (AGI). 🎯 **We have released a full set comprising 150 development samples and 900 validation samples. We have released 10,500 test questions without their answers.** The development set is used for few-shot/in-context learning, and the validation set is used for debugging models, selecting hyperparameters, or quick evaluations. The answers and explanations for the test set questions are withheld. You can submit your model's predictions for the **test set** on **[EvalAI](https://eval.ai/web/challenges/challenge-page/2179/overview)**. ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6230d750d93e84e233882dbc/2Ulh9yznm1dvISV4xJ_Ok.png) ### Dataset Creation MMMU was created to challenge multimodal models with tasks that demand college-level subject knowledge and deliberate reasoning, pushing the boundaries of what these models can achieve in terms of expert-level perception and reasoning. The data for the MMMU dataset was manually collected by a team of college students from various disciplines, using online sources, textbooks, and lecture materials. - **Content:** The dataset contains 11.5K college-level problems across six broad disciplines (Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, Tech & Engineering) and 30 college subjects. - **Image Types:** The dataset includes 30 highly heterogeneous image types, such as charts, diagrams, maps, tables, music sheets, and chemical structures, interleaved with text. ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6230d750d93e84e233882dbc/Mbf8O5lEH8I8czprch0AG.png) ## 🏆 Mini-Leaderboard We show a mini-leaderboard here and please find more information in our paper or [**homepage**](https://mmmu-benchmark.github.io/). | Model | Val (900) | Test (10.5K) | |----------------------------|:---------:|:------------:| | Gemini Ultra* | **59.4** | - | | GPT-4V(ision) (Playground) | 56.8 | **55.7** | | Gemini Pro* | 47.9 | - | | Yi-VL-34B* | 45.9 | 41.6 | | Qwen-VL-PLUS* | 45.2 | 40.8 | | InfiMM-Zephyr-7B* | 39.4 | 35.5 | | SVIT* | 38.0 | 34.1 | | Emu2-Chat* | 36.3 | 34.1 | | BLIP-2 FLAN-T5-XXL | 35.4 | 34.0 | | InstructBLIP-T5-XXL | 35.7 | 33.8 | | LLaVA-1.5-13B | 36.4 | 33.6 | | Qwen-VL-7B | 35.9 | 32.9 | | mPLUG-OWL2* | 32.7 | 32.1 | | BLIP-2 FLAN-T5-XL | 34.4 | 31.0 | | InstructBLIP-T5-XL | 32.9 | 30.6 | | SPHINX* | 32.9 | 32.9 | | Gemini Nano2* | 32.6 | - | | CogVLM | 32.1 | 30.1 | | Otter | 32.2 | 29.1 | | LLaMA-Adapter2-7B | 29.8 | 27.7 | | MiniGPT4-Vicuna-13B | 26.8 | 27.6 | | Fuyu-8B | 27.9 | 27.4 | | Kosmos2 | 24.4 | 26.6 | | OpenFlamingo2-9B | 28.7 | 26.3 | | Frequent Choice | 22.1 | 23.9 | | Random Choice | 26.8 | 25.8 | *: results provided by the authors. ## Limitations Despite its comprehensive nature, MMMU, like any benchmark, is not without limitations. The manual curation process, albeit thorough, may carry biases. And the focus on college-level subjects might not fully be a sufficient test for Expert AGI. However, we believe it should be necessary for an Expert AGI to achieve strong performance on MMMU to demonstrate their broad and deep subject knowledge as well as expert-level understanding and reasoning capabilities. In future work, we plan to incorporate human evaluations into MMMU. This will provide a more grounded comparison between model capabilities and expert performance, shedding light on the proximity of current AI systems to achieving Expert AGI. ## Disclaimers The guidelines for the annotators emphasized strict compliance with copyright and licensing rules from the initial data source, specifically avoiding materials from websites that forbid copying and redistribution. Should you encounter any data samples potentially breaching the copyright or licensing regulations of any site, we encourage you to notify us. Upon verification, such samples will be promptly removed. ## Contact - Xiang Yue: xiangyue.work@gmail.com - Yu Su: su.809@osu.edu - Wenhu Chen: wenhuchen@uwaterloo.ca ## Citation **BibTeX:** ```bibtex @article{yue2023mmmu, title={MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI}, author={Xiang Yue and Yuansheng Ni and Kai Zhang and Tianyu Zheng and Ruoqi Liu and Ge Zhang and Samuel Stevens and Dongfu Jiang and Weiming Ren and Yuxuan Sun and Cong Wei and Botao Yu and Ruibin Yuan and Renliang Sun and Ming Yin and Boyuan Zheng and Zhenzhu Yang and Yibo Liu and Wenhao Huang and Huan Sun and Yu Su and Wenhu Chen}, journal={arXiv preprint arXiv:2311.16502}, year={2023}, } ```
提供机构：
aslessor
原始信息汇总

数据集概述

基本信息

  • 语言: 英语
  • 许可证: Apache 2.0
  • 数据量: 10K < n < 100K
  • 任务类别:
    • 问答
    • 视觉问答
    • 多选题
  • 数据集名称: mmmu

数据集配置

会计 (Accounting)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 262599字节
    • validation: 30个样本, 1598285字节
    • test: 380个样本, 22135625字节
  • 下载大小: 37363379字节
  • 数据集大小: 23996509字节

农业 (Agriculture)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 22082656字节
    • validation: 30个样本, 119217558字节
    • test: 287个样本, 993664077字节
  • 下载大小: 1158036990字节
  • 数据集大小: 1134964291字节

建筑与工程 (Architecture_and_Engineering)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 137750字节
    • validation: 30个样本, 721378字节
    • test: 551个样本, 16054607字节
  • 下载大小: 48763955字节
  • 数据集大小: 16913735字节

艺术 (Art)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 6241184字节
    • validation: 30个样本, 29934534字节
    • test: 231个样本, 237801390字节
  • 下载大小: 585798641字节
  • 数据集大小: 273977108字节

艺术理论 (Art_Theory)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 7435106字节
    • validation: 30个样本, 33481558字节
    • test: 429个样本, 553174647字节
  • 下载大小: 930525695字节
  • 数据集大小: 594091311字节

基础医学科学 (Basic_Medical_Science)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 814310字节
    • validation: 30个样本, 4125930字节
    • test: 326个样本, 48125891字节
  • 下载大小: 84666454字节
  • 数据集大小: 53066131字节

生物学 (Biology)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 574342字节
    • validation: 30个样本, 8491863字节
    • test: 345个样本, 132966151字节
  • 下载大小: 410242502字节
  • 数据集大小: 142032356字节

化学 (Chemistry)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 262397字节
    • validation: 30个样本, 1518573字节
    • test: 603个样本, 37219529字节
  • 下载大小: 108345562字节
  • 数据集大小: 39000499字节

临床医学 (Clinical_Medicine)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 1467945字节
    • validation: 30个样本, 10882484字节
    • test: 325个样本, 98201863字节
  • 下载大小: 160611488字节
  • 数据集大小: 110552292字节

计算机科学 (Computer_Science)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 440523字节
    • validation: 30个样本, 2072018字节
    • test: 371个样本, 32047381字节
  • 下载大小: 55640991字节
  • 数据集大小: 34559922字节

设计 (Design)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 2259873字节
    • validation: 30个样本, 17923120字节
    • test: 169个样本, 77676331字节
  • 下载大小: 142866617字节
  • 数据集大小: 97859324字节

诊断与实验室医学 (Diagnostics_and_Laboratory_Medicine)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 2056117字节
    • validation: 30个样本, 37106233字节
    • test: 162个样本, 157003069字节
  • 下载大小: 603957093字节
  • 数据集大小: 196165419字节

经济学 (Economics)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 171434字节
    • validation: 30个样本, 1487048字节
    • test: 267个样本, 11852300字节
  • 下载大小: 20777635字节
  • 数据集大小: 13510782字节

电子学 (Electronics)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 123632字节
    • validation: 30个样本, 641377字节
    • test: 256个样本, 5717686字节
  • 下载大小: 11602832字节
  • 数据集大小: 6482695字节

能源与电力 (Energy_and_Power)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 105006字节
    • validation: 30个样本, 1641935字节
    • test: 432个样本, 14748428字节
  • 下载大小: 35246567字节
  • 数据集大小: 16495369字节

金融 (Finance)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 296124字节
    • validation: 30个样本, 1071060字节
    • test: 355个样本, 12065803字节
  • 下载大小: 29551521字节
  • 数据集大小: 13432987字节

地理 (Geography)

  • 特征:
    • id: string
    • question: string
    • options: string
    • explanation: string
    • image_1: image
    • image_2: image
    • image_3: image
    • image_4: image
    • image_5: image
    • image_6: image
    • image_7: image
    • img_type: string
    • answer: string
    • topic_difficulty: string
    • question_type: string
    • subfield: string
  • 分割:
    • dev: 5个样本, 1494060字节
    • validation: 30个样本, 6671316字节
    • test: 565个样本, 1372
5,000+
优质数据集
54 个
任务类型
进入经典数据集
关于我们
遇见数据集——让每个数据集都被发现，让每一次遇见都有价值。
数发科官网www.sfktec.com.cn
联系我们
youjian
selectdataset@iotsh.com.cn
商业合作
数据驱动未来，携手共赢发展
二维码
社区交流群

面向社区/商业的数据集话题

二维码
科研交流群

面向高校/科研机构的开源数据集话题

数据驱动未来

携手共赢发展

商业合作