lmms-lab/LMMs-Eval-Lite

该数据集包含多个视觉问答（VQA）和图像理解任务的数据集配置信息。每个数据集包含问题、选项、答案、图像等特征，并且每个数据集都有一个名为lite的分割，包含500个示例。这些数据集可用于训练和评估视觉问答模型。

This dataset contains configuration information for multiple visual question answering (VQA) and image understanding tasks. Each dataset includes features such as questions, options, answers, images, and each dataset has a split named lite containing 500 examples. These datasets can be used for training and evaluating visual question answering models.

原始信息汇总

数据集概述

数据集列表

ai2d

• 特征:
  • question: string
  • options: sequence of string
  • answer: string
  • image: image
• 分割:
  • lite: 500个样本, 90543302.1658031字节
• 下载大小: 81458737字节
• 数据集大小: 90543302.1658031字节

chartqa

• 特征:
  • type: string
  • question: string
  • answer: string
  • image: image
• 分割:
  • lite: 500个样本, 23170424.2字节
• 下载大小: 23219432字节
• 数据集大小: 23170424.2字节

coco2017_cap_val

• 特征:
  • question_id: string
  • image: image
  • question: string
  • answer: sequence of string
  • id: int64
  • license: int8
  • file_name: string
  • coco_url: string
  • height: int32
  • width: int32
  • date_captured: string
• 分割:
  • lite: 500个样本, 81724646.1字节
• 下载大小: 81036195字节
• 数据集大小: 81724646.1字节

docvqa_val

• 特征:
  • questionId: string
  • question: string
  • question_types: sequence of string
  • image: image
  • docId: int64
  • ucsf_document_id: string
  • ucsf_document_page_no: string
  • answers: sequence of string
  • data_split: string
• 分割:
  • lite: 500个样本, 334538449.19872874字节
• 下载大小: 249349131字节
• 数据集大小: 334538449.19872874字节

flickr30k_test

• 特征:
  • image: image
  • caption: sequence of string
  • sentids: sequence of string
  • img_id: string
  • filename: string
• 分割:
  • lite: 500个样本, 69689341.17644653字节
• 下载大小: 66621555字节
• 数据集大小: 69689341.17644653字节

gqa

• 特征:
  • id: string
  • imageId: string
  • question: string
  • answer: string
  • fullAnswer: string
  • isBalanced: bool
  • groups: struct
    • global: string
    • local: string
  • entailed: string
  • equivalent: string
  • types: struct
    • structural: string
    • semantic: string
    • detailed: string
  • annotations: sequence of struct
    • question: struct
      • objectId: string
      • value: string
    • answer: struct
      • objectId: string
      • value: string
    • fullAnswer: struct
      • objectId: string
      • value: string
  • semantic: list
    • operation: string
    • argument: string
    • dependencies: sequence of int32
  • semanticStr: string
• 分割:
  • lite: 500个样本, 243022.3008427413字节
• 下载大小: 107530字节
• 数据集大小: 243022.3008427413字节

infovqa_val

• 特征:
  • questionId: string
  • question: string
  • answers: sequence of string
  • answer_type: sequence of string
  • image: image
  • image_url: string
  • operation/reasoning: sequence of string
  • ocr: string
  • data_split: string
• 分割:
  • lite: 500个样本, 304765105.6765441字节
• 下载大小: 233689969字节
• 数据集大小: 304765105.6765441字节

mmbench_cn_dev

• 特征:
  • index: int64
  • question: string
  • hint: string
  • answer: string
  • A: string
  • B: string
  • C: string
  • D: string
  • category: string
  • image: image
  • source: string
  • L2-category: string
  • comment: string
  • split: string
• 分割:
  • lite: 500个样本, 11861120.35112035字节
• 下载大小: 12795903字节
• 数据集大小: 11861120.35112035字节

mmbench_en_dev

• 特征:
  • index: int64
  • question: string
  • hint: string
  • answer: string
  • A: string
  • B: string
  • C: string
  • D: string
  • category: string
  • image: image
  • source: string
  • L2-category: string
  • comment: string
  • split: string
• 分割:
  • lite: 500个样本, 11871291.175791176字节
• 下载大小: 12524588字节
• 数据集大小: 11871291.175791176字节

nocaps_val

• 特征:
  • image: image
  • image_coco_url: string
  • image_date_captured: string
  • image_file_name: string
  • image_height: int32
  • image_width: int32
  • image_id: int32
  • image_license: int8
  • image_open_images_id: string
  • annotations_ids: sequence of int32
  • annotations_captions: sequence of string
• 分割:
  • lite: 500个样本, 157984760.66666666字节
• 下载大小: 155545761字节
• 数据集大小: 157984760.66666666字节

ok_vqa_val2014

• 特征:
  • question_id: string
  • image: image
  • question: string
  • answers: sequence of string
  • question_type: string
  • answer_type: string
• 分割:
  • lite: 500个样本, 82607924.29647246字节
• 下载大小: 80223931字节
• 数据集大小: 82607924.29647246字节

refcoco_bbox_val

• 特征:
  • question_id: string
  • image: image
  • question: string
  • answer: sequence of string
  • segmentation: sequence of float32
  • bbox: sequence of float32
  • iscrowd: int8
  • file_name: string
• 分割:
  • lite: 500个样本, 87885477.24435365字节
• 下载大小: 88424601字节
• 数据集大小: 87885477.24435365字节

seedbench

• 特征:
  • answer: string
  • choice_a: string
  • choice_b: string
  • choice_c: string
  • choice_d: string
  • data_id: string
  • data_type: string
  • question: string
  • question_id: string
  • question_type_id: int16
  • image: sequence of image
  • segment: sequence of int64
• 分割:
  • lite: 500个样本, 755921749.3379655字节
• 下载大小: 181839440字节
• 数据集大小: 755921749.3379655字节

textcaps_val

• 特征:
  • question_id: string
  • question: string
  • image: image
  • image_id: string
  • image_classes: sequence of string
  • flickr_original_url: string
  • flickr_300k_url: string
  • image_width: int64
  • image_height: int64
  • set_name: string
  • image_name: string
  • image_path: string
  • caption_id: sequence of int64
  • caption_str: sequence of string
  • reference_strs: sequence of string
• 分割:
  • lite: 500个样本, 145274544.53569174字节
• 下载大小: 135721574字节
• 数据集大小: 145274544.53569174字节

textvqa_val

• 特征:
  • image_id: string
  • question_id: int32
  • question: string
  • question_tokens: sequence of string
  • image: image
  • image_width: int32
  • image_height: int32
  • flickr_original_url: string
  • flickr_300k_url: string
  • answers: sequence of string
  • image_classes: sequence of string
  • set_name: string
  • ocr_tokens: sequence of string
• 分割:
  • lite: 500个样本, 143485382.6字节
• 下载大小: 139843809字节
• 数据集大小: 143485382.6字节

vizwiz_vqa_val

• 特征:
  • question_id: string
  • image: image
  • question: string
  • answers: sequence of string
  • category: string
• 分割:
  • lite: 500个样本, 242880108.01111367字节
• 下载大小: 232689462字节
• 数据集大小: 242880108.01111367字节

vqav2_val

• 特征:
  • question_type: string
  • multiple_choice_answer: string
  • answers: list
    • answer: string
    • answer_confidence: string
    • answer_id: int64
  • image_id: int64
  • answer_type: string
  • question_id: int64
  • question: string
  • image: image
• 分割:
  • lite: 500个样本, 79046522.98300941字节
• 下载大小: 78981610字节
• 数据集大小: 79046522.98300941字节

搜集汇总

背景与挑战

背景概述

LMMs-Eval-Lite是一个多模态评估数据集，包含17个子集，每个子集500行数据，涵盖图像、文本和时间序列，用于加速模型开发过程中的全面评估。数据集涉及科学、自然等多个领域的问题和答案。