wangyueqian/HawkEye-IT

该数据集主要基于VideoChat2-IT，包含了多个视频和图像相关的任务，如视觉问答、视频分类、视频推理、视频对话、视频问答、视频字幕、图像分类、图像字幕、图像推理、图像对话和图像问答。数据集中的任务指令是通过GPT-3.5/4自动生成的。数据集的视频和图像来源于多个公开数据集，如InternVid、VideoChat、Kinetics-710、SthSthV2、NExTQA、CLEVRER、WebVid、YouCook2、TextVR、TGIF和EgoQA等。

原始信息汇总

数据集概述

数据集许可

• License: MIT

使用限制

• 用户同意不使用该数据集进行可能对人类主体造成伤害的实验。
• 数据可能受其他协议约束，使用前需仔细阅读相关协议以确保合规使用。
• 视频版权属于原始视频创作者或平台，仅用于学术研究。

任务类别

• Visual-Question-Answering
• Question-Answering

数据集字段

• Name: text
• Company/Organization: text
• Country: text
• E-Mail: text

语言

• Language: en

数据集大小

• Size Categories: 1M<n<10M

配置详情

• Config Name: Temporal
  • Data Files:
    • split: internvid_grounding, path: video/temporal/internvid_grounding/train.json
    • split: internvid_caption, path: video/temporal/internvid_caption/train.json
    • split: anetc_grounding, path: video/temporal/anetc_grounding/train.json
    • split: charades_sta_grounding, path: video/temporal/charades_sta_grounding/train.json
• Config Name: Video Classification
  • Data Files:
    • split: ssv2, path: video/classification/ssv2/train.json
    • split: k710, path: video/classification/k710/train.json
• Config Name: Video Reasoning
  • Data Files:
    • split: clevrer_mc, path: video/reasoning/clevrer_mc/train.json
    • split: next_qa, path: video/reasoning/next_qa/train.json
    • split: clevrer_qa, path: video/reasoning/clevrer_qa/train.json
• Config Name: Video Conversation
  • Data Files:
    • split: videochat2, path: video/conversation/videochat2/train.json
    • split: videochatgpt, path: video/conversation/videochatgpt/train.json
    • split: videochat1, path: video/conversation/videochat1/train.json
• Config Name: Video VQA
  • Data Files:
    • split: webvid_qa, path: video/vqa/webvid_qa/train.json
    • split: tgif_transition_qa, path: video/vqa/tgif_transition_qa/train.json
    • split: tgif_frame_qa, path: video/vqa/tgif_frame_qa/train.json
    • split: ego_qa, path: video/vqa/ego_qa/train.json
• Config Name: Video Caption
  • Data Files:
    • split: textvr, path: video/caption/textvr/train.json
    • split: youcook2, path: video/caption/youcook2/train.json
    • split: webvid, path: video/caption/webvid/train.json
    • split: videochat, path: video/caption/videochat/train.json
• Config Name: Image Classification
  • Data Files:
    • split: imagenet, path: image/classification/imagenet/train.json
    • split: coco_itm, path: image/classification/coco_itm/train.json
• Config Name: Image Caption
  • Data Files:
    • split: textcaps, path: image/caption/textcaps/train.json
    • split: minigpt4, path: image/caption/minigpt4/train.json
    • split: coco, path: image/caption/coco/train.json
    • split: paragraph_captioning, path: image/caption/paragraph_captioning/train.json
    • split: llava, path: image/caption/llava/train.json
• Config Name: Image Reasoning
  • Data Files:
    • split: llava, path: image/reasoning/llava/train.json
    • split: clevr, path: image/reasoning/clevr/train.json
    • split: visual_mrc, path: image/reasoning/visual_mrc/train.json
• Config Name: Image Conversation
  • Data Files:
    • split: llava, path: image/conversation/llava/train.json
• Config Name: Image VQA
  • Data Files:
    • split: okvqa, path: image/vqa/okvqa/train.json
    • split: docvqa, path: image/vqa/docvqa/train.json
    • split: ocr_vqa, path: image/vqa/ocr_vqa/train.json
    • split: vqav2_chinese, path: image/vqa/vqav2_chinese/train.json
    • split: vqav2, path: image/vqa/vqav2/train.json
    • split: st_vqa, path: image/vqa/st_vqa/train.json
    • split: text_vqa, path: image/vqa/text_vqa/train.json
    • split: gqa, path: image/vqa/gqa/train.json
    • split: okvqa_chinese, path: image/vqa/okvqa_chinese/train.json
    • split: viquae, path: image/vqa/viquae/train.json
    • split: a_okvqa, path: image/vqa/a_okvqa/train.json