Inst-IT/Inst-IT-Dataset

Inst-IT数据集是一个大规模的多模态指令调优数据集，包含21k视频和51k图像（将图像视为静态的单帧视频）。数据集提供了细粒度的注释，包括21k视频级描述、207k帧级描述（51k图像，156k视频帧）和335k开放式问答对。帧级描述包括单个实例的注释、整个图像的注释以及时间变化的注释。数据集用于增强多模态模型的理解能力，并提供了详细的数据结构和下载方法。

The Inst-IT Dataset is a large-scale instruction tuning dataset with multi-level fine-grained annotations. It includes 21,000 videos and 51,000 images, along with video-level descriptions, frame-level descriptions, and open-ended QA pairs. The annotation structure includes captions for individual instances, the entire image or frame, and temporal changes. The dataset aims to enhance multimodal instance understanding and has been used to improve model performance on various benchmarks.