REC-COCO (Relations in Captions)

字幕中的关系 (REC-COCO) 是一个新的数据集，其中包含字幕令牌和图像中的边界框之间的关联。REC-COCO基于MS-COCO和V-COCO数据集。对于V-COCO中的每个图像，我们从MS-COCO收集其相应的标题，并自动将V-COCO中的概念三元组与标题中的令牌对齐。这需要找到诸如人之类的概念的令牌。结果，REC-COCO包含与每个主题和对象相对应的标题和标记，以及主题和对象的边界框。

The Relationship in Captions (REC-COCO) is a novel dataset that establishes associations between caption tokens and bounding boxes within images. REC-COCO is constructed based on the MS-COCO and V-COCO datasets. For every image present in V-COCO, we retrieve its corresponding captions from MS-COCO, and automatically align the conceptual triples contained in V-COCO with the tokens within the captions. This process involves identifying tokens corresponding to concepts such as 'person'. Ultimately, REC-COCO encompasses captions and tokens corresponding to each subject and object, alongside the bounding boxes of these subjects and objects.

搜集汇总

背景与挑战

背景概述

REC-COCO是一个基于MS-COCO和V-COCO构建的数据集，专注于字幕中关系的研究。它通过自动对齐概念三元组与标题令牌，将字幕令牌与图像中的边界框关联，并包含与主题和对象对应的标题、标记及边界框信息。