FanLu31/CompreCap

CompreCap基准以人类注释的场景图为特征，专注于全面图像描述评估。它为图像中的常见对象提供了新的语义分割注释，平均掩码覆盖率为95.83%。除了对对象的细致注释外，CompreCap还包括对象属性的高质量描述以及对象之间的方向关系描述，构成了一个完整的有向场景图结构。基于CompreCap基准，研究人员可以全面评估大型视觉语言模型生成的图像描述的质量。

The CompreCap benchmark is characterized by human-annotated scene graph and focuses on the evaluation of comprehensive image captioning. It provides new semantic segmentation annotations for common objects in images, with an average mask coverage of 95.83%. Beyond the careful annotation of objects, CompreCap also includes high-quality descriptions of the attributes bound to the objects, as well as directional relation descriptions between the objects, composing a complete and directed scene graph structure. The annotations of segmentation masks, category names, the descriptions of attributes and relationships are saved in ./anno.json. Based on the CompreCap benchmark, researchers can comprehensively accessing the quality of image captions generated by large vision-language models.