Human Attention in Image Captioning

本研究推出了一个名为‘Human Attention in Image Captioning’的新数据集，包含同步记录的眼动和口头描述信息，用于分析人类在自由观看和图像描述任务中的注意力差异。数据集由埃克塞特大学等机构创建，包含14000个实例，是目前最大的此类数据集之一。数据集内容丰富，包括图像、眼动序列和口头描述的文本转录。创建过程涉及使用高精度眼动追踪设备在实验室环境中收集数据。该数据集主要应用于图像描述任务，旨在解决机器与人类在图像描述中注意力机制的差异问题，以及如何通过视觉显著性提升图像描述模型的性能。

This study introduces a novel dataset titled *Human Attention in Image Captioning*, which contains synchronously recorded eye-tracking data and verbal descriptions to analyze differences in human attention between free viewing and image captioning tasks. Developed by institutions including the University of Exeter, this dataset includes 14,000 instances and is currently one of the largest datasets of its kind. The dataset features comprehensive content, encompassing images, eye-tracking sequences, and textual transcriptions of verbal descriptions. Its creation involved collecting data in a controlled laboratory environment using high-precision eye-tracking equipment. Primarily applied to image captioning tasks, this dataset aims to address two critical research issues: the discrepancies in attention mechanisms between humans and machines during image captioning, and how to improve the performance of image captioning models through visual saliency.