SentiCap

图像识别和语言建模的最新进展正在使图像内容的自动描述成为现实。但是，当前系统缺少书面描述的程式化、非事实方面。一种这样的风格是带有情感的描述，这在日常交流中很常见，并且会影响决策和人际关系。我们设计了一个系统来描述带有情感的图像，并展示了一个自动生成带有正面或负面情绪的字幕的模型。我们提出了一种具有词级正则化的新型切换循环神经网络，它能够仅使用 2000 多个包含情感的训练句子来生成情感图像说明。我们使用不同的自动和众包指标评估字幕。我们的模型在图像字幕的常见质量指标方面具有优势。在 84.6% 的案例中，生成的正面字幕被认为至少与事实字幕一样具有描述性。在这些积极的标题中，88% 被众包工作者确认为具有适当的情绪。

Recent advances in image recognition and language modeling have enabled the automatic generation of natural language descriptions for image content. However, current systems lack the stylized, non-factual aspects of written descriptive text. One prominent such style is emotion-infused descriptions, which are common in daily communication and influence decision-making and interpersonal relationships. We developed a system for generating emotion-aware image descriptions, and present a model that automatically produces image captions with either positive or negative emotional tones. We propose a novel switched recurrent neural network with word-level regularization, which is capable of generating emotional image captions using only over 2,000 emotion-containing training sentences. We evaluate the generated captions using various automatic and crowdsourcing-based metrics. Our model demonstrates superior performance across common quality metrics for image captioning. In 84.6% of cases, the generated positive captions were rated as at least as descriptive as factual captions. Among these positive captions, 88% were confirmed by crowdworkers to carry appropriate emotional sentiments.

搜集汇总

背景与挑战

背景概述

SentiCap是一个用于生成带有情感的图像描述的数据集，采用新型神经网络技术，仅需少量训练数据即可生成高质量的情感描述，评估显示其生成结果在描述性和情感表达上表现优异。