Data_Sheet_1_A multimodal deep learning architecture for smoking detection with a small data approach.PDF

Covert tobacco advertisements often raise regulatory measures. This paper presents that artificial intelligence, particularly deep learning, has great potential for detecting hidden advertising and allows unbiased, reproducible, and fair quantification of tobacco-related media content. We propose an integrated text and image processing model based on deep learning, generative methods, and human reinforcement, which can detect smoking cases in both textual and visual formats, even with little available training data. Our model can achieve 74% accuracy for images and 98% for text. Furthermore, our system integrates the possibility of expert intervention in the form of human reinforcement. Using the pre-trained multimodal, image, and text processing models available through deep learning makes it possible to detect smoking in different media even with few training data.

隐蔽式烟草广告往往会引发监管措施。本研究表明，人工智能尤其是深度学习，在隐蔽广告检测领域具备巨大应用潜力，且可实现对烟草相关媒体内容的无偏、可复现且公平的量化分析。本研究提出了一种融合深度学习、生成式方法与人类强化机制的图文联合处理模型，可在文本与视觉双模态下识别吸烟相关场景，即便在训练数据稀缺的条件下仍可有效运行。该模型在图像模态上的识别准确率可达74%，文本模态则可达98%。此外，本系统还支持以人类强化的形式引入专家干预环节。依托深度学习提供的预训练多模态、图像及文本处理模型，即便训练数据有限，也可实现跨媒体场景下的吸烟相关内容检测。