ViQuAE

ViQuAE 是 KVQAE（关于命名实体的基于知识的视觉问答）的数据集，该任务包括使用知识库回答有关基于视觉上下文的命名实体的问题。它是第一个涵盖广泛实体类型（例如人、地标和产品）的 KVQAE 数据集。我们认为，KVQAE 是一个清晰、定义明确的任务，可以轻松评估，使其适合跟踪多模态实体表示质量的进度。多模态实体表示是一个核心问题，它将使人机交互更加自然。例如，在看电影时，人们可能会想“我在哪里见过这位女演员？”或“她曾经获得过奥斯卡奖吗？”

ViQuAE is a dataset for KVQAE (Knowledge-based Visual Question Answering about Named Entities), where the task involves answering questions about named entities grounded in visual context using a knowledge base. It is the first KVQAE dataset that covers a wide range of entity types such as people, landmarks, and products. We argue that KVQAE is a clear, well-defined task that enables straightforward evaluation, making it suitable for tracking progress in the quality of multimodal entity representations. Multimodal entity representation is a core issue that will make human-computer interaction more natural. For example, when watching a movie, one might wonder "Where have I seen this actress before?" or "Has she ever won an Oscar Award?"

搜集汇总

背景与挑战

背景概述

ViQuAE是一个公开的预训练数据集，专注于AIGC、自然语言处理和视觉问答领域，特别是事实视觉问答任务，采用MIT许可证。该数据集受到较高关注，体积为1.4MB。