RefEgo

RefEgo数据集是由理化学研究所、筑波大学和奈良科学技术研究所合作创建的，基于Ego4D的大规模自我中心视频数据集。该数据集包含超过12,000个视频片段，总时长超过41小时，用于视频基础的指称表达理解标注。RefEgo数据集旨在解决从第一人称视角识别和跟踪场景中物体的问题，适用于增强现实设备或自主机器人。数据集的创建过程涉及使用对象检测模型和人工标注，以确保在真实世界的第一人称感知中准确地定位文本指称对象。该数据集的应用领域包括日常任务辅助和语言交流，目标是实现对直观语言表达的精确视觉定位和跟踪。RefEgo Dataset is a large-scale egocentric video dataset co-created by RIKEN, the University of Tsukuba, and the Nara Institute of Science and Technology, based on the Ego4D dataset. It contains over 12,000 video clips with a total duration of more than 41 hours, and is used for annotations of video-based referring expression understanding tasks. The RefEgo dataset aims to solve the problem of identifying and tracking objects in the scene from a first-person perspective, and is applicable to augmented reality devices or autonomous robots. The creation of the dataset involved the use of object detection models and manual annotations to ensure accurate localization of textually referred objects in real-world first-person perception scenarios. Its application fields include daily task assistance and language communication, with the goal of achieving precise visual localization and tracking for intuitive natural language expressions.

搜集汇总

背景与挑战

背景概述

RefEgo是一个基于Ego4D的大规模自我中心视频数据集，包含超过12,000个视频片段（总时长超41小时），用于指称表达理解标注，旨在从第一人称视角识别和跟踪物体，适用于增强现实和自主机器人领域。数据集通过对象检测模型和人工标注确保准确性，支持日常任务辅助和语言交流应用。