视觉感知数据集 JackRabbot

斯坦福视觉与学习实验室研究团队在一篇名为「JRDB：用于人类环境中导航的视觉感知的数据集和基准」的研究中介绍了他们从其社交移动操纵器 JackRabbot 收集的新颖数据集 JRDB。 数据集包括 64 分钟的多模式传感器数据，包括 15 fps 的立体圆柱 360 度 RGB 视频，两个 Velodyne 16 激光雷达的 3D 点云，两个 Sick 激光雷达的线 3D 点云，音频信号，30 fps 的 RGBD 视频，360 度球形鱼眼镜头拍摄的图像和机器人车轮的编码器值。 该数据集包括来自传统上代表性不足的场景（例如室内环境和行人区域）的数据，这些数据来自固定和导航机器人平台。该数据集已标注了超过 230 万个包围盒，这些包围盒分布在场景中的所有人周围的 5 个单独的摄像头中，以及 180 万个相关的 3D 长方体，总计超过 3500 条时间一致的轨迹。 除此以外，研究团队为 2D 和 3D 人检测和跟踪启动了基准和度量。借助计划在将来进行进一步注释的数据集，研究团队希望提供新的数据源以及测试平台，以用于研究机器人自主导航以及人类环境中围绕社交机器人的所有感知任务。

A research team from the Stanford Vision and Learning Lab introduced the novel JRDB dataset, collected from their social mobile manipulator JackRabbot, in a study titled "JRDB: A Dataset and Benchmark for Visual Perception for Navigation in Human Environments". The dataset contains 64 minutes of multimodal sensor data, including stereoscopic cylindrical 360° RGB video at 15 fps, 3D point clouds from two Velodyne 16 LiDARs, line-based 3D point clouds from two Sick LiDARs, audio signals, RGBD video at 30 fps, images captured by a 360° spherical fisheye lens, and encoder values from the robot's wheels. This dataset includes data from traditionally underrepresented scenarios such as indoor environments and pedestrian areas, collected from both stationary and navigational robotic platforms. The dataset has been annotated with over 2.3 million bounding boxes across 5 separate cameras for all persons in the scene, as well as 1.8 million corresponding 3D bounding cuboids, totaling more than 3500 temporally consistent trajectories. Additionally, the research team has established benchmarks and metrics for 2D and 3D human detection and tracking tasks. With plans to add further annotations to the dataset in the future, the research team intends to provide a new data source and testbed for research on robotic autonomous navigation and all perception tasks involving social robots operating in human environments.

搜集汇总

背景与挑战

背景概述

视觉感知数据集JackRabbot是一个由斯坦福团队收集的多模式传感器数据集，包含丰富的标注信息，适用于机器人导航和社交机器人感知任务的研究。数据集特点包括360度RGB视频、3D点云和大量标注的包围盒与3D长方体。