CrowdBot_v2: Pedestrian–Robot crowd navigation dataset with pedestrian tracking

OverviewCrowdBot_v2 is a revised and extended release of the CrowdBot dataset originally introduced in Paez-Granados et al. (2021), “3D point cloud and RGBD of pedestrians in robot crowd navigation: detection and tracking” (DOI: 10.21227/ak77-d722). It contains synchronized multi-sensor recordings of the Qolo personal mobility robot navigating through real-world crowds in the city of Lausanne (Switzerland) during farmer’s markets and Christmas market days. The dataset covers crowd densities from light flows of about 0.1 pedestrians per square meter up to around 1.0 ppsm. Qolo is equipped with two 3D LiDARs (front and rear Velodyne VLP-16, 20 Hz) and a forward-facing Intel RealSense D435 RGB-D camera. We provide robot state information (pose, velocity, controller state) together with approximately 250 k frames (≈200 min) of multi-sensor data. Sensor data and structureCrowdBot_v2 includes both the raw sensor data and processed outputs: Raw data in ROS bag format from the two VLP-16 LiDARs and the defaced forward-facing RGB-D camera. Synchronized and calibrated multi-sensor streams, eliminating the temporal and spatial misalignment present in the original release. A unified directory structure: rosbags_<sequence> – raw rosbags (defaced_<stamp>.bag) processed_<sequence> – processed data per sequence alg_res/ – algorithm results detections/ – merged 2D/3D pedestrian detections tracks/ – pedestrian tracking results lidars/, lidars_2d/ – LiDAR data ped_data/ – pedestrian-level proximity and motion metrics source_data/ – robot TF and timestamps (tf_robot/, timestamp/) checkpoints/ – trained model checkpoints (*.pth) used in the released pipeline. Detection, tracking and behavioral metricsCompared to the original CrowdBot dataset, CrowdBot_v2 provides refined people detection and tracking based on a merged 3D/2D Lidar detection pipeline. Pedestrians are detected using a 3D Person-MinkUNet model on LiDAR point clouds combined with 2D detections from DrSPAAM on the RGB-D images. These detections are fused and tracked in 3D using AB3DMOT, leading to substantially fewer false positives and smoother trajectories. In addition to detections and tracks, we release pedestrian-centric behavioral metrics. For each tracked pedestrian we provide smoothed positions and velocities, as well as proxemic and motion descriptors such as minimum distance to the robot, intrusion counts into a robot comfort zone, angular velocity, jerk, and other indicators of interaction dynamics. Each pedestrian frame is labeled as interacting or non-interacting with the navigating robot, enabling studies of human–robot versus human–human interactions (HRI vs HHI). RGB-D availabilityUnlike the first release of the dataset, CrowdBot_v2 includes both LiDAR and defaced forward-facing RGB-D camera data available in the refined rosbags. Code and usageThe GitHub repository SCAI-Lab/crowd_analysis_public provides the analysis pipeline used in our accompanying publication, including scripts for dataset integration, pedestrian metric computation, and comparative experiments on human–human and human–robot interactions across the CrowdBot, JRDB, and SiT datasets. To connect a downloaded copy of CrowdBot_v2 with the analysis tools, specify the dataset root path in datasets_configs/data_path_Crowdbot.yaml within the repository. Intended useCrowdBot_v2 is intended for research on pedestrian behavior, crowd-robot interaction, detection and tracking in crowds, and evaluation of navigation and proxemic models. When using this dataset, please cite both the original CrowdBot data paper and the present Zenodo record.## 概述 CrowdBot_v2 是对原始 CrowdBot 数据集的修订与扩展版本，该原始数据集由 Paez-Granados 等人于 2021 年发表的《面向机器人人群导航的行人3D点云与RGBD（RGBD）数据：检测与跟踪》（DOI: 10.21227/ak77-d722）首次提出。本数据集包含 Qolo 个人移动机器人在瑞士洛桑的农贸市场与圣诞市集期间，于真实人群中导航时采集的同步多传感器记录数据。 本数据集覆盖的人群密度范围约为 0.1 人/平方米至 1.0 人/平方米。Qolo 机器人搭载两台 3D 激光雷达（LiDAR）（前后均为 Velodyne VLP-16，帧率 20 Hz）与一台前向 Intel RealSense D435 RGB-D 相机。我们提供了机器人状态信息（位姿、速度、控制器状态），以及约 25 万帧（约 200 分钟）的多传感器数据。 ## 传感器数据与组织结构 CrowdBot_v2 包含原始传感器数据与处理后输出结果： 1. 来自两台 VLP-16 激光雷达与经过面部脱敏处理的前向 RGB-D 相机的 ROS 包（ROS bag）格式原始数据。 2. 经过同步与校准的多传感器数据流，消除了原始版本中存在的时空错位问题。 3. 统一的目录结构： - `rosbags_<sequence>`：原始 ROS 包文件（命名格式为 `defaced_<stamp>.bag`） - `processed_<sequence>`：各序列的处理后数据 - `alg_res/`：算法结果 - `detections/`：融合后的2D/3D行人检测结果 - `tracks/`：行人跟踪结果 - `lidars/`、`lidars_2d/`：激光雷达数据 - `ped_data/`：行人级别的接近度与运动指标 - `source_data/`：机器人TF变换（TF）与时间戳数据（包含 `tf_robot/`、`timestamp/` 子目录） - `checkpoints/`：本次发布的处理管线中使用的预训练模型权重文件（*.pth） ## 检测、跟踪与行为指标 相较于原始 CrowdBot 数据集，CrowdBot_v2 提供了基于融合3D/2D激光雷达检测管线的优化行人检测与跟踪结果。行人检测首先通过在激光雷达点云上运行 3D Person-MinkUNet 模型，结合 RGB-D 图像上由 DrSPAAM 生成的2D检测结果完成，随后通过 AB3DMOT 进行3D融合与跟踪，大幅降低了假阳性率并获得更平滑的轨迹。 除检测与跟踪结果外，本次发布还包含以行人为中心的行为指标。针对每个被跟踪的行人，我们提供了平滑后的位置与速度数据，以及近距交互与运动描述符，例如与机器人的最小距离、侵入机器人舒适区的次数、角速度、加加速度（jerk）以及其他交互动态相关指标。每个行人帧均被标记为与导航机器人产生交互或未产生交互，支持开展人机交互（HRI, Human-Robot Interaction）与人人间交互（HHI, Human-Human Interaction）相关研究。 ## RGB-D数据可用性 与数据集的首个发布版本不同，CrowdBot_v2 的优化 ROS 包中同时包含激光雷达与经过面部脱敏处理的前向 RGB-D 相机数据。 ## 代码与使用方式 GitHub 仓库 SCAI-Lab/crowd_analysis_public 提供了本研究配套论文中使用的分析管线，包含数据集集成、行人指标计算以及针对 CrowdBot、JRDB 与 SiT 数据集的人人间交互、人机交互对比实验脚本。若要将下载的 CrowdBot_v2 与该分析工具对接，请在仓库内的 `datasets_configs/data_path_Crowdbot.yaml` 中指定数据集根目录路径。 ## 预期用途 CrowdBot_v2 适用于行人行为、人群机器人交互、人群中的检测与跟踪以及导航与近距交互模型评估等方向的研究。使用本数据集时，请同时引用原始 CrowdBot 数据论文与本次 Zenodo 发布记录。