CrowdBot_v2: Pedestrian–Robot crowd navigation dataset with pedestrian tracking

OverviewCrowdBot_v2 is a revised and extended release of the CrowdBot dataset originally introduced in Paez-Granados et al. (2021), “3D point cloud and RGBD of pedestrians in robot crowd navigation: detection and tracking” (DOI: 10.21227/ak77-d722). It contains synchronized multi-sensor recordings of the Qolo personal mobility robot navigating through real-world crowds in the city of Lausanne (Switzerland) during farmer’s markets and Christmas market days. The dataset covers crowd densities from light flows of about 0.1 pedestrians per square meter up to around 1.0 ppsm. Qolo is equipped with two 3D LiDARs (front and rear Velodyne VLP-16, 20 Hz) and a forward-facing Intel RealSense D435 RGB-D camera. We provide robot state information (pose, velocity, controller state) together with approximately 250 k frames (≈200 min) of multi-sensor data. Sensor data and structureCrowdBot_v2 includes both the raw sensor data and processed outputs: Raw data in ROS bag format from the two VLP-16 LiDARs and the defaced forward-facing RGB-D camera. Synchronized and calibrated multi-sensor streams, eliminating the temporal and spatial misalignment present in the original release. A unified directory structure: rosbags_<sequence> – raw rosbags (defaced_<stamp>.bag) processed_<sequence> – processed data per sequence alg_res/ – algorithm results detections/ – merged 2D/3D pedestrian detections tracks/ – pedestrian tracking results lidars/, lidars_2d/ – LiDAR data ped_data/ – pedestrian-level proximity and motion metrics source_data/ – robot TF and timestamps (tf_robot/, timestamp/) checkpoints/ – trained model checkpoints (*.pth) used in the released pipeline. Detection, tracking and behavioral metricsCompared to the original CrowdBot dataset, CrowdBot_v2 provides refined people detection and tracking based on a merged 3D/2D Lidar detection pipeline. Pedestrians are detected using a 3D Person-MinkUNet model on LiDAR point clouds combined with 2D detections from DrSPAAM on the RGB-D images. These detections are fused and tracked in 3D using AB3DMOT, leading to substantially fewer false positives and smoother trajectories. In addition to detections and tracks, we release pedestrian-centric behavioral metrics. For each tracked pedestrian we provide smoothed positions and velocities, as well as proxemic and motion descriptors such as minimum distance to the robot, intrusion counts into a robot comfort zone, angular velocity, jerk, and other indicators of interaction dynamics. Each pedestrian frame is labeled as interacting or non-interacting with the navigating robot, enabling studies of human–robot versus human–human interactions (HRI vs HHI). RGB-D availabilityUnlike the first release of the dataset, CrowdBot_v2 includes both LiDAR and defaced forward-facing RGB-D camera data available in the refined rosbags. Code and usageThe GitHub repository SCAI-Lab/crowd_analysis_public provides the analysis pipeline used in our accompanying publication, including scripts for dataset integration, pedestrian metric computation, and comparative experiments on human–human and human–robot interactions across the CrowdBot, JRDB, and SiT datasets. To connect a downloaded copy of CrowdBot_v2 with the analysis tools, specify the dataset root path in datasets_configs/data_path_Crowdbot.yaml within the repository. Intended useCrowdBot_v2 is intended for research on pedestrian behavior, crowd-robot interaction, detection and tracking in crowds, and evaluation of navigation and proxemic models. When using this dataset, please cite both the original CrowdBot data paper and the present Zenodo record.数据集概述： CrowdBot_v2是对原始CrowdBot数据集的修订与扩展版本，该原始数据集最早由Paez-Granados等人于2021年在"3D point cloud and RGBD of pedestrians in robot crowd navigation: detection and tracking"（DOI: 10.21227/ak77-d722）中提出。本数据集包含Qolo个人移动机器人在瑞士洛桑市的农贸市场与圣诞市集期间，于真实人群环境中导航时采集的同步多传感器记录数据。 数据集覆盖的人群密度范围约为每平方米0.1名行人至1.0 ppsm（每平方米行人）。Qolo配备两台三维激光雷达（LiDAR）（前后各一台Velodyne VLP-16，采样频率20 Hz）以及一台正向Intel RealSense D435 RGB-D相机。本数据集提供机器人状态信息（位姿、速度、控制器状态），以及约25万帧（时长约200分钟）的多传感器数据。 ### 传感器数据与组织结构 CrowdBot_v2包含原始传感器数据与经过处理的输出结果： - 两台VLP-16激光雷达（LiDAR）与经过面部打码处理的正向RGB-D相机的ROS包（ROS bag）格式原始数据； - 经过同步与标定的多传感器数据流，消除了原始版本中存在的时空错位问题； - 统一的目录结构： - rosbags_<序列编号> —— 原始ROS包文件（defaced_<时间戳>.bag） - processed_<序列编号> —— 各序列的处理后数据 - alg_res/ —— 算法结果 - detections/ —— 融合后的二维/三维行人检测结果 - tracks/ —— 行人跟踪结果 - lidars/、lidars_2d/ —— 激光雷达数据 - ped_data/ —— 行人级别的邻近度与运动指标 - source_data/ —— 机器人TF（坐标变换）与时间戳数据（tf_robot/、timestamp/） - checkpoints/ —— 本次发布的处理流程中使用的预训练模型权重文件（*.pth） ### 检测、跟踪与行为指标 相较于原始CrowdBot数据集，CrowdBot_v2提供了基于融合三维/二维激光雷达检测流程的优化版行人检测与跟踪方案。行人检测通过在激光雷达点云上使用3D Person-MinkUNet模型，结合RGB-D图像上由DrSPAAM生成的二维检测结果实现。所有检测结果将通过AB3DMOT进行三维融合与跟踪，大幅减少了误检数量并获得更平滑的运动轨迹。 除检测与跟踪结果外，本数据集还发布了以行人为中心的行为指标。针对每一名被跟踪的行人，我们提供了平滑后的位置与速度数据，以及邻近度与运动描述符，包括与机器人的最小距离、闯入机器人舒适区的次数、角速度、加加速度（jerk）以及其他交互动态相关指标。每个行人帧均被标记为与导航机器人产生交互或未产生交互，支持人机交互（HRI, Human-Robot Interaction）与人与人交互（HHI, Human-Human Interaction）的相关研究。 ### RGB-D数据可用性 相较于数据集的首个发布版本，CrowdBot_v2在优化后的ROS包中同时提供了激光雷达与经过面部打码处理的正向RGB-D相机数据。 ### 代码与使用方法 GitHub仓库SCAI-Lab/crowd_analysis_public提供了本配套论文中使用的分析流程，包括数据集集成脚本、行人指标计算脚本，以及针对CrowdBot、JRDB与SiT数据集的人机、人人交互对比实验脚本。如需将下载的CrowdBot_v2数据集与该分析工具关联，请在仓库内的datasets_configs/data_path_Crowdbot.yaml文件中指定数据集根目录路径。 ### 预期用途 本数据集适用于行人行为、人群-机器人交互、人群中的检测与跟踪，以及导航与邻近度模型评估相关研究。使用本数据集时，请同时引用原始CrowdBot数据论文与本次Zenodo存档记录。