CrowdBot_v2: Pedestrian–Robot crowd navigation dataset with pedestrian tracking

OverviewCrowdBot_v2 is a revised and extended release of the CrowdBot dataset originally introduced in Paez-Granados et al. (2021), “3D point cloud and RGBD of pedestrians in robot crowd navigation: detection and tracking” (DOI: 10.21227/ak77-d722). It contains synchronized multi-sensor recordings of the Qolo personal mobility robot navigating through real-world crowds in the city of Lausanne (Switzerland) during farmer’s markets and Christmas market days. The dataset covers crowd densities from light flows of about 0.1 pedestrians per square meter up to around 1.0 ppsm. Qolo is equipped with two 3D LiDARs (front and rear Velodyne VLP-16, 20 Hz) and a forward-facing Intel RealSense D435 RGB-D camera. We provide robot state information (pose, velocity, controller state) together with approximately 250 k frames (≈200 min) of multi-sensor data. Sensor data and structureCrowdBot_v2 includes both the raw sensor data and processed outputs: Raw data in ROS bag format from the two VLP-16 LiDARs and the defaced forward-facing RGB-D camera. Synchronized and calibrated multi-sensor streams, eliminating the temporal and spatial misalignment present in the original release. A unified directory structure: rosbags_<sequence> – raw rosbags (defaced_<stamp>.bag) processed_<sequence> – processed data per sequence alg_res/ – algorithm results detections/ – merged 2D/3D pedestrian detections tracks/ – pedestrian tracking results lidars/, lidars_2d/ – LiDAR data ped_data/ – pedestrian-level proximity and motion metrics source_data/ – robot TF and timestamps (tf_robot/, timestamp/) checkpoints/ – trained model checkpoints (*.pth) used in the released pipeline. Detection, tracking and behavioral metricsCompared to the original CrowdBot dataset, CrowdBot_v2 provides refined people detection and tracking based on a merged 3D/2D Lidar detection pipeline. Pedestrians are detected using a 3D Person-MinkUNet model on LiDAR point clouds combined with 2D detections from DrSPAAM on the RGB-D images. These detections are fused and tracked in 3D using AB3DMOT, leading to substantially fewer false positives and smoother trajectories. In addition to detections and tracks, we release pedestrian-centric behavioral metrics. For each tracked pedestrian we provide smoothed positions and velocities, as well as proxemic and motion descriptors such as minimum distance to the robot, intrusion counts into a robot comfort zone, angular velocity, jerk, and other indicators of interaction dynamics. Each pedestrian frame is labeled as interacting or non-interacting with the navigating robot, enabling studies of human–robot versus human–human interactions (HRI vs HHI). RGB-D availabilityUnlike the first release of the dataset, CrowdBot_v2 includes both LiDAR and defaced forward-facing RGB-D camera data available in the refined rosbags. Code and usageThe GitHub repository SCAI-Lab/crowd_analysis_public provides the analysis pipeline used in our accompanying publication, including scripts for dataset integration, pedestrian metric computation, and comparative experiments on human–human and human–robot interactions across the CrowdBot, JRDB, and SiT datasets. To connect a downloaded copy of CrowdBot_v2 with the analysis tools, specify the dataset root path in datasets_configs/data_path_Crowdbot.yaml within the repository. Intended useCrowdBot_v2 is intended for research on pedestrian behavior, crowd-robot interaction, detection and tracking in crowds, and evaluation of navigation and proxemic models. When using this dataset, please cite both the original CrowdBot data paper and the present Zenodo record.概述：CrowdBot_v2是对原始CrowdBot数据集的修订与扩展版本，该数据集最初由Paez-Granados等人于2021年在论文《3D点云与机器人人群导航中的行人RGBD数据：检测与跟踪》（DOI: 10.21227/ak77-d722）中提出。本数据集包含Qolo个人移动机器人在瑞士洛桑市的农贸市场与圣诞市集期间，于真实人群中导航时采集的同步多传感器记录数据。 本数据集覆盖的人群密度范围约为0.1人/平方米至1.0人/平方米。Qolo机器人搭载了两台3D激光雷达（3D LiDAR，前、后Velodyne VLP-16，采样频率20 Hz）以及一台朝前的英特尔RealSense D435 RGB-D相机。我们同时提供了机器人状态信息（位姿、速度、控制器状态），以及约25万帧（时长约200分钟）的多传感器数据。 传感器数据与组织结构：CrowdBot_v2包含原始传感器数据与处理后输出结果： 1. 原始数据：来自两台VLP-16激光雷达与已完成人脸打码的朝前RGB-D相机的ROS bag格式数据； 2. 同步与校准后的多传感器数据流：消除了原始版本中存在的时空错位问题； 3. 统一的目录结构： - rosbags_<序列编号>：原始ROS包文件（命名格式为defaced_<时间戳>.bag） - processed_<序列编号>：单序列处理后数据 - alg_res/：算法结果文件 - detections/：融合后的2D/3D行人检测结果 - tracks/：行人跟踪结果 - lidars/、lidars_2d/：激光雷达数据 - ped_data/：行人级邻近度与运动指标数据 - source_data/：机器人TF变换与时间戳文件（包含tf_robot/与timestamp/子目录） - checkpoints/：发布的分析管线中使用的预训练模型权重文件（*.pth） 检测、跟踪与行为指标：相较于原始CrowdBot数据集，CrowdBot_v2提供了基于融合3D/2D激光雷达检测管线的优化版行人检测与跟踪功能。行人检测通过在激光雷达点云上运行3D Person-MinkUNet模型，并结合基于RGB-D图像的DrSPAAM 2D检测结果实现。这些检测结果将通过AB3DMOT进行3D空间融合与跟踪，从而大幅减少误报并生成更平滑的轨迹。 除检测与跟踪结果外，本数据集还发布了以行人为中心的行为指标。针对每个被跟踪的行人，我们提供了平滑后的位置与速度数据，以及包括与机器人的最小距离、闯入机器人舒适区的次数、角速度、加加速度在内的空间邻近度与运动描述符，以及其他交互动态相关指标。每个行人帧均被标记为与导航机器人产生交互或未产生交互，从而支持人机交互（Human-Robot Interaction, HRI）与人-人交互（Human-Human Interaction, HHI）的相关研究。 RGB-D数据可用性：与数据集的首个版本不同，CrowdBot_v2在优化后的ROS包中同时提供了激光雷达与已完成人脸打码的朝前RGB-D相机数据。 代码与使用方式：GitHub仓库SCAI-Lab/crowd_analysis_public提供了本配套论文中使用的分析管线，包含数据集集成、行人指标计算以及针对CrowdBot、JRDB与SiT数据集的人-人、人-机交互对比实验脚本。如需将下载的CrowdBot_v2与该分析工具对接，请在仓库内的datasets_configs/data_path_Crowdbot.yaml文件中指定数据集根目录路径。 预期用途：CrowdBot_v2旨在用于行人行为、人群与机器人交互、人群中的检测与跟踪，以及导航与空间邻近度模型评估等相关研究。使用本数据集时，请同时引用原始CrowdBot数据论文与本Zenodo存档记录。