RESONATE: real-world social navigation benchmark dataset

RESONATE: real-world social navigation benchmark dataset Stefano Trepella¹, Andrea Ostuni¹, Mauro Martini¹, Irene Borgini², Marcello Chiaberge¹ (S. Trepella and A. Ostuni contributed equally) ¹ Department of Electronics and Telecommunications, Politecnico di Torino, Corso Duca degli Abruzzi 24, 10129 Torino, Italy ² Innovation Center, Intesa Sanpaolo, Corso Inghilterra 3, 10138 Torino, Italy Contact: [e-mail] Dataset DOI: https://doi.org/10.5281/zenodo.23763544 If you use this dataset, please cite the paper: S. Trepella, A. Ostuni, M. Martini, I. Borgini, M. Chiaberge, "RESONATE: a Real Social Navigation Testbed for Autonomous Service Robots". ArXiv preprint arXiv:2611.02937, 2026. https://doi.org/10.48550/arXiv.2611.02937 and the dataset: S. Trepella, A. Ostuni, M. Martini, I. Borgini, M. Chiaberge, "RESONATE: real-world social navigation benchmark dataset", Zenodo, 2026. https://doi.org/10.5281/zenodo.23763544 Licence: CC BY 4.0. Real-robot social navigation benchmark: a Clearpath Jackal navigates past pedestrians in 8 scenarios. 8 navigation algorithms are tested, with 5 runs per algorithm per scenario (320 runs in total). Robot and pedestrian poses were tracked with a Vicon motion-capture system. The dataset contains: the raw ROS 2 recordings the smoothed trajectories as CSV per-run quantitative social-navigation metrics per-run human ratings the scenario and map definitions trajectory animations The metrics were computed with the social_evaluation package (https://github.com/PIC4SeR/social_evaluation) using the settings in config/metrics.yaml. The navigation plugins, maps and Nav2 configuration files are at https://github.com/PIC4SeR/RESONATE. Experimental setup Robot: Clearpath Jackal (Intel i7-6700) with an RPLidar 2D lidar mounted 35 cm above the ground, used for obstacle detection. Tracking: 10 Vicon cameras tracked the robot and every person with millimetre accuracy. The robot was also localised with Vicon. Each participant wore a U-shaped 33 cm × 30 cm neck-worn frame carrying a unique marker pattern. Area: about 4.0 m × 5.0 m. The map changes between scenarios. Software: ROS 2 Humble and Nav2. Only the local controller changes between algorithms. The global planner (NavFn), the behaviour tree and the rest of the stack are the same for all of them. Limits, the same for every controller: v ∈ [0.0, 0.6] m/s and ω ∈ [−1.5, 1.5] rad/s. All controllers ran onboard at 20 Hz. Run outcomes: success (goal reached without collision), collision with the environment, collision with an agent, timeout (goal not reached within 60 s) or navigation failure (the navigation stack failed). Layout raw_bags/<scenario>/<algorithm>/run_<k>/ ROS 2 bag (sqlite3) of run k = 1..5: metadata.yaml + *.db3 raw_bags/run_end_times.csv wall-clock end time of each ORCA / SFM run raw_bags/{S5_Crossing3,S6_NarrowTurn}/ORCA/notes.txt operator notes for these runs trajectories/<scenario>/<algorithm>/run_<k>/ robot.csv, pedestrians.csv (smoothed, see below) trajectories/<scenario>/goals.csv robot goal (map frame) per run metrics/runs.csv one row per run: result, metrics, human ratings scenarios/<scenario>.yaml robot/pedestrian start poses and goals scenarios/maps/ occupancy maps (ROS map_server format) config/ settings for extraction, metrics and trajectory plots (social_evaluation package) videos/<scenario>/<algorithm>/run_<k>.mp4 top-down animation of each run Folder names follow the paper's scenario and algorithm labels. The .db3 files keep the names they were recorded with: scenario_<k>_0.db3 is run k. Scenarios folder paper label map people description S1_Passing S1 - Passing corridor 1 The robot traverses a hallway starting from the centre. One person walks the opposite way, keeping to the right-hand side. S2_Overtaking S2 - Overtaking corridor 1 The robot traverses a hallway starting from the right-hand side. A person starts ahead of it and walks the same way. S3_Crossing1 S3 - Crossing 1 crossing 2 The robot drives to a goal straight ahead while two people cross its path from the same side. The second person starts when the first has almost reached their goal. S4_Crossing2 S4 - Crossing 2 crossing 2 As S3, but the second person starts from the opposite side. S5_Crossing3 S5 - Crossing 3 open 3 The robot traverses the room while three people cross in an X shape, two of them side by side. S6_NarrowTurn S6 - Narrow Turn narrow_turn 2 The robot turns into a passage while one person walks out of it. The robot then reaches a goal near a second person who walks around a fixed point. S7_Mixed S7 - Mixed mixed 3 The robot traverses a room with three people: one stands still, one crosses the robot's path near the start and one passes the robot near the goal. S8_CuriousPerson S8 - Curious Person open 1 The robot traverses the room while one person tracks it and tries to block its path, keeping about 50 cm away. In S1–S7 the participants walk at 0.4–1 m/s and treat the robot as another person. They change their path only to avoid a collision. In S8 the participant deliberately chases and disturbs the robot. The paper rates S1–S4 as moderate and S5–S8 as hard. Algorithms folder paper label description DWA DWA Nav2 DWB controller, obstacles only DWA_GSC DWAGSC DWA + Gaussian social costmap layer around people SFM-DWA SFM-DWA DWA + social-work critic: cost = sum of the social-force norms (Social Force Model) along the candidate trajectory MPPI MPPI Nav2 MPPI controller, obstacles only MPPI_GSC MPPIGSC MPPI + Gaussian social costmap layer around people SFM-MPPI SFM-MPPI MPPI + social-work critic (as in SFM-DWA) ORCA ORCA Optimal Reciprocal Collision Avoidance computes the velocity command directly (standalone local controller) SFM SFM The Social Force Model computes the velocity command directly (standalone local controller) The Gaussian costmap layer raises the cost around each person, mostly in front of them, so the robot avoids cutting across their path. ORCA and SFM already model the people, so they were run without the social cost modules. Each controller was tuned before the benchmark. Recording periods (bag start times): DWA, MPPI, their GSC variants, SFM-DWA and SFM-MPPI were recorded July–October 2024, ORCA June–July 2025 and SFM July 2025. Raw bags Topics: topic type content /vicon/odom nav_msgs/Odometry robot pose (map frame) and velocity from Vicon /vicon/people people_msgs/People tracked pedestrians from Vicon /objective geometry_msgs/PoseStamped robot navigation goal /jackal_velocity_controller/cmd_vel_unstamped geometry_msgs/Twist velocity command /scan sensor_msgs/LaserScan robot lidar /tf, /tf_static tf2_msgs/TFMessage transforms The topic set can vary slightly between campaigns. Each bag's metadata.yaml lists what it contains. Read the bags with ros2 bag (ROS 2 Humble) or with the rosbags Python package, which does not need ROS. Trajectories The trajectories are extracted from /vicon/odom and /vicon/people and trimmed to the run. They are then smoothed with an exponential moving average (alpha = 0.1, about a 0.95 s time constant at 10 Hz). Finally the robot and pedestrian series are aligned on a common time base. These are the trajectories the metrics were computed on. robot.csv: time (s, Unix epoch), position_{x,y,z} (m, map frame), quaternion q{x,y,z,w}, roll/pitch/yaw (rad), frame_id, velocity_{x,y,z} (m/s) and angular_velocity_{x,y,z} (rad/s), taken as published on /vicon/odom. pedestrians.csv: one row per pedestrian per sample. The columns are time, position_{x,y} (m), yaw (rad), velocity_{x,y} (m/s), angular_velocity_z (rad/s), behavior, group_id (-1 = no group) and id (pedestrian identifier, constant within a scenario). Metrics and ratings (metrics/runs.csv) Each row is one run, identified by scenario, algorithm and run (the folder names). The result column is Success or Failure: <reason>, where the reason is a timeout, a collision with the environment, a collision with an agent or a navigation failure. Over the 320 runs there are 278 successes, 24 timeouts, 8 collisions with the environment, 7 collisions with an agent and 3 navigation failures. Distances are measured between body surfaces, with robot radius 0.27 m and person radius 0.3 m. The goal is reached when the robot centre is within 0.5 m of the goal. All other parameters are in config/metrics.yaml. column meaning time_to_reach_goal s until first within goal tolerance, or the whole run if never path_length m travelled by the robot cumulative_heading_changes rad, summed absolute heading change avg_distance_to_closestperson, minimum/maximum_distance_to_people m {intimate,personal,social,public}_space_intrusions % of time with a person in that zone (body gap < 0.45 / 0.45–1.2 / 1.2–3.6 / > 3.6 m) group_*_space_intrusions same, measured against groups of people robot_on_person_collision, person_on_robot_collision collision counts time_not_moving s with robot speed below 0.08 m/s completed 1 if the run ended at the goal final_distance_to_target, minimum_distance_to_target m avg_robot_linear_speed, avg_robot_angular_speed m/s, rad/s avg_acceleration, avg_overacceleration m/s², m/s³ (jerk), differenced over 0.5 s avg_pedestrian_velocity, avg_closest_pedestrian_velocity m/s social_force_on_agents, social_force_on_robot social-force magnitude, integrated over time social_work, social_work_per_second, social_work_interaction social work: total, per second, and per second while someone is within 3.6 m obstacle_force_on_agents, obstacle_force_on_robot obstacle-force magnitude, integrated over time Safety, Friendliness, Naturalness, Avoidance foresight human ratings, 1–5, one mean per run The paper reports path length, time to goal and heading changes over successful runs only. It reports the social metrics over all runs. Human ratings 30 volunteers took part: Italian, 76 % male and 24 % female, aged 20–40 (mean 27, SD 5). Most of them reported some familiarity with robots. Each run had its own combination of participants. The people who walked in the runs rated them in a survey at the end of the experiment. They were not told which controller was running. They rated every run, whether it succeeded or failed. The four criteria use a 5-point Likert scale (1 = worst, 5 = best): Safety: avoids collisions, near-collisions and intrusions into personal space. Friendliness: follows expected social norms, such as keeping right in corridors and not cutting in front of crossing people. Naturalness: the robot's trajectory is smooth. Avoidance foresight: anticipates people's paths and reacts to sudden changes while keeping a safe distance. Each value in runs.csv is the mean over the participants who rated that run. Ratings are missing for 19 runs. Safety was not rated in S8 (Curious Person), because there the person deliberately seeks close contact. Participants gave written informed consent. The study was approved by the Ethics Committee of Politecnico di Torino (protocol n. 67359/2026) and follows the Declaration of Helsinki. The ratings are anonymised. avg_acceleration and avg_overacceleration are missing for one run.