Human from an Articulated Robot Perspective: the HARPER dataset

The HARPER dataset is a novel corpus for 3D human body pose estimation and forecasting specifically captured from a quadruped robot’s sensors, offering a unique “robot’s-eye” perspective. It consists of synchronized recordings from Spot’s built-in stereo cameras and a 6-camera OptiTrack motion-capture system, covering 15 interactive actions (10 of which involve physical contact). Alongside the raw data, HARPER provides benchmark splits and evaluation protocols for 3D pose estimation, pose forecasting, and collision-prediction tasks, facilitating reproducible research and direct comparison with published baselines.

Offers:

Robot-Centric ViewpointHARPER focuses on data captured by Boston Dynamics’ quadruped robot Spot, whose low vantage point leads to partial occlusions and unusual viewing angles that challenge standard pose-estimation models

Actions and InteractionsThe dataset includes 15 predefined action classes (e.g., walking, avoiding, physical contact), with 10 classes involving direct human–robot contact, recorded in dyadic interactions between a human subject and the robot

Multi-Sensor SetupEach sequence is captured simultaneously by Spot’s stereo cameras and a high-precision OptiTrack system, yielding sub-millimeter-accurate 3D skeletal ground truth

Benchmarks IncludedHARPER ships with standardized splits and baseline results for three tasks:

3D Human Pose Estimation

Human Pose Forecasting

Collision PredictionThis ensures that future methods can be evaluated directly against the published baselines