Socially Compliant Navigation Dataset (SCAND)

<p>This dataset provides human-teleoperated socially compliant navigation demonstrations. </p>
<p>Social navigation is the capability of an autonomous agent to navigate in a socially compliant manner such that it recognizes and reacts to the objectives of other navigating agents, at least somewhat adjusting its own path in response, while also projecting signals that can help the other agents reciprocate. Some examples of socially compliant navigation behavior include sticking to the right of the road, following a crowd, overtaking a crowd, etc. </p>
<p><img src="https://dataverse.tdl.org/api/access/datafile/241569" alt="socially compliant navigation" height="200"></p>
<p>The image above shows two socially compliant navigation strategies around human crowds: a) moving with traffic, and b) sticking to the right of the road.</p>
<p>SCAND contains 25 miles and 8.7 hours of robot driven trajectories through a variety of social environments around the University of Texas at Austin campus.</p>
<p> The image below shows the area within the UT Austin campus where data was collected by teleoperating the robots. </p>
<p><img src="https://dataverse.tdl.org/api/access/datafile/267219" alt="data collection area" height="200"></p>
<h3>SCAND CONTENTS</h3>
<ol>
<li>138 robot-driven trajectories.</li>
<li>15 days of social navigation data on 2 robots: a wheeled Clearpath Jackal, and a legged Boston Dynamics Spot.</li>
<li>Indoor and outdoor environments @ UT Austin campus.</li>
<li>2 highly crowded football game days (including a concert at the same time!)</li>
<li>Mild to heavily crowded environments.</li>
<li>4 different human demonstrators.</li>
</ol>
<p>Each SCAND trajectory includes: RGB Azure Kinect camera, velodyne pointclouds, stereo camera, visual odometry (Spot), wheel odometry (Jackal), leg positions (Spot), joystick commands issued by the demonstrator, and monocular cameras (Spot). All data is stored in the ROSBAG data format http://wiki.ros.org/Bags/Format. </p>
<p>For each trajectory the accompanying video contains a speed up version of the robot's camera view, intended to provide a quick overlook of the scene in a particular trajectory for the end user. </p>
<p>SCAND files follow the following file naming convention : {Demonstrator tag}_{Robot name}_{Start location}_{End location}_{Day}_{Month}_{Date}_{Trajectory number}.bag/.mp4/.avi </p>

<p>The wheeled Jackal and legged Spot robots used to collect the data are shown below along with the multi-modal sensors implemented in both.</p>
<p><img src="https://dataverse.tdl.org/api/access/datafile/241570" gbrecs=true&key=3c9dd214-0aeb-4a96-ada7-f94babe252db" alt="Jackal and Spot" height="200"></p>
<p>The image below illustrates five example scenarios from SCAND showing the RGB image and accompanying LIDAR. The scenarios have tags indicating the characteristics present in the trajectories such as: street crossing, narrow doorway, navigating through large crowds, vehicle interaction and crossing stationary queue. </p>
<p><img src="https://dataverse.tdl.org/api/access/datafile/241571" alt="five scenarios" height="100"></p>

<p>In SCAND, we provide twelve different labels of socially compliant navigation events, along an entire trajectory for all the trajectories in the dataset. To label the dataset, the trajectories were manually annotated by monitoring the camera information. The labels are included in the AB_README.txt file alongside their frequency of appearance and in the metadata corresponding to each file published in this dataset. Note that the labels provided are coarse because they are not associated with a timestamp.</p>
<p>Detailed information about the trajectories, the human demonstrators, the robots, the sensors, and the software used to collect and process the data is also included in the AB_README.txt file published along with the data. </p>
<p> In Related Materials we point to software and hardware resources utilized in this project. </p>

<h3>CAMERA PARAMETERS </h3>
<p>Some Spot bag files contain parameters from Spot's built-in camera. The bag files lack the camera parameters from the Azure Kinect camera used for recording. To overcome this limitation, users have used camera intrinsic from Azure Kinect cameras that they have in the lab:</p>

<p>camera_matrix = np.array([[608.1159057617188, 0., 639.1864013671875], [0., 607.8717041015625, 363.0690612792969], [0., 0., 1.]])</p>

<p>dist_coeffs = np.array([0.6671934723854065, -3.1600592136383057, 0.0004678548430092633, -0.0004770300292875618, 1.8223274946212769, 0.5343461036682129, -2.9515035152435303, 1.7315576076507568])</p>

<p> Robot camera extrinsics were not recorded. Users may be able to empirically estimate the extrinsic values based upon the observed positions of the camera and LiDAR from the ifollowing mage of the robots.</p>
<p>Karnan, Haresh; Nair, Anirudh; Xiao, Xuesu; Warnell, Garrett; Pirk, Soeren; Toshev, Alexander; Hart, Justin; Biswas, Joydeep; Stone, Peter, 2022, "AB_robots_sensors.png", Socially Compliant Navigation Dataset (SCAND), https://doi.org/10.18738/T8/0PRYRH/7LIACN, Texas Data Repository, V5</p>

<h3>DATA USAGE EXAMPLES</h3>
<p>In Related Publications, users can find references to publications describing this study including examples of experiments conducted with the dataset.</p>
<p>The objective behind the SCAND dataset is to provide a rich set of socially compliant robot navigation demonstrations. An imitative policy was trained using the behavior cloning algorithm. Below is an example figure of how the behavior cloned policy trained using SCAND was tested using real-world deployments in a human participant study. Results were perceived to be socially compliant in comparison to the classical move_base navigation stack.</p>
<p><img src="https://dataverse.tdl.org/api/access/datafile/267220" alt="Data reuse experiment" height="200"></p>
<p>This dataset can be used to develop autonomous mobile robots that can navigate within human crowds in a socially compliant manner, and to analyze human reactions and behaviors in the presence of mobile robots of different morphologies.</p>
<h3>DOWNLOADING LARGE DATASETS</h3>
<p>The entire SCAND is ~ 400 gig of size. A few individual rosbag files are larger than 4 gig. These characteristics present limitations for downloading the entire dataset as well as individual large files through the DataVerse interface. Users that want to download data in bulk can use the download_data. py script provided in this dataset.</p>