Learning to Grasp Unknown Objects in Domestic Environments with GP-net+

This record includes data for the paper "Learning to Grasp Unknown Objects in Domestic Environments", currently under review.Simulation environment with pre-trained GP-net+ modelThe paper presents a simulation environment for grasping objects in domestic environments. The presented objects and furniture units, as well as a pre-trained GP-net+ model can be found in the "gpnetplus_simulation_data.zip" file. After this zip file is downloaded, it can be unpacked it into the GP-net+ directory. It includes all necessary data to use the simulation environment, for example, for testing GP-net+ or other grasping models in simulated domestic environments.

ROS model

The paper additionally presents an ROS package that can be deployed for grasping unknown objects in domestic environments with simulated or real robots. We make a ROS-compatbile model of GP-net+ available in the "ros_gpnet_plus.zip" file, which can be used with the ROS package.

Training dataset

We used the simulation environment in our paper to generate a training dataset and train GP-net+. This training dataset is included in this record and can be used to replicate our results or train modifications of GP-net+.

To improve handling of the training dataset (total size 25GB+), we split the dataset into several .zip files, named val.zip (validation data) and train_[0-6].zip (training data). Download all files individually and extract them into a single folder. Combine all files train_[0-6].zip directory into a single directory called 'train', for example, by using the 'move_train_data.sh' script provided.The final structure for the dataset should look similar to this:gpnet_data

|-- val

     |-- depth_image_0000000.npz

     |-- depth_image_0000001.npz

     ...

     |--segmask_image_0052346.npz

|-- train

    |-- depth_image_0000000.npz

    |-- depth_image_0000001.npz

    ...

    |-- segmask_image_0602506.npz

    |-- segmask_image_0602507.npz

For generation of the training and simulation data, the following mesh databases have been used:B. Calli, A. Walsman, A. Singh, S. Srinivasa, P. Abbeel, and A. M. Dollar,"Benchmarking in Manipulation Research: Using the Yale-CMU-Berkeley Object and Model Set," IEEE Robotics and Automation Magazine, vol. 22, no. 3, pp. 36–52, 2015A. Singh, J. Sha, K. S. Narayan, T. Achim, and P. Abbeel, "BigBIRD: A large-scale 3D database of object instances," 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 509–516, 2014.A. X. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu, "ShapeNet: An Information-Rich 3D Model Repository," Tech. Rep. arXiv:1512.03012 [cs.GR], Stanford University — Princeton University — Toyota Technological Institute at Chicago, 2015.

D. Morrison, P. Corke, and J. Leitner, "EGAD! An Evolved Grasping Analysis Dataset for Diversity and Reproducibility in Robotic Manipulation," IEEE Robotics and Automation Letters, vol. 5, no. 3, pp. 4368–4375, 2020