Data from: SimPLE: A visuotactile method learned in simulation to precisely pick, localize, regrasp, and place objects

Existing robotic systems have a clear tension between generality and
precision. Deployed solutions for robotic manipulation tend to fall into
the paradigm of one robot solving a single task, lacking precise
generalization, i.e., the ability to solve many tasks without compromising
on precision. This paper explores solutions for precise and general
pick-and-place. In precise pick-and-place, i.e. kitting, the robot
transforms an unstructured arrangement of objects into an organized
arrangement, which can facilitate further manipulation. We propose simPLE
(simulation to Pick Localize and PLacE) as a solution to precise
pick-and-place. simPLE learns to pick, regrasp and place objects
precisely, given only the object CAD model and no prior experience. We
develop three main components: task-aware grasping, visuotactile
perception, and regrasp planning. Task-aware grasping computes affordances
of grasps that are stable, observable, and favorable to placing. The
visuotactile perception model relies on matching real observations against
a set of simulated ones through supervised learning. Finally, we compute
the desired robot motion by solving a shortest path problem on a graph of
hand-to-hand regrasps. On a dual-arm robot equipped with visuotactile
sensing, we demonstrate pick-and-place of 15 diverse objects with simPLE.
The objects span a wide range of shapes and simPLE achieves successful
placements into structured arrangements with 1mm clearance over 90% of the
time for 6 objects, and over 80% of the time for 11 objects.