Learning contact-rich whole-body manipulation with example-guided reinforcement learning

Humans employ a diversity of skills and strategies to effectively
manipulate various objects, ranging from dexterous in-hand manipulation
(fine motor skills) to complex whole-body manipulation (gross motor
skills). The latter involves full-body engagement and extensive contact
with various body parts beyond just the hands, where the compliance of our
skin and muscles plays a crucial role in increasing contact stability and
mitigating uncertainty. For robots, synthesizing such contact-rich
behaviors has fundamental challenges due to the rapidly growing
combinatorics inherent to this amount of contact, making explicit
reasoning about all contact interactions intractable. We explore the use
of example-guided reinforcement learning to generate robust whole-body
skills for the manipulation of large and unwieldy objects. Our method’s
effectiveness is demonstrated on Toyota Research Institute’s Punyo robot,
a humanoid upper-body with highly deformable, pressure-sensing skin.
Training is conducted in simulation with only a single example motion per
object manipulation task, and policies are easily transferred to hardware
owing to domain randomization and the robot’s compliance. The resulting
agent can manipulate various everyday objects, such as a water jug and
large boxes, in a similar fashion to the example motion. Additionally, we
show blind dexterous whole-body manipulation, relying solely on
proprioceptive and tactile feedback without object pose tracking. Our
analysis highlights the critical role of compliance in facilitating
whole-body manipulation with humanoid robots.