Let's DENSE: A Novel Protocol for Efficiently Collecting Dense and Diverse Data for Tactile Slip Detection in Robotic Grasping

Tactile dataset generated using a newly proposed, object-agnostic, protocol - the Dense Protocol- that allows collecting tactile data for training robust slip detection models during robotic object grasping. The proposed protocol steps are described in detail in [1], and a summary is provided below. The DENSE protocol The DENSE protocol is split into three main stages: generation of a valid set of grasp poses for each object; robotic object grasp execution and tactile data collection; and data labelling. In this work, the grasps generated by the proposed protocol are performed using the EZGripper, a low-cost and underactuated dual-fingered robotic gripper. To collect tactile data, a uSkin tactile sensor, based on magnetic technology [2], is installed on a single finger. Grasp pose sampling: Since grasp configurations executed by autonomous systems can result in a wide range of contact points between the fingertips of the robot and the object, we believe that the training dataset should be composed of different gripper-object interactions. For instance, the position of the sensor with respect to the object centre of mass will dictate the intensity (i.e. direction of rotation or translation, and velocity) of the slip. This is especially true when the fingertips are only partially contacting objects. To generate such variability in a repeatable and controlled manner, we propose to discretise the object dimensions with a resolution equal to half the smallest side of the bounding box of the tactile sensor, *d*, which will be referred to as the object discretisation step. The discretisation step was selected so that, during data collection, the sensor is in contact with the entirety of the (reachable) object surface at least twice across all generated grasp poses. Reducing the discretisation step would result in more grasp experiments, and therefore, more time to collect the data, while increasing it would result in the generated data not containing grasp information on some parts of the objects. For an arbitrary object placed on a table, the corresponding object discretisation step corresponds to defining a virtual 2-dimensional grid - contained within a vertical plane aligned with the major axis of the object - with a spatial resolution *d*, and covering the whole space that the fingertips of the robot can reach for a given orientation of the gripper, which is assumed to be always vertically aligned with gravity. Each vertex of this virtual grid corresponds to a candidate position defining the centre of contact between the fingertips and the object. To avoid collecting data that is unrepresentative of the behaviour we try to detect, grasp configurations should be kept if and only if the object remains within the fingers of the gripper when the latter closes. Similarly, all grasp configurations resulting in undesired contact between the table and the end-effector should also be discarded. If the width of a given object is not divisible by *d*, padding is applied on each side of the object to result in a discrete number of contact points along this axis. Grasp execution: A grasp experiment for an isolated object with a given pose on a table consists of the following pick-and-lift procedure: 1) Move the robot arm to a given grasp configuration 2) Close the robot fingers (i.e. grasp the object) 3) Start collecting tactile data, tb 4) Raise the robot arm to a pre-defined pose, i.e. the robot lifts the object, lasting some period of time Tr (in this work, Tr ≈ 0.2s) 5) Once the robot arm is static, wait a period of 2 seconds,Ts(Ts = 2s) 6) Stop recording tactile data, te For a given grasp pose, the above steps can be repeated R times. A higher number of repetitions R allows for recording more variability related to experimental errors (e.g. hardware controller, object placement) but requires more time to execute overall. While running each grasp experiment for the different grasp configurations, some experiments will lead to the object slipping (with rotational or translational momentum) from the gripper as soon as the robot arm moves, while others will remain firmly grasped. We believe that capturing both behaviours is crucial to train classifiers that can cope with different interactions between a gripper and a set of objects. In practice, to simplify the experimental procedure, during the grasp execution step, instead of generating individual robot joint states corresponding to each sampled grasp of a given object, we predefine one robot joint state for each height of the grasps to be explored (e.g. three grasp heights for the cuboid wood bar object). Furthermore, graph paper is attached to the tabletop, so for each grasp height, the object is moved by a step of d cm horizontally (along the major axis of each object) until its pose matches the sampled robot grasp configuration, allowing for efficient data collection with minimal overhead. This work assumes rigid objects, and the grasp force to be the same for all objects and to be kept constant during the data collection procedure. The grasp force was chosen with the following criteria: large enough so that grasp poses near to the objects’ center of mass would generally lead to stable grasps (non-slips); small enough so that some of the grasp poses would generate slips; small enough not to damage any of the objects. Based on these criteria, the chosen grasp force was approximately 10 N, which is within the sensing range of the tactile sensor (0-14 N, as reported by the manufacturer). Data labelling: Since data will be collected during the execution of grasps that involve the movement of a robotic arm, automatic labelling methods of the individual tactile sample are unfeasible without somehow controlling or limiting the grasping task. Instead, to label slips for each collected tactile sample, we assume that all samples from a grasping experiment correspond to the same label (Slip or No-Slip). After each experiment, all individual tactile samples recorded (sampled at 100 Hz) are labelled according to whether the experimenter observed a slipping or a stable grasp during that experiment. We believe such an assumption to be reasonable since we record data only for 2.2 seconds (from the moment the object is raised above the table, Tr + Ts ), which does not allow objects to fall, but only to start slipping or to remain stable. If objects were to fall within the first two seconds after lifting them, we would advise experimenters to increase the grasping strength while making sure not to damage the object. In other words, we propose to rely on the observation of experimenters to label whether all samples of the sequence correspond to slips (object moves within the fingers of the robot) or a static contact. Although the resulting labels will correspond to an approximation of the real events, this approach saves labelling time and resources. Data collection: In this work, to validate the effectiveness of the proposed protocol, data is collected with seven objects. These objects are an empty cardboard can, an unopened soda can, three cuboid wood bars - two of them wrapped in either baking paper or duct tape to change their respective coefficients of friction - a metal bar, and a brush. This set of objects includes a variety of shapes (cuboid, cylindrical, composite), weights (between 47 and 356 g), dimensions and textures. Given the variety of object properties chosen (size, mass, and friction), it is observed that both slips and stable grasps can occur either under partial contact or full contact between the sensor and the objects, depending largely on the distance between the centre of the grasp and the centre of mass of the object. For each sampled grasp pose of each object, R = 10 experiments are collected and labelled, leading to a total of 2100 grasps across the seven objects. This corresponds to 70 minutes of tactile data, and approximately 420K tactile samples, with 230K labelled as slip events, and 190K as non-slip events. The resulting dataset has been made publicly available.