WireFishing-M

This dataset, WireFishing-M, contains multimodal sensor data collected from a robotic cable insertion task using a Franka Emika Panda 7-DOF robotic manipulator, an Allegro Robot Hand, and a DIGIT tactile sensor. The task simulates the "wire fishing" operation, where flexible cables are inserted into a transparent L-shaped PVC pipe. Data collected includes high-resolution tactile images, synchronized RGB video from multiple views, robot joint states, end-effector poses, and force estimates. Seven different types of cables were used to capture a range of deformable object behaviors.

Due to Mendeley’s file size limitations, we provide a subset of representative trials for each cable type in this repository. The full dataset is accessible via Dropbox at the following link:
https://www.dropbox.com/scl/fo/0fi6vh4yx42rayw8q3g2b/ADai0G6wGWgbERmVV2MNZSo?rlkey=tww2m03qbenj1npr7czzryndy&st=5b4rgsdd&dl=0

File Naming Convention
Cable_(number)_Human: Trials where the robot hand was manually guided by a human to insert the cable. These trials have a higher success rate and represent expert-like insertion behavior.
Cable_(number)_Random: Trials where the robot performed insertions from randomized positions and orientations, leading to more variable success rates and diverse contact conditions.

Each top-level folder contains subfolders named by the data collection date (e.g., 20241204). Within each date folder, multiple subfolders are named by trial number (e.g., trial_1, trial_2), but these numbers do not correspond to specific experimental conditions—they are simply used to organize the data and prevent excessive file counts in a single directory.

Inside each trial folder, the following five folders are included:
global_view/: Contains bottom-view RGB camera images (.jpg) captured from beneath the robot workspace.
inner_view/: Contains side-view images (.jpg) of the pipe opening used to visually verify task success or failure.
local_view/: Contains front-view RGB camera images (.jpg) showing the robot and cable during insertion.
tactile_view/: Contains grayscale tactile images (.jpg) from the DIGIT sensor. Each image is vertically stacked to show three fingertips: index finger (top), thumb (middle), and middle finger (bottom).
robot_data/: Contains a single .npy file with time-series robot data. The data array format is as follows:
[0–2]: End-effector force (Fx, Fy, Fz)
[3–5]: End-effector torque (Tx, Ty, Tz)
[6–11]: End-effector pose (position x, y, z and quaternion x, y, z, w)
[12–18]: Joint angles of the 7-DOF Franka Emika Panda robot
[19]: Insertion success label (1 = success, 0 = failure)