FreeTacMan

# 📦 FreeTacman
## Robot-free Visuo-Tactile Data Collection System for Contact-rich Manipulation

## 🎯 Overview

This dataset supports the paper **[FreeTacman: Robot-free Visuo-Tactile Data Collection System for Contact-rich Manipulation](http://arxiv.org/abs/2506.01941)**. It contains a large-scale, high-precision visuo-tactile manipulation dataset with over 3000k visuo-tactile image pairs, more than 10k trajectories across 50 tasks.
Please refer to our 🚀 [Website](http://opendrivelab.com/freetacman) | 📄 [Paper](http://arxiv.org/abs/2506.01941) | 💻 [Code](https://github.com/OpenDriveLab/FreeTacMan) | 🛠️ [Hardware Guide](https://docs.google.com/document/d/1Hhi2stn_goXUHdYi7461w10AJbzQDC0fdYaSxMdMVXM/edit?addon_store&tab=t.0#heading=h.rl14j3i7oz0t) | 📺 [Video](https://opendrivelab.github.io/FreeTacMan/landing/FreeTacMan_demo_video.mp4) | 🌐 [X](https://x.com/OpenDriveLab/status/1930234855729836112) for more details.

## 🔬 Potential Applications
The FreeTacman dataset enables diverse research directions in visuo-tactile learning and manipulation:

- **System Reproduction**: For researchers interested in hardware implementation, you can reproduce FreeTacMan from scratch using our 🛠️ [Hardware Guide](https://docs.google.com/document/d/1Hhi2stn_goXUHdYi7461w10AJbzQDC0fdYaSxMdMVXM/edit?addon_store&tab=t.0#heading=h.rl14j3i7oz0t) and 💻 [Code](https://github.com/OpenDriveLab/FreeTacMan).

- **Multimodal Imitation Learning**: Transfer to other LED-based tactile sensors (such as GelSight) for developing robust multimodal imitation learning frameworks.

- **Tactile-aware Grasping**: Utilize the dataset for pre-training tactile representation models and developing tactile-aware reasoning systems.

- **Simulation-to-Real Transfer**: Leverage the dynamic tactile interaction sequences to enhance tactile simulation fidelity, significantly reducing the sim2real gap.

## 📂 Dataset Structure

The dataset is organized into 50 task categories, each containing:
- **Video files**: Synchronized video recordings from the wrist-mounted and visuo-tactile cameras for each demonstration
- **Trajectory files**: Detailed tracking data for tool center point pose and gripper distance

## 🧾 Data Format

### Video Files

- **Format**: MP4
- **Views**: Wrist-mounted camera and visuo-tactile camera perspectives per demonstration

### Trajectory Files

Each trajectory file contains the following data columns:

#### Timestamp
- `timestamp` - Unix Timestamp

#### Tool Center Point (TCP) Data

- `TCP_pos_x`, `TCP_pos_y`, `TCP_pos_z` - TCP position
- `TCP_euler_x`, `TCP_euler_y`, `TCP_euler_z` - TCP orientation (euler angles)
- `quat_w`, `quat_x`, `quat_y`, `quat_z` - TCP orientation (quaternion representation)

#### Gripper Data
- `gripper_distance` - Gripper opening distance

## 📝 Citation

If you use this dataset in your research, please cite:

```bibtex
@article{wu2025freetacman,
title={Freetacman: Robot-free visuo-tactile data collection system for contact-rich manipulation},
author={Wu, Longyan and Yu, Checheng and Ren, Jieji and Chen, Li and Jiang, Yufei and Huang, Ran and Gu, Guoying and Li, Hongyang},
journal={arXiv preprint arXiv:2506.01941},
year={2025}
}
```

## 💼 License

This dataset is released under the MIT License. See LICENSE file for details.

## 📧 Contact

For questions or issues regarding the dataset, please contact: Longyan Wu ([e-mail]).

# 📦 FreeTacman
## 面向密集接触操作的无机器人视觉-触觉（visuo-tactile）数据采集系统

## 🎯 概述
本数据集配套论文**《FreeTacman：面向密集接触操作的无机器人视觉-触觉数据采集系统》**（arXiv:2506.01941），包含大规模高精度视觉-触觉操作数据集，涵盖超300万对视觉-触觉图像、覆盖50个任务的1万余条轨迹。
更多详情请访问：🚀 [项目官网](http://opendrivelab.com/freetacman) | 📄 [学术论文](http://arxiv.org/abs/2506.01941) | 💻 [开源代码](https://github.com/OpenDriveLab/FreeTacMan) | 🛠️ [硬件搭建指南](https://docs.google.com/document/d/1Hhi2stn_goXUHdYi7461w10AJbzQDC0fdYaSxMdMVXM/edit?addon_store&tab=t.0#heading=h.rl14j3i7oz0t) | 📺 [演示视频](https://opendrivelab.github.io/FreeTacMan/landing/FreeTacMan_demo_video.mp4) | 🌐 [X平台账号](https://x.com/OpenDriveLab/status/1930234855729836112)

## 🔬 潜在应用场景
FreeTacman数据集可支撑视觉-触觉学习与操作领域的多元研究方向：
- **系统复现**：针对硬件实现感兴趣的研究者，可通过硬件搭建指南与开源代码从零复现FreeTacMan系统。
- **多模态模仿学习**：可迁移至其他基于LED的触觉传感器（如GelSight）以构建鲁棒的多模态模仿学习框架。
- **触觉感知抓取**：利用该数据集预训练触觉表征模型，开发具备触觉感知能力的推理系统。
- **仿真到实机迁移**：借助动态触觉交互序列提升触觉仿真的真实度，显著缩小仿真到实机迁移差距。

## 📂 数据集结构
数据集按50个任务类别组织，每个类别包含：
- **视频文件**：每个演示轨迹同步采集的腕部安装相机与视觉-触觉相机的录制视频
- **轨迹文件**：工具中心点（Tool Center Point, TCP）位姿与夹爪间距的详细跟踪数据

## 🧾 数据格式

### 视频文件
- **格式**：MP4
- **视角**：每个演示轨迹包含腕部相机与视觉-触觉相机的双视角录制

### 轨迹文件
每个轨迹文件包含以下数据字段：

#### 时间戳
- `timestamp`：Unix时间戳

#### 工具中心点（TCP）数据
- `TCP_pos_x`、`TCP_pos_y`、`TCP_pos_z`：TCP位置坐标
- `TCP_euler_x`、`TCP_euler_y`、`TCP_euler_z`：TCP姿态（欧拉角表示）
- `quat_w`、`quat_x`、`quat_y`、`quat_z`：TCP姿态（四元数表示）

#### 夹爪数据
- `gripper_distance`：夹爪张开间距

## 📝 引用格式
若您在研究中使用该数据集，请引用如下文献：
bibtex
@article{wu2025freetacman,
title={Freetacman: Robot-free visuo-tactile data collection system for contact-rich manipulation},
author={Wu, Longyan and Yu, Checheng and Ren, Jieji and Chen, Li and Jiang, Yufei and Huang, Ran and Gu, Guoying and Li, Hongyang},
journal={arXiv preprint arXiv:2506.01941},
year={2025}
}

## 💼 许可证
本数据集采用MIT许可证发布，详情请参阅LICENSE文件。

## 📧 联系方式
若对数据集有任何疑问或问题，请联系：吴隆彦（邮箱：[e-mail]）。