agow1/ego-masquerade

---
license: mit
task_categories:
- object-detection
tags:
- hand-object-interaction
- egocentric
- epic-kitchens-format
- hand-detection
size_categories:
- n<1K
---

# Hand-Object Interaction Dataset

Egocentric hand-object interaction dataset with HOI detections in EPIC-Kitchens format.

## Dataset Summary

| Task | Videos |
|------|--------|
| assembling | 11 |
| group_objects | 10 |
| pick_n_place | 20 |
| put_bin | 10 |
| **Total** | **51** |

All frames are extracted at **1920x1080** resolution.

## Directory Structure

```
<task_name>/
<video_id>/
rgb_frames.zip # Zipped extracted frames (frame_0000001.jpg, ...)
hand_det.pkl # Phantom hand detection format
```

## File Descriptions

- **`rgb_frames.zip`**: Zipped directory of extracted video frames at 1920x1080 resolution. Frame filenames follow the pattern `frame_NNNNNNN.jpg`.
- **`hand_det.pkl`**: Pickled hand detections in Phantom format. Contains per-frame bounding boxes for hands (left/right) and active objects with contact state.

## Usage

```python
import pickle
import zipfile
from pathlib import Path

# Extract frames
with zipfile.ZipFile("assembling/1/rgb_frames.zip", "r") as z:
z.extractall("assembling/1/rgb_frames")

# Load hand detections
with open("assembling/1/hand_det.pkl", "rb") as f:
hand_dets = pickle.load(f)
```

## Processing Pipeline

1. **Frame extraction** — Extract frames from source videos at 1920x1080
2. **Hand-object detection** — Faster R-CNN based detector to extract hand and object bounding boxes
3. **Detection conversion** — Convert raw detections to EPIC-Kitchens format
4. **Phantom conversion** — Convert to Phantom hand detection format

## Citation

If you use this dataset, please cite:

```bibtex
@misc{agow-hoa-dataset,
title={Hand-Object Interaction Dataset},
author={AGOW},
year={2026},
}
```