arkimjh/MMSI-train

---
license: cc-by-4.0
task_categories:
- video-text-to-text
- visual-question-answering
language:
- en
tags:
- video
- social-interaction
- training
- multi-person
- gaze
- gesture-recognition
size_categories:
- 1K<n<10K
---

# MMSI Train Dataset

## Description

MMSI training dataset containing original and extended video clips with QA annotations for social reasoning tasks.

## Dataset Statistics

| Type | Videos | Size |
|------|--------|------|
| Original Videos | 6,544 | ~1.5 GB |
| Extended Videos | 5,567 | ~18 GB |
| **Total** | **12,111** | **~19.5 GB** |

| JSON | Description |
|------|-------------|
| mmsi_train_mpp_ctx5.json | MPP task, 5-context |
| mmsi_train_mpp_ctx15.json | MPP task, 15-context |
| mmsi_train_pcr_ctx5.json | PCR task, 5-context |
| mmsi_train_pcr_ctx15.json | PCR task, 15-context |
| mmsi_train_sti_ctx5.json | STI task, 5-context |
| mmsi_train_sti_ctx15.json | STI task, 15-context |

## Dataset Structure

```
MMSI-train/
├── json/
│ ├── mmsi_train_mpp_ctx5.json
│ ├── mmsi_train_mpp_ctx15.json
│ ├── mmsi_train_pcr_ctx5.json
│ ├── mmsi_train_pcr_ctx15.json
│ ├── mmsi_train_sti_ctx5.json
│ └── mmsi_train_sti_ctx15.json
└── tars/
├── mmsi_train_original_videos_part001.tar
├── mmsi_train_extended_videos_part001.tar
├── mmsi_train_extended_videos_part002.tar
└── ...
```

## Usage

```python
from huggingface_hub import snapshot_download

# Download everything
snapshot_download(
repo_id="arkimjh/MMSI-train",
repo_type="dataset",
local_dir="./MMSI-train"
)

# Extract video archives
import tarfile
from pathlib import Path

for tar_file in Path("./MMSI-train/tars").glob("*.tar"):
with tarfile.open(tar_file) as tf:
tf.extractall("./MMSI-train/videos/")
```

## Citation

TBD