Octopus1/HiSync

---
license: cc-by-nc-4.0
task_categories:
- other
tags:
- gesture-recognition
- action-recognition
- multimodal
- IMU
- video
size_categories:
- 100K<n<1M
---

# Directory Structure

Published data is organized by collection batch ID.

```text
HiSync_publish/
├── 1/ # Batch ID
│ ├── user1_20250726_132447/ # Sample directory
│ │ ├── cam_1/
│ │ ├── cam_2/
│ │ ├── cam_3/
│ │ ├── person_keypoints.json
│ │ └── meta.json
│ ├── user2_20250726_135244/
│ │ └── ...
│ └── IMU/
│ ├── IMU_Palm/
│ │ └── *.csv
│ ├── IMU_Ring/
│ │ └── *.csv
│ └── IMU_Wrist/
│ └── *.csv
├── 2/
│ └── ...
└── 18/
└── ...
```

Notes:

1. Each sample directory is named `userX_YYYYMMDD_HHMMSS`.
2. Each sample directory contains camera data, `person_keypoints.json`, and `meta.json`.
3. IMU data is aggregated per batch under `batch_id/IMU/IMU_{Palm|Ring|Wrist}`, not in individual sample directories.

# Data Format

Example `meta.json` for a sample:

```json
{
"user": "user10",
"action": "Right",
"perspective": "Eye-level",
"distance": "10-15m",
"camera": {
"cam_0": "telephone",
"cam_2": "iphone",
"cam_1": "cam"
},
"IMU": {
"Palm": {
"timestamp": "5/IMU/IMU_Palm/calibrated_imu_20250727_150254.csv"
},
"Ring": {
"timestamp": "5/IMU/IMU_Ring/calibrated_imu_20250727_150254.csv"
},
"Wrist": {
"timestamp": "5/IMU/IMU_Wrist/calibrated_imu_20250727_150254.csv"
}
}
}
```

Field Constraints:

1. `action` is standardized to: `Right`, `Left`, `Approach`, `Retreat`, `Summon`, `Ascend`, `Descend`, `No-Gesture`.
2. `perspective` is standardized to: `Upward`, `Eye-level`, `Downward`.
3. `distance` is standardized as range strings: `3-5m`, `5-10m`, `10-15m`, `15-20m`, `20-25m`, `25-34m`.
4. `IMU.*.timestamp` may be `null`; handle null values during parsing.

> A small portion of the data may have missing camera or IMU modalities. Ensure robust error handling during reading.

Example `person_keypoints.json`:

```json
{
"0": [
{
"frame_idx": 0,
"filename": "frame_0000.png",
"keypoints": [
[1204.31, 181.52, 0.9949],
[1208.77, 170.23, 0.9781],
[1198.12, 170.84, 0.9675],
[1216.43, 181.65, 0.8360],
[1187.55, 183.20, 0.6951]
],
"bbox": [1133, 106, 1373, 813]
},
{
"frame_idx": 1,
"filename": "frame_0001.png",
"keypoints": [
[1204.88, 181.46, 0.9955],
[1209.06, 170.34, 0.9761],
[1197.93, 170.95, 0.9748]
],
"bbox": [1132, 106, 1374, 813]
}
],
"1": [
{
"frame_idx": 0,
"filename": "frame_0000.png",
"keypoints": [],
"bbox": [0, 0, 0, 0]
}
]
}
```

Notes:

1. Top-level keys (e.g., `"0"`, `"1"`) represent person IDs (string format).
2. Each camera corresponds to a frame list with elements containing `frame_idx`, `filename`, `keypoints`, and `bbox`.
3. Each point in `keypoints` is `[x, y, score]`, following COCO-17 order.
4. When no person is detected in a frame, `keypoints` may be an empty array; handle this during parsing.