ryushinn/Taichi-HD-BBox-Keypoint

---
license: mit
task_categories:
- image-to-text
tags:
- human-pose-estimation
- keypoints
- bounding-boxes
- rtmpose
- mmpose
- taichi-hd
pretty_name: Taichi-HD BBox Keypoint
size_categories:
- 100K<n<1M
---

# Taichi-HD BBox Keypoint

This dataset contains estimated person bounding boxes and COCO-17 body keypoints for [`ryushinn/Taichi-HD`](https://huggingface.co/datasets/ryushinn/Taichi-HD). It is an annotations-only sidecar dataset: it does **not** duplicate the source images.

Rows preserve the same split names and row order as the source dataset. To pair an annotation row with its image, load the same split from `ryushinn/Taichi-HD` and use the same row index.

## Dataset structure

| Split | Rows | Detected person rows |
| --- | ---: | ---: |
| `train` | 887,334 | 887,302 |
| `test` | 64,199 | 64,191 |

Each row has exactly four columns:

| Column | Type / shape | Description |
| --- | --- | --- |
| `bboxes_xyxy` | `float32[4]` | Highest-confidence detected person box as `[x1, y1, x2, y2]` in source-image pixel coordinates. |
| `bbox_scores` | `float32` | Confidence score for the selected person bounding box. |
| `keypoints_xy` | `float32[17][2]` | Estimated COCO-17 body keypoints in source-image pixel coordinates. |
| `keypoint_scores` | `float32[17]` | Confidence score for each keypoint. |

**For missing detections, `bboxes_xyxy` and `keypoints_xy` contain NaN coordinates, `bbox_scores` is `0.0`, and `keypoint_scores` contains zeros.**

## Keypoint order

The 17 keypoints follow the COCO body convention:

| Index | Name |
| ---: | --- |
| 0 | nose |
| 1 | left_eye |
| 2 | right_eye |
| 3 | left_ear |
| 4 | right_ear |
| 5 | left_shoulder |
| 6 | right_shoulder |
| 7 | left_elbow |
| 8 | right_elbow |
| 9 | left_wrist |
| 10 | right_wrist |
| 11 | left_hip |
| 12 | right_hip |
| 13 | left_knee |
| 14 | right_knee |
| 15 | left_ankle |
| 16 | right_ankle |

Skeleton edges used for visualization:

```python
[
(15, 13), (13, 11), (16, 14), (14, 12),
(11, 12), (5, 11), (6, 12), (5, 6),
(5, 7), (6, 8), (7, 9), (8, 10),
(1, 2), (0, 1), (0, 2), (1, 3),
(2, 4), (3, 5), (4, 6),
]
```

## Model configuration

Annotations were generated with OpenMMLab models:

- Person detector: **RTMDet-M COCO person detector**
- Config: `rtmdet_m_8xb32-300e_coco.py`
- Checkpoint: `rtmdet_m_8xb32-300e_coco_20220719_112220-229f527c.pth`
- Body pose estimator: **RTMPose-M COCO**
- Config: `rtmpose-m_8xb256-420e_coco-256x192.py`
- Checkpoint: `rtmpose-m_simcc-coco_pt-aic-coco_420e-256x192-d8dd5ca4_20230127.pth`

Reference: Jiang et al., **RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose**, arXiv:2303.07399.

## Usage

```python
from datasets import concatenate_datasets, load_dataset

images = load_dataset("ryushinn/Taichi-HD", split="train")
ann = load_dataset("ryushinn/Taichi-HD-BBox-Keypoint", split="train")

ds = concatenate_datasets([images, ann], axis=1)
# You may want to filter out low-confidence detections
# ds = ds.filter(lambda row: [v > 0.05 for v in row["bbox_scores"]], batched=True)
idx = 0
image = ds[idx]["image"]
bbox = ds[idx]["bboxes_xyxy"]
keypoints_xy = ds[idx]["keypoints_xy"]
keypoint_scores = ds[idx]["keypoint_scores"]
```

## Visualization previews

The following preview images show source frames with the estimated person bounding box and COCO-17 skeleton overlayed.

| Train row 0 | Train row 443667 |
| --- | --- |
|  |  |

| Test row 0 | Test row 32099 |
| --- | --- |
|  |  |

## Notes

These annotations are model-estimated pseudo-labels, not manual ground-truth annotations. They are intended for research workflows where reproducible person bounding boxes and COCO-17 body pose estimates are useful alongside the original `ryushinn/Taichi-HD` videos/frames.