allenai/WildDet3D-Bench

---
license: cc-by-4.0
task_categories:
- object-detection
tags:
- 3d-object-detection
- 3d-bounding-box
- monocular-3d
- in-the-wild
- benchmark
pretty_name: WildDet3D Benchmark
size_categories:
- 1K<n<10K
---

# WildDet3D Benchmark

In-the-wild 3D object detection benchmark (val and test splits) from COCO, LVIS, and Objects365.

| Split | Images | Annotations | Categories |
|-------|--------|-------------|------------|
| Val | 2,470 | 9,256 | 785 |

**Note:** The test set is held out for hidden evaluation and is not publicly available. Please submit predictions to [TODO: evaluation server] for test set evaluation.

## Download

```bash
pip install huggingface_hub

# Download everything
huggingface-cli download allenai/WildDet3D-Bench --repo-type dataset --local-dir WildDet3D-Bench
```

After downloading, extract depth and camera archives:

```bash
cd WildDet3D-Bench
tar xzf packed/depth_val.tar.gz
tar xzf packed/camera_val.tar.gz
```

## Directory Structure

```
WildDet3D-Bench/
├── annotations/
│ ├── InTheWild_v3_val.json
│ ├── InTheWild_v3_test.json
│ └── InTheWild_v3_val_class_map.json
├── depth/{val,test}/ # Monocular depth maps
│ └── {source}_{formatted_id}.npz # float32, mm (divide by 1000 for meters)
└── camera/{val,test}/ # Camera intrinsics
└── {source}_{formatted_id}.json # K + image_size
```

## Source Images

Images must be downloaded from their original sources:

| Source | Directory | Download |
|--------|-----------|----------|
| COCO val2017 | `images/coco_val/` | https://cocodataset.org/ |
| COCO train2017 | `images/coco_train/` | https://cocodataset.org/ |
| Objects365 val | `images/obj365_val/` | https://www.objects365.org/ |

## Annotation Format (COCO3D)

Same format as [WildDet3D-Data](https://huggingface.co/datasets/weikaih/WildDet3D-Data). Key fields:

- **`valid3D`**: `true` = valid 3D annotation, `false` = 3D box filtered out (2D box still valid)
- **`center_cam`**: 3D box center in camera coordinates (meters)
- **`dimensions`**: `[width, height, length]` in meters (Omni3D convention)
- **`R_cam`**: 3x3 rotation matrix (gravity-aligned, local Y = up)
- **`bbox3D_cam`**: 8 corner points of the 3D bounding box
- **`bbox2D_proj`**: 2D bounding box `[x1, y1, x2, y2]`

## License

CC BY 4.0