ZhengGuangze/Flock4D

---
license: cc-by-nc-sa-4.0
---
# Flock4D (tar.gz format)

This dataset contains the Flock4D dataset converted to the VLBM/Flock4D-compatible format.
To facilitate easier downloading and storage, the 1000 sequences have been compressed into `.tar.gz` archives in chunks of 50 sequences per archive.

## Dataset Description

- **Source**: Flock4D
- **Format**: VLBM / Flock4D-compatible per-sequence layout compressed into tar.gz chunks
- **Contents**: RGB images, dense depth maps, 2D/3D trajectories, camera intrinsics and extrinsics, visibility masks, and scene metadata

### Scale

Flock4D provides 1000 sequences of birds (flocks) flying in various environments.
There are 24 species of birds included:
`cannada_goose`, `common_starling`, `cormorant`, `crane`, `crested_bis`, `crow`, `dove`, `duck`, `dunlin`, `eagle`, `egret`, `flamingo`, `jackdaw`, `mallard`, `parrot`, `pelican`, `pigeon`, `red_billed_starling`, `seagull`, `snow_goose`, `stork`, `swallow`, `tit`, `warbler`.

## Dataset Structure

The original dataset is structured by sequence. In this Hugging Face repository, the sequences are grouped and compressed into tarballs (e.g., `flock4d_00000_00049.tar.gz`).
After extracting a `.tar.gz` archive, each sequence directory follows this layout:

```
{Species}_{Background}_4k_{ID}/
├── rgbs/
│ ├── rgb_00000.jpg
│ ├── rgb_00001.jpg
│ └── ...
├── depths/
│ ├── depth_00000.npz
│ ├── depth_00001.npz
│ └── ...
├── intrinsics.npy
├── extrinsics.npy
├── trajs_2d.npy
├── trajs_3d.npy
├── visibilities.npy
└── scene_info.json
```

### File Descriptions

- `rgbs/`: RGB frames saved as JPEG (`rgb_XXXXX.jpg`).
- `depths/`: Dense depth maps saved as compressed NumPy archives (`depth_XXXXX.npz`). Each archive stores a float16 array.
- `intrinsics.npy`: Camera intrinsic matrices for each frame `(T, 3, 3)`.
- `extrinsics.npy`: World-to-camera extrinsic matrices (W2C) for each frame `(T, 4, 4)`.
- `trajs_2d.npy`: 2D trajectories `(T, N, 2)` -- pixel coordinates (x, y).
- `trajs_3d.npy`: 3D trajectories `(T, N, 3)` -- world-space coordinates (x, y, z); zero-filled where invisible.
- `visibilities.npy`: Visibility flags `(T, N)` (1.0 visible, 0.0 not visible).
- `scene_info.json`: JSON file with per-sequence metadata, including camera properties and scene assets.

## Usage Example (Python)

To use the dataset, first download the tarballs and extract them:

```bash
mkdir -p data/flock4d
tar -xvf flock4d_00000_00049.tar.gz -C data/flock4d/
```

Then load the annotations in Python:

```python
import numpy as np
from PIL import Image
from pathlib import Path
import json

seq_dir = Path("data/flock4d/cannada_goose_abandoned_parking_4k_313")

# Load annotations
trajs_2d = np.load(seq_dir / "trajs_2d.npy") # (T, N, 2)
trajs_3d = np.load(seq_dir / "trajs_3d.npy") # (T, N, 3)
vis = np.load(seq_dir / "visibilities.npy") # (T, N)
intrinsics = np.load(seq_dir / "intrinsics.npy") # (T, 3, 3)
extrinsics = np.load(seq_dir / "extrinsics.npy") # (T, 4, 4)

# Load context
frame_idx = 0
rgb = Image.open(seq_dir / "rgbs" / f"rgb_{frame_idx:05d}.jpg")
depth_npz = np.load(seq_dir / "depths" / f"depth_{frame_idx:05d}.npz")
depth = depth_npz['depth'] # float16 array (H, W)

# Load scene info
with open(seq_dir / "scene_info.json", 'r') as f:
scene_info = json.load(f)

print(scene_info)
```