Hurtubisedavid/CIN-VBMLR

---
license: cc-by-nd-4.0
task_categories:
- depth-estimation
- video-classification
- object-detection
tags:
- camera-calibration
- depth-from-defocus
- cinema
- arri
- machine-learning
pretty_name: CINE-VBMLR Dataset
size_categories:
- 10B<n<100B
---

# CINE-VBMLR: Cinema-grade Variable Blur Dataset for Camera Tracking

## Dataset Summary

**CINE-VBMLR** is a specialized dataset designed for high-end cinema camera calibration and tracking. It introduces a novel approach called **CINE-VBMLR** (Cinema Variational Bayesian Multinomial Logistic Regression), adapted from variable blur models originally used in eye-gaze estimation, to solve camera pose and intrinsic parameters in cinematic environments.

This dataset was acquired at **MELS Studios** using professional cinema equipment (ARRI), providing high-dynamic-range (HDR) imagery coupled with precise frame-by-frame Lens Data System (LDS) metadata.

The primary goal is to train and validate Python-based machine learning models that leverage **Depth from Defocus (DfD)** to improve camera tracking accuracy where traditional pinhole models fail (e.g., shallow depth of field shots).

## Dataset Structure

### Data Organization
The dataset is structured to separate raw linear pixel data from optical metadata:

* **`/data`**: Sequences of OpenEXR (`.exr`) files.
* **Format:** RGB Float16 (Half) or Float32.
* **Color Space:** Linear (converted from ARRI LogC3 via ACES/CST).
* **Resolution:** 3200 x 1800 (Source Resolution).
* **`/metadata`**: CSV files containing frame-accurate ARRI LDS data.

### Metadata Schema (LDS)
Each video frame corresponds to a row in the CSV files, containing ground truth values extracted via ARRI Meta Extract:

| Column Key | Description | Unit |
| :--- | :--- | :--- |
| `Master TC` | Source Timecode (aligned with EXR filenames) | HH:MM:SS:FF |
| `LDS Focus Distance` | Exact focus distance of the lens | Meters/Feet |
| `LDS Iris` | Aperture value (T-Stop) | T-Stop |
| `LDS Focal Length` | Focal length (constant or zooming) | mm |
| `Camera Tilt` | IMU Tilt data from the camera body | Degrees |
| `Camera Roll` | IMU Roll data from the camera body | Degrees |
| `Lens Model` | e.g., ARRI Signature Prime 21mm | String |

## Acquisition Details

* **Location:** MELS Studios (Montreal).
* **Camera System:** ARRI ALEXA Mini LF.
* **Lens:** ARRI Signature Primes (e.g., 21mm).
* **Recording Format:** Apple ProRes 4444 (Converted to Linear EXR for scientific analysis).
* **Resolution:** 3.2K (3200x1800).
* **Framerate:** 23.976 fps.

## Theoretical Background: The CINE-VBMLR Method

This dataset supports the **CINE-VBMLR** method, which adapts VBMLR estimation techniques to camera tracking.

1. **Origin (VBMLR):** The method derives from the *Variable Blur Model* described in gaze estimation research, where blur circles on the retina (or sensor) are used to infer depth and orientation [1].
2. **Adaptation to Cinema:** Unlike simple eyes or webcams, cinema lenses have complex optical characteristics. We utilize **Depth from Defocus** optimization techniques in the spatial domain [2][3] to model the Point Spread Function (PSF) and Circle of Confusion (CoC).
3. **Joint Estimation:** The goal is to perform joint estimation of camera blur and pose [4], using the precise metadata in this dataset as ground truth for supervised learning or validation.

## References

If you use this dataset, please cite the following foundational papers:

* **[1] CINE-VBMLR Foundation (In Preparation):**
* Hurtubise, D., et al. *Real-time eye gaze estimation on a computer screen*. (Manuscript in preparation).

* **[2] Spatial Domain Defocus:**
* Ziou, D., & Deschenes, F. (2001). Depth from defocus estimation in spatial domain. *Computer Vision and Image Understanding*, 81(2), 143-165.

* **[3] Optimal Parameters:**
* Mannan, F., & Langer, M. S. (2015, October). Optimal camera parameters for depth from defocus. In *2015 International Conference on 3D Vision* (pp. 326-334). IEEE.

* **[4] Joint Estimation:**
* LeBlanc, J. W., Thelen, B. J., & Hero, A. O. (2018). Joint camera blur and pose estimation from aliased data. *Journal of the Optical Society of America A*, 35(4), 639-651.

## License

This dataset is licensed under **CC-BY-ND-4.0** (Creative Commons Attribution-NonCommercial 4.0 International).

* **Attribution:** You must give appropriate credit to the authors and Studios B79/MELS.
* **Non-Commercial:** You may not use this material for commercial purposes without explicit permission.

* **CIN-VBMLR Code:** The source code in this repository (hosted on Github) is released under the **Apache License 2.0**.
* **CIN-VBMLR Dataset:** The associated dataset and models (hosted on Hugging Face) are released under the **CC-BY-NC-4.0** license.

For business cooperation or commercial licensing inquiries, please send an email to **David Hurtubise** at [[e-mail]](mailto:[e-mail]).

---
**Created by:** David Hurtubise, Djemel Ziou, Marie-Flavie Auclair Fortier (Université de Sherbrooke)