r3d-bench

# R3D-Bench This is the official R3D-Bench dataset from [R3D: Quantitative 3D Spatial Reasoning for Egocentric Wearables](https://arxiv.org/abs/2607.02921). R3D-Bench is a benchmark for quantitative 3D spatial reasoning from natural egocentric RGB-D video. 3,033 question-answer annotations over 57 Aria Digital Twin (ADT) sequences. This is an **evaluation-only** benchmark — there is no training split; all data is in the `test` split. Note that project Aria video data must be downloaded separately. See full setup instructions in our [github repository](https://github.com/facebookresearch/r3d). ## Configs - **qa_annotations** (default): one row per question. Fields include `question_text`, `gt_answer`, `question_type`, `sequence_id`, spatial/temporal descriptions, and a nested `referenced_objects` list. This is the benchmark you evaluate against. - **segmentations**: per-frame SAM3 masks (RLE) tracking relevant objects in the scene. Used in R3D-Bench evaluations for most methodologies. These are optional, but useful. - **meshes**: per-object SAM3D reconstructed meshes with the GLB geometry embedded (`glb` bytes), used for evaluating the R3D methodology on R3D-Bench. They are available for use with other methodologies. - **gt_bboxes**: optional. Ground-truth oriented-box trajectories from ADT that the reference answers were derived from. These are mainly for debugging. Evaluations do not use them. ## Usage ```python from datasets import load_dataset qa = load_dataset("facebook/r3d-bench", "qa_annotations", split="test") ``` ## Source video The RGB-D frames are not redistributed here. Obtain the ADT sequences from https://www.projectaria.com/datasets/adt/ and extract frames as described in the R3D code repo. ## Citation When using our work, please cite as follows: ```bibtex @misc{horton2026r3dquantitative3dspatial, title={R3D: Quantitative 3D Spatial Reasoning for Egocentric Wearables}, author={Maxwell Horton and Wei Lu and Quan Tran and Yury Astashonok and Kirmani Ahmed and Babak Damavandi and Anuj Kumar and Xiao Zhang and Seungwhan Moon}, year={2026}, eprint={2607.02921}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2607.02921}, } ``` ## License Derived from the Aria Digital Twin (ADT) dataset; use is subject to the ADT license/terms.