remyxai/OpenSpaces

---
license: apache-2.0
size_categories:
- 1K<n<10K
task_categories:
- visual-question-answering
- depth-estimation
pretty_name: OpenSpaces
dataset_info:
features:
- name: images
sequence: image
- name: messages
list:
- name: content
list:
- name: index
dtype: int64
- name: text
dtype: string
- name: type
dtype: string
- name: role
dtype: string
splits:
- name: train
num_bytes: 1139972963.3616219
num_examples: 9255
- name: test
num_bytes: 126745778.42237805
num_examples: 1029
download_size: 1260277343
dataset_size: 1266718741.784
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: test
path: data/test-*
tags:
- vqasynth
- remyx
---


# OpenSpaces

The OpenSpaces dataset is created using [VQASynth](https://github.com/remyxai/VQASynth) to synthesize spatialVQA data using images from the first 30K rows
of the localized narratives split of [the cauldron](https://huggingface.co/datasets/HuggingFaceM4/the_cauldron).

Compared to the [related dataset](remyxai/vqasynth_spacellava) used to train SpaceLLaVA,
the OpenSpaces emphasizes greater diversity in the image distribution instead of focusing on warehouse scenes.
The following chart shows the distribution of images over tags labeled by CLIP embedding similarity:

The OpenSpaces dataset also includes relatively more of the qualitative distance estimation queries than the quanlitative queries about the relative position of objects in a scene.
This histogram shows the distribution of estimated distances between objects in the image scenes

## Citation
```
@article{chen2024spatialvlm,
title = {SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities},
author = {Chen, Boyuan and Xu, Zhuo and Kirmani, Sean and Ichter, Brian and Driess, Danny and Florence, Pete and Sadigh, Dorsa and Guibas, Leonidas and Xia, Fei},
journal = {arXiv preprint arXiv:2401.12168},
year = {2024},
url = {https://arxiv.org/abs/2401.12168},
}

@misc{VQASynth,
author = {remyxai},
title = {VQASynth},
year = {2024},
note = {GitHub repository},
url = {https://github.com/remyxai/VQASynth/tree/main}
}
```