JoyAI-Image-OpenSpatial

# JoyAI-Image-OpenSpatial Spatial understanding dataset built on [OpenSpatial](https://github.com/VINHYU/OpenSpatial), used in [JoyAI-Image](https://github.com/jd-opensource/JoyAI-Image). The full dataset contains about **~3M** multi-turn visual-spatial QA samples across **7 open-source datasets** and web data. The open-source datasets contain ARKitScenes, ScanNet, ScanNet++, HyperSim, Matterport3D, WildRGB-D, and Ego-Exo4D. Tasks cover a wide range of spatial understanding capabilities including 3D object grounding, depth ordering, spatial relation reasoning, distance estimation, and more. We have released **~2.3M** QA samples constructed from the open-source datasets. The remaining web data will be open-sourced in a future release. ## Quick Start ```python from datasets import load_dataset ds = load_dataset("jdopensource/JoyAI-Image-OpenSpatial", split="train", streaming=True) for sample in ds: print(sample["conversations"]) break ``` ## Data Format Each parquet file contains the following columns: | Column | Type | Description | |---|---|---| | `conversations` | `list[{from, value}]` | Multi-turn conversation pairs (`human` / `gpt`). The human turn provides camera parameters and a spatial reasoning question; the gpt turn provides structured spatial annotations (e.g., 3D bounding boxes, depth ordering, spatial relations). | | `id` | `string` | Unique sample identifier | | `data_source` | `string` | Source dataset (e.g., `arkitscenes`, `scannet`, `scannetpp`, `hypersim`, `matterport3d`, `wildrgbd`, `Ego-Exo4D`) | | `images` | `list[{bytes, path}]` | Embedded image data (PNG bytes) | | `type` | `string` | Data type label | | `meta_info` | `string` | JSON string with image dimensions (`width`, `height`, `resized_width`, `resized_height`) | ## TODO - [ ] Release 3D lifting data