SpatialLM-Dataset

# SpatialLM Dataset

<!-- markdownlint-disable first-line-h1 -->
<!-- markdownlint-disable html -->
<!-- markdownlint-disable no-duplicate-header -->

<div align="center">
<picture>
<source srcset="https://cdn-uploads.huggingface.co/production/uploads/63efbb1efc92a63ac81126d0/_dK14CT3do8rBG3QrHUjN.png" media="(prefers-color-scheme: dark)">
<img src="https://cdn-uploads.huggingface.co/production/uploads/63efbb1efc92a63ac81126d0/bAZyeIXOMVASHR6-xVlQU.png" width="60%" alt="SpatialLM""/>
</picture>
</div>
<hr style="margin-top: 0; margin-bottom: 8px;">
<div align="center" style="margin-top: 0; padding-top: 0; line-height: 1;">
<a href="https://manycore-research.github.io/SpatialLM" target="_blank" style="margin: 2px;"><img alt="Project"
src="https://img.shields.io/badge/🌐%20Website-SpatialLM-ffc107?color=42a5f5&logoColor=white" style="display: inline-block; vertical-align: middle;"/></a>
<a href="https://arxiv.org/abs/2506.07491" target="_blank" style="margin: 2px;"><img alt="arXiv"
src="https://img.shields.io/badge/arXiv-Techreport-b31b1b?logo=arxiv&logoColor=white" style="display: inline-block; vertical-align: middle;"/></a>
<a href="https://github.com/manycore-research/SpatialLM" target="_blank" style="margin: 2px;"><img alt="GitHub"
src="https://img.shields.io/badge/GitHub-SpatialLM-24292e?logo=github&logoColor=white" style="display: inline-block; vertical-align: middle;"/></a>
</div>
<div align="center" style="line-height: 1;">
<a href="https://huggingface.co/manycore-research/SpatialLM1.1-Qwen-0.5B" target="_blank" style="margin: 2px;"><img alt="Hugging Face"
src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-SpatialLM-ffc107?color=ffc107&logoColor=white" style="display: inline-block; vertical-align: middle;"/></a>
<a href="https://huggingface.co/datasets/manycore-research/SpatialLM-Dataset" target="_blank" style="margin: 2px;"><img alt="Dataset"
src="https://img.shields.io/badge/%F0%9F%A4%97%20Dataset-Dataset-ffc107?color=ffc107&logoColor=white" style="display: inline-block; vertical-align: middle;"/></a>
<a href="https://huggingface.co/datasets/manycore-research/SpatialLM-Testset" target="_blank" style="margin: 2px;"><img alt="Dataset"
src="https://img.shields.io/badge/%F0%9F%A4%97%20Dataset-Testset-ffc107?color=ffc107&logoColor=white" style="display: inline-block; vertical-align: middle;"/></a>
</div>

The SpatialLM dataset is a large-scale, high-quality synthetic dataset designed by professional 3D designers and used for real-world production. It contains point clouds from 12,328 diverse indoor scenes comprising 54,778 rooms, each paired with rich ground-truth 3D annotations. SpatialLM dataset provides an additional valuable resource for advancing research in indoor scene understanding, 3D perception, and related applications. For more details about the dataset construction, annotations, and benchmark tasks, please refer to the [paper](https://arxiv.org/abs/2506.07491).

<table style="table-layout: fixed;">
<tr>
<td style="text-align: center; vertical-align: middle; width: 25%"> <img src="https://cdn-uploads.huggingface.co/production/uploads/63efbb1efc92a63ac81126d0/YFQzBUC_sGufXqpGL6YhV.jpeg" alt="exmaple a" width="100%" style="display: block;"></td>
<td style="text-align: center; vertical-align: middle; width: 25%"> <img src="https://cdn-uploads.huggingface.co/production/uploads/63efbb1efc92a63ac81126d0/jRbPzBwhtDMWUwueodYax.jpeg" alt="exmaple c" width="100%" style="display: block;"></td>
<td style="text-align: center; vertical-align: middle; width: 25%"> <img src="https://cdn-uploads.huggingface.co/production/uploads/63efbb1efc92a63ac81126d0/DpNKunoD-2-1spx6cXDxa.jpeg" alt="exmaple b" width="100%" style="display: block;"></td>
<td style="text-align: center; vertical-align: middle; width: 25%"> <img src="https://cdn-uploads.huggingface.co/production/uploads/63efbb1efc92a63ac81126d0/o-JgD-oY0oK0yhryWUexv.jpeg" alt="exmaple d" width="100%" style="display: block;"></td>
</tr>
</tr>
</table>

## Dataset Structure

The dataset is organized into the following folder structure:

```bash
SpatialLM-Dataset/
├── pcd/ # Point cloud PLY files for rooms
│ └── .ply
├── layout/ # GT room layout
│ └── .txt
├── examples/ # 10 point cloud and layout examples
│ └── .ply
│ └── .txt
├── extract.sh # Extraction script
├── dataset_info.json # Dataset configuration file for training
├── spatiallm_train.json # SpatialLM conversations data for training
├── spatiallm_val.json # SpatialLM conversations data for validation
├── spatiallm_test.json # SpatialLM conversations data for testing
└── split.csv # Metadata CSV file
```

## Metadata

The dataset metadata is provided in the `split.csv` file with the following columns:

- **id**: Unique identifier for each sampled point cloud and layout following the naming convention `{scene_id}_{room_id}_{sample}` (e.g., `scene_001523_00_2`)
- **room_type**: The functional type of each room (e.g., bedroom, living room)
- **scene_id**: Unique identifier for multi-room apartment scenes
- **room_id**: Unique identifier for individual rooms within a scene
- **sample**: Point cloud sampling configuration for each room (4 types available):
- **0**: Most complete observations (8 panoramic views randomly sampled)
- **1**: Most sparse observations (8 perspective views randomly sampled)
- **2**: Less complete observations (16 perspective views randomly sampled)
- **3**: Less sparse observations (24 perspective views randomly sampled)
- **split**: Dataset partition assignment (`train`, `val`, `test`, `reserved`)

The dataset is divided into 11,328/500/500 scenes for train/val/test splits, and 199,286/500/500 sampled point clouds accordingly, where multiple point cloud samples of the same room are randomly selected for the val/test splits for simplicity.

## Data Extraction

Point clouds and layouts are compressed in zip files. To extract the files, run the following script:

```bash
cd SpatialLM-Dataset
chmod +x extract.sh
./extract.sh
```

## Conversation Format

The `spatiallm_train.json`, `spatiallm_val.json`, and `spatiallm_test.json` data follows the **SpatialLM format** with ShareGPT-style conversations:

```json
{
"conversations": [
{
"from": "human",
"value": "<point_cloud>Detect walls, doors, windows, boxes. The reference code is as followed: ..."
},
{
"from": "gpt",
"value": "<|layout_s|>wall_0=...<|layout_e|>"
}
],
"point_clouds": ["pcd/ID.ply"]
}
```

## Usage

Use the [SpatialLM code base](https://github.com/manycore-research/SpatialLM/tree/main) for reading the point cloud and the layout data.

```python
from spatiallm import Layout
from spatiallm.pcd import load_o3d_pcd

# Load Point Cloud
point_cloud = load_o3d_pcd(args.point_cloud)

# Load Layout
with open(args.layout, "r") as f:
layout_content = f.read()
layout = Layout(layout_content)
```

## Visualization

Use `rerun` to visualize the point cloud and the GT structured 3D layout output:

```bash
python visualize.py --point_cloud examples/scene_008456_00_3.ply --layout examples/scene_008456_00_3.txt --save scene_008456_00_3.rrd
rerun scene_008456_00_3.rrd
```

## SpatialGen dataset

For access to photorealistic RGB/Depth/Normal/Semantic/Instance panoramic renderings and camera trajectories used to generate the SpatialLM point clouds, please refer to the [SpatialGen project](https://manycore-research.github.io/SpatialGen) for more details.

## Citation

If you find this work useful, please consider citing:

```bibtex
@inproceedings{SpatialLM,
title = {SpatialLM: Training Large Language Models for Structured Indoor Modeling},
author = {Mao, Yongsen and Zhong, Junhao and Fang, Chuan and Zheng, Jia and Tang, Rui and Zhu, Hao and Tan, Ping and Zhou, Zihan},
booktitle = {Advances in Neural Information Processing Systems},
year = {2025}
}
```

# SpatialLM 数据集

<div align="center">
<picture>
<source srcset="https://cdn-uploads.huggingface.co/production/uploads/63efbb1efc92a63ac81126d0/_dK14CT3do8rBG3QrHUjN.png" media="(prefers-color-scheme: dark)">
<img src="https://cdn-uploads.huggingface.co/production/uploads/63efbb1efc92a63ac81126d0/bAZyeIXOMVASHR6-xVlQU.png" width="60%" alt="SpatialLM"/>
</picture>
</div>
<hr style="margin-top: 0; margin-bottom: 8px;">
<div align="center" style="margin-top: 0; padding-top: 0; line-height: 1;">
<a href="https://manycore-research.github.io/SpatialLM" target="_blank" style="margin: 2px;"><img alt="🌐 项目网站"
src="https://img.shields.io/badge/🌐%20Website-SpatialLM-ffc107?color=42a5f5&logoColor=white" style="display: inline-block; vertical-align: middle;"/></a>
<a href="https://arxiv.org/abs/2506.07491" target="_blank" style="margin: 2px;"><img alt="arXiv 技术报告"
src="https://img.shields.io/badge/arXiv-Techreport-b31b1b?logo=arxiv&logoColor=white" style="display: inline-block; vertical-align: middle;"/></a>
<a href="https://github.com/manycore-research/SpatialLM" target="_blank" style="margin: 2px;"><img alt="GitHub 仓库"
src="https://img.shields.io/badge/GitHub-SpatialLM-24292e?logo=github&logoColor=white" style="display: inline-block; vertical-align: middle;"/></a>
</div>
<div align="center" style="line-height: 1;">
<a href="https://huggingface.co/manycore-research/SpatialLM1.1-Qwen-0.5B" target="_blank" style="margin: 2px;"><img alt="🤗 Hugging Face 模型"
src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-SpatialLM-ffc107?color=ffc107&logoColor=white" style="display: inline-block; vertical-align: middle;"/></a>
<a href="https://huggingface.co/datasets/manycore-research/SpatialLM-Dataset" target="_blank" style="margin: 2px;"><img alt="🤗 Hugging Face 主数据集"
src="https://img.shields.io/badge/%F0%9F%A4%97%20Dataset-Dataset-ffc107?color=ffc107&logoColor=white" style="display: inline-block; vertical-align: middle;"/></a>
<a href="https://huggingface.co/datasets/manycore-research/SpatialLM-Testset" target="_blank" style="margin: 2px;"><img alt="🤗 Hugging Face 测试集"
src="https://img.shields.io/badge/%F0%9F%A4%97%20Dataset-Testset-ffc107?color=ffc107&logoColor=white" style="display: inline-block; vertical-align: middle;"/></a>
</div>

SpatialLM 数据集是一款由专业3D设计师打造的大规模高质量合成数据集，可服务于实际生产场景。该数据集包含来自12328个多样化室内场景的点云数据，涵盖54778个房间，每个场景均配有丰富的真实三维标注。SpatialLM 数据集为室内场景理解、三维感知及相关应用的研究提供了宝贵的支撑资源。有关数据集构建、标注规则及基准任务的更多细节，请参阅[论文](https://arxiv.org/abs/2506.07491)。

<table style="table-layout: fixed;">
<tr>
<td style="text-align: center; vertical-align: middle; width: 25%"><img src="https://cdn-uploads.huggingface.co/production/uploads/63efbb1efc92a63ac81126d0/YFQzBUC_sGufXqpGL6YhV.jpeg" alt="示例a" width="100%" style="display: block;"></td>
<td style="text-align: center; vertical-align: middle; width: 25%"><img src="https://cdn-uploads.huggingface.co/production/uploads/63efbb1efc92a63ac81126d0/jRbPzBwhtDMWUwueodYax.jpeg" alt="示例c" width="100%" style="display: block;"></td>
<td style="text-align: center; vertical-align: middle; width: 25%"><img src="https://cdn-uploads.huggingface.co/production/uploads/63efbb1efc92a63ac81126d0/DpNKunoD-2-1spx6cXDxa.jpeg" alt="示例b" width="100%" style="display: block;"></td>
<td style="text-align: center; vertical-align: middle; width: 25%"><img src="https://cdn-uploads.huggingface.co/production/uploads/63efbb1efc92a63ac81126d0/o-JgD-oY0oK0yhryWUexv.jpeg" alt="示例d" width="100%" style="display: block;"></td>
</tr>
</tr>
</table>

## 数据集组织架构

该数据集的目录结构如下：
bash
SpatialLM-Dataset/
├── pcd/ # 房间点云PLY(PLY)文件
│ └── .ply
├── layout/ # 真实房间布局标注
│ └── .txt
├── examples/ # 10组点云与布局示例
│ └── .ply
│ └── .txt
├── extract.sh # 数据解压脚本
├── dataset_info.json # 训练用数据集配置文件
├── spatiallm_train.json # 训练用SpatialLM对话数据
├── spatiallm_val.json # 验证用SpatialLM对话数据
├── spatiallm_test.json # 测试用SpatialLM对话数据
└── split.csv # 元数据CSV文件

## 元数据说明

数据集元数据存储于`split.csv`文件中，包含以下列：
- **id**：每个采样点云和布局的唯一标识符，命名格式为`{scene_id}_{room_id}_{sample}`，例如`scene_001523_00_2`
- **room_type**：单房间的功能类型，例如卧室、客厅
- **scene_id**：多房间公寓场景的唯一标识符
- **room_id**：单一场景内单个房间的唯一标识符
- **sample**：单房间的点云采样配置（共4种类型）：
- **0**：最完整观测模式（随机采样8张全景视图）
- **1**：最稀疏观测模式（随机采样8张透视视图）
- **2**：欠完整观测模式（随机采样16张透视视图）
- **3**：欠稀疏观测模式（随机采样24张透视视图）
- **split**：数据集分区标签，可选值为`train`、`val`、`test`、`reserved`

该数据集按照训练/验证/测试集划分为11328/500/500个场景，对应采样点云数分别为199286/500/500。为简化流程，验证与测试集随机选取同一房间的多个点云采样样本。

## 数据解压方法

点云与布局数据以ZIP格式压缩，如需解压文件，请运行以下脚本：
bash
cd SpatialLM-Dataset
chmod +x extract.sh
./extract.sh

## 对话数据格式

`spatiallm_train.json`、`spatiallm_val.json`与`spatiallm_test.json`遵循带有ShareGPT风格对话的**SpatialLM格式**，示例如下：
json
{
"conversations": [
{
"from": "human",
"value": "<point_cloud>检测墙体、门、窗与物体盒。参考代码如下：..."
},
{
"from": "gpt",
"value": "<|layout_s|>wall_0=...<|layout_e|>"
}
],
"point_clouds": ["pcd/ID.ply"]
}

## 使用方式

可使用[SpatialLM代码库](https://github.com/manycore-research/SpatialLM/tree/main)读取点云与布局数据，示例代码如下：
python
from spatiallm import Layout
from spatiallm.pcd import load_o3d_pcd

# 加载点云数据
point_cloud = load_o3d_pcd(args.point_cloud)

# 加载布局标注
with open(args.layout, "r") as f:
layout_content = f.read()
layout = Layout(layout_content)

## 数据可视化

可使用`rerun`工具可视化点云与真实结构化三维布局输出，命令如下：
bash
python visualize.py --point_cloud examples/scene_008456_00_3.ply --layout examples/scene_008456_00_3.txt --save scene_008456_00_3.rrd
rerun scene_008456_00_3.rrd

## SpatialGen 数据集

如需获取用于生成SpatialLM点云的照片级真实感RGB/深度/法向/语义/实例全景渲染图及相机轨迹，请参阅[SpatialGen项目](https://manycore-research.github.io/SpatialGen)获取更多细节。

## 引用格式

如您认为本工作对您的研究有所帮助，请引用以下文献：
bibtex
@inproceedings{SpatialLM,
title = {SpatialLM: Training Large Language Models for Structured Indoor Modeling},
author = {Mao, Yongsen and Zhong, Junhao and Fang, Chuan and Zheng, Jia and Tang, Rui and Zhu, Hao and Tan, Ping and Zhou, Zihan},
booktitle = {Advances in Neural Information Processing Systems},
year = {2025}
}

搜集汇总

背景与挑战

背景概述

SpatialLM-Dataset是一个大规模、高质量的合成数据集，由专业3D设计师设计，用于实际生产。它包含12,328个室内场景的点云，覆盖54,778个房间，每个房间都配有丰富的3D标注，旨在推动室内场景理解和3D感知等研究。数据集结构清晰，包括点云文件、布局文件、对话数据和元数据，适用于训练和评估相关模型。