SAGE-3D_Collision_Mesh

# SAGE-3D Collision Mesh: Physics-Enabled Collision Bodies for 3D Gaussian Scenes [Paper](https://huggingface.co/papers/2510.21307) | [Project Page](https://sage-3d.github.io) | [Code](https://github.com/Galery23/SAGE-3D_Official) High-precision collision geometry dataset extracted from 1,000 indoor Mesh scenes, enabling physically accurate navigation and interaction in virtual environments. <div align="center"> <img src="./src/SAGE-3D_Collision_Mesh.png" alt="SAGE-3D Collision Mesh"> <p><em>Collision Mesh of InteriorGS data captured on Issac Sim 5.0.</em></p> </div> --- ## 📢 News - **2025-12-15**: Released SAGE-3D Collision Mesh dataset with collision bodies for 1000 InteriorGS scenes. --- ## 📋 Overview 3D Gaussian Splatting (3DGS) has emerged as a powerful representation for photorealistic scene rendering. However, a critical limitation prevents its use in interactive applications: **3DGS lacks physical collision geometry**, making it impossible for embodied agents to interact realistically with the environment. Without collision detection, agents can pass through walls, float through furniture, or exhibit other physically implausible behaviors. To address this challenge, we present **SAGE-3D Collision Mesh**, a comprehensive dataset of **physics-enabled collision bodies** extracted from the InteriorGS dataset. This dataset provides: - **554,000+ collision bodies** corresponding to object instances across 1000 indoor scenes - **Convex hull decomposition** for efficient and accurate collision detection - **USD format** compatible with modern physics simulation platforms (Isaac Sim, Omniverse, etc.) - **Tight integration** with InteriorGS 3DGS scenes for seamless visual-physical alignment Each collision body is carefully decomposed into convex hulls, ensuring computational efficiency while maintaining geometric fidelity. This enables: **Real-time collision detection** - Optimized convex hulls enable fast physics simulation **Embodied AI research** - Support for manipulation, navigation, and interaction tasks **Sim-to-real transfer** - Realistic physics simulation reduces the sim-to-real gap ### Key Features | Feature | Description | |---------|-------------| | **Scenes** | 1000 indoor environments (homes, offices, museums, restaurants, etc.) | | **Objects** | 554,000+ collision body instances | | **Format** | USD (Universal Scene Description) | | **Compatibility** | Isaac Sim 5.0+, NVIDIA Omniverse | --- ## 🗂️ Dataset Structure ``` Collision_Mesh/ ├── 839873/ # Scene-specific collision data │ └── 839873_collision.usd # Collision mesh in USD format ├── 839874/ │ └── 839874_collision.usd ├── 839875/ │ └── 839875_collision.usd └── ... ``` --- ## 💡 Sample Usage This dataset provides physics-enabled collision bodies in USD format, which are crucial for enabling physically accurate navigation and interaction in virtual environments. These meshes are primarily used to integrate with 3D Gaussian Splatting (3DGS) scenes (e.g., from [spatialverse/SAGE-3D_InteriorGS_usdz](https://huggingface.co/datasets/spatialverse/SAGE-3D_InteriorGS_usdz)) to create complete USDA scene files compatible with physics simulation platforms like NVIDIA Isaac Sim. ### 1. Download the SAGE-3D Collision Mesh Dataset First, clone this dataset repository using `git lfs`: ```bash git lfs install git clone https://huggingface.co/datasets/spatialverse/SAGE-3D_Collision_Mesh ``` ### 2. Prepare Isaac Sim Environment The full SAGE-3D project, including the use of these collision meshes, relies on NVIDIA Isaac Sim. Please refer to the [official SAGE-3D GitHub repository](https://github.com/Galery23/SAGE-3D_Official#isaac-sim-setup) for detailed instructions on setting up Isaac Sim (version 5.0+). ### 3. Build USDA Scene Files with Collision Data Once you have downloaded this collision mesh dataset and obtained the USDZ scene files (e.g., from [spatialverse/SAGE-3D_InteriorGS_usdz](https://huggingface.co/datasets/spatialverse/SAGE-3D_InteriorGS_usdz)), you can integrate the collision bodies into USDA scene files using the `sage3d_usda_builder.py` script provided in the [SAGE-3D Official repository](https://github.com/Galery23/SAGE-3D_Official). ```bash # Example command to build USDA Scene Files python Code/benchmark/scene_data/sage3d_usda_builder.py \ --usdz-dir /path/to/usdz \ --out-dir /path/to/output/usda \ --template Data/template.usda \ --usdz-placeholder "@usdz_root[gauss.usda]@" \ --collision-placeholder "@collision_root@" \ --usdz-path-template "/path/to/usdz/{scene_id}.usdz[gauss.usda]" \ --collision-path-template "/path/to/Collision_Mesh/{scene_id}/{scene_id}_collision.usd" \ --overwrite ``` **Parameters:** - `--usdz-dir`: Directory containing USDZ scene files (e.g., from [spatialverse/SAGE-3D_InteriorGS_usdz](https://huggingface.co/datasets/spatialverse/SAGE-3D_InteriorGS_usdz)) - `--out-dir`: Output directory for USDA files - `--template`: Path to USDA template file (found in the [SAGE-3D_Official GitHub repository](https://github.com/Galery23/SAGE-3D_Official)) - `--usdz-placeholder`: Placeholder string in template for USDZ reference (default: `@usdz_root[gauss.usda]@`) - `--collision-placeholder`: Placeholder string in template for collision payload (default: `@collision_root@`) - `--usdz-path-template`: Template for USDZ reference path (use `{scene_id}` as placeholder) - `--collision-path-template`: Template for collision payload path (use `{scene_id}` as placeholder to point to files in this dataset) - `--overwrite`: Overwrite existing USDA files --- ## 🔗 Related Datasets This dataset is part of the **SAGE-3D** project: 1. **InteriorGS**: Source 3DGS scenes with semantic annotations → [spatialverse/InteriorGS](https://huggingface.co/datasets/spatialverse/InteriorGS) 2. **SAGE-3D InteriorGS USDZ**: USDZ format scenes for Isaac Sim → [spatialverse/SAGE-3D_InteriorGS_usdz](https://huggingface.co/datasets/spatialverse/SAGE-3D_InteriorGS_usdz) 3. **SAGE-3D Collision Mesh** (This dataset): Physics-enabled collision bodies → [spatialverse/SAGE-3D_Collision_Mesh](https://huggingface.co/datasets/spatialverse/SAGE-3D_Collision_Mesh) 4. **SAGE-3D VLN Data**: Navigation trajectories and instructions → [spatialverse/SAGE-3D_VLN_Data](https://huggingface.co/datasets/spatialverse/SAGE-3D_VLN_Data) --- ## 📄 License This dataset is released under CC-BY-NC-4.0. --- ## 📜 Citation If you use SAGE-3D Collision Mesh in your research, please cite our paper: ```bibtex @inproceedings{miao2026towards, title = {Towards Physically Executable {3D} Gaussian for Embodied Navigation}, author = {Miao, Bingchen and Wei, Rong and Ge, Zhiqi and Sun, Xiaoquan and Gao, Shiqi and Zhu, Jingzhe and Wang, Renhan and Tang, Siliang and Xiao, Jun and Tang, Rui and Li, Juncheng}, booktitle = {The Fourteenth International Conference on Learning Representations}, year = {2026}, url = {https://openreview.net/forum?id=HB6KvsqcAn} } ``` Please also cite the InteriorGS dataset: ```bibtex @misc{InteriorGS2025, title = {InteriorGS: A 3D Gaussian Splatting Dataset of Semantically Labeled Indoor Scenes}, author = {SpatialVerse Research Team, Manycore Tech Inc.}, year = {2025}, howpublished = {\url{https://huggingface.co/datasets/spatialverse/InteriorGS}} } ``` --- <div align="center"> <strong>SAGE-3D: Semantically and Physically-Aligned Gaussian Environments for 3D Navigation</strong> </div>