V2AI/nuCraft

# nuCraft: High-res 3D Occupancy Dataset for Unified 3D Scene Understanding > https://github.com/V2AI/nuCraft_API ## Overview nuCraft is a cutting-edge, high-resolution 3D semantic occupancy dataset designed to meet the increasing demand for accurate and comprehensive 3D scene understanding in autonomous driving. Derived from the widely-utilized nuScenes dataset, nuCraft distinguishes itself by offering an **8x increase in resolution** over previous benchmarks, with grid resolution increased to **0.1 meters**. ## Context and Motivation The motivation behind nuCraft stems from the inherent limitations present in existing 3D semantic occupancy datasets, such as low resolution and inaccurate annotations. These limitations hinder the effective unification and understanding of complex urban scenes needed for innovative autonomous driving technologies. In response, nuCraft addresses these issues by ensuring more precise semantic annotations and higher resolution, thus facilitating more refined and accurate occupancy predictions. ## Key Features * **High Resolution**: nuCraft provides semantic occupancy grids at a resolution of **[1024 x 1024 x 80]** with a voxel size of 0.1m, compared to the **[512 x 512 x 40]** 0.2m grids in existing datasets. * **Enhanced Data Quality**: It mitigates the impact of noisy and sparse raw data, delivering more accurate and consistent annotations. * **Comprehensive Semantic Labels**: The dataset includes fine-grained semantic labels for both static elements (like buildings and roads) and dynamic elements (such as vehicles and pedestrians). ## Data Generation Pipeline The creation of nuCraft involves several meticulous steps: 1. **Pre-Processing**: This includes static/moving parts separation to enhance the clarity of static and dynamic scene components and continuous scenes grouping to form longer, cohesive sequences. 2. **LiDAR Sequence Aggregation**: Advanced pose estimation techniques like Kiss-ICP are employed to ensure better alignment of LiDAR frames, resulting in more reliable aggregated point clouds. 3. **Mesh Reconstruction**: Multi-level octree structures and semantic mesh reconstruction techniques ensure high-quality and noise-free occupancy grids. 4. **Post-Processing**: Further data refinement is done to remove outliers, reduce noise, and generate visibility masks for both LiDAR and camera sensors. ## Sources and Inspiration nuCraft is built upon the groundwork laid by the **nuScenes** dataset and integrates advanced methodologies elaborated in papers dealing with 3D occupancy prediction like **SemanticKITTI**, **OpenOccupancy** and **Occ3D**. The design and implementation of **nuCraft** are aimed at overcoming the specific deficiencies observed in these early datasets, inspired by academic research and practical requirements of modern autonomous driving systems. ## Conclusion nuCraft represents a significant step forward in 3D scene understanding, integrating high-resolution data with advanced processing techniques to provide a superior dataset for researchers and developers. Its release is poised to drive innovations and improve the accuracy of autonomous driving technologies. --- license: cc-by-nc-nd-4.0 ---# nuCraft：面向统一三维场景理解的高分辨率三维语义占据（3D Occupancy）数据集 > https://github.com/V2AI/nuCraft_API ## 概述 nuCraft是一款前沿的高分辨率三维语义占据（3D semantic occupancy）数据集，旨在满足自动驾驶领域对精准且全面的三维场景理解日益增长的需求。该数据集源自广泛应用的nuScenes数据集，相较此前的基准数据集分辨率提升**8倍**，栅格分辨率提升至**0.1米**。 ## 背景与研发动机 nuCraft的研发动机源于现有三维语义占据数据集存在的固有缺陷，例如分辨率偏低、标注不够精准等。这些缺陷阻碍了创新自动驾驶技术所需的复杂城市场景统一理解与有效分析。为此，nuCraft通过提供更精准的语义标注与更高的分辨率来解决上述问题，从而助力实现更精细、准确的占据预测任务。 ## 核心特性 * **高分辨率**：nuCraft的语义占据栅格分辨率为**[1024 × 1024 × 80]**，体素尺寸为0.1米；而现有数据集的栅格分辨率仅为**[512 × 512 × 40]**，体素尺寸为0.2米。 * **数据质量优化**：该数据集缓解了原始数据噪声与稀疏性带来的负面影响，可生成更精准且一致的标注结果。 * **语义标签全面精细**：数据集涵盖静态元素（如建筑物、道路）与动态元素（如车辆、行人）的细粒度语义标注。 ## 数据生成流程 nuCraft的构建包含多个精细步骤： 1. **预处理**：包括动静分量分离，以增强场景静态与动态组件的清晰度；以及连续场景分组，以生成更长的连贯序列。 2. **激光雷达（LiDAR）序列聚合**：采用Kiss-ICP等先进位姿估计技术，确保激光雷达帧的精准对齐，从而得到更可靠的聚合点云。 3. **网格重建**：通过多层八叉树结构与语义网格重建技术，生成高质量、无噪声的占据栅格。 4. **后处理**：进一步优化数据，剔除异常值、降低噪声，并为激光雷达与相机传感器生成可见性掩码。 ## 数据来源与研发灵感 nuCraft以广泛应用的**nuScenes**数据集为基础，整合了3D占据预测相关论文中提出的先进方法，例如**SemanticKITTI**、**OpenOccupancy**与**Occ3D**。该数据集的设计与实现旨在克服上述早期数据集存在的特定缺陷，其研发灵感来源于学术研究与现代自动驾驶系统的实际需求。 ## 总结 nuCraft是三维场景理解领域的重要突破，通过整合高分辨率数据与先进处理技术，为研究者与开发者提供了一款优质数据集。该数据集的发布有望推动自动驾驶技术的创新，并提升其精准度。 ---授权协议：CC BY-NC-ND 4.0---