应用于室内空间具身智能机器人训练的真实感三维场景数据

具身智能机器人需要在多种场景下实现与环境的交互操作，面临着数据采集困难，实现成本高等技术瓶颈，本数据集可输出多样性室内空间3D场景数据，在视觉层面加强渲染效果，做到材质表现贴近真实，物理层面构建铰链化资产，模拟真实资产的交互动作，围绕OPEN USD数据框架，在仿真平台下可实现机器人高保真的训练，让具身智能更好的理解场景，加速机器人落地应用。本算法旨在处理三维模型，通过一系列步骤实现模型的分割、实例重组及格式转换，以生成新的实例模型，用于场景渲染和机器人训练等应用。

1.模型分割：本步骤接收任意初始三维模型作为输入，三维模型包括位置、尺寸、材质、顶点信息、法相信息、面片信息字段，运用拓扑连通性聚类算法将该组合模型拆分为多个面片组（face group），获取模型类型字段。此步骤有效提取模型的结构特征，有助于后续的实例重组。

2.模型实例重组：在此步骤中，对三维模型的位置、尺寸、材质、顶点信息、法相信息、面片信息字段进行分割，再利用Qwen-VL-Max和GroundingDino算法对分割后的部件进行组合，形成独立的模型实例，并获取其中的标签字段。标签字段能够使每个模型实例能够基于原模型的结构和信息进行识别和应用。

3.模型格式转换：本步骤将拆分获得的实例模型及其对应的材质信息转换为OpenUSD格式，并获取其中的碰撞体设置信息字段和动画约束信息字段，以使模型能够在场景中动起来。
数据字段中的物理属性、UV贴图路径、纹理细节、音效绑定、光源属性等字段为模型自身属性，并非所有模型都有相应字段。

Embodied intelligent robots need to perform interactive operations with the environment in various scenarios, and face technical bottlenecks such as difficult data collection and high implementation costs. This dataset provides diverse 3D indoor scene data, with enhanced rendering effects in terms of vision to make material representations close to real ones. On the physical level, hinged assets are constructed to simulate the interactive actions of real assets. Based on the OPEN USD data framework, high-fidelity training of robots can be realized in simulation platforms, enabling embodied intelligence to better understand scenes and accelerating the practical deployment and application of robots.

This algorithm is designed to process 3D models, and realizes model segmentation, instance recombination and format conversion through a series of steps to generate new instance models for applications such as scene rendering and robot training.

1. Model Segmentation: This step takes any initial 3D model as input. The 3D model includes fields such as position, size, material, vertex information, normal information and face information. Topological connectivity clustering algorithms are applied to split the combined model into multiple face groups and obtain the model type field. This step effectively extracts the structural features of the model, which is conducive to subsequent instance recombination.

2. Model Instance Recombination: In this step, the fields including position, size, material, vertex information, normal information and face information of the 3D model are first segmented. Then, the Qwen-VL-Max and GroundingDino algorithms are used to combine the segmented components to form independent model instances, and the label field is obtained. The label field enables each model instance to be identified and applied based on the structure and information of the original model.

3. Model Format Conversion: This step converts the split instance models and their corresponding material information into the OpenUSD format, and obtains the collision body setting information field and animation constraint information field therein, so that the models can move in the scene.

Fields such as physical properties, UV map paths, texture details, sound effect bindings and light source properties in the data fields are the inherent attributes of the models, and not all models have corresponding fields.

搜集汇总

背景与挑战

背景概述

该数据集是一个专为室内空间具身智能机器人训练设计的高真实感三维场景数据集合，包含552条以上的记录，以xlsx格式存储，涵盖了模型ID、位置、尺寸、材质、顶点信息、动画约束等详细字段。它通过先进的算法实现模型分割、实例重组和OpenUSD格式转换，提供材质表现贴近真实、物理交互模拟的铰链化资产，旨在解决机器人训练中数据采集困难和成本高的技术瓶颈，加速机器人在多样化室内环境中的高保真训练和实际应用。