LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation
收藏资源简介：
3D content creation has achieved significant progress in terms of both quality and speed. Although current feed-forward models can produce 3D objects in seconds, their resolution is constrained by the intensive computation required during training. In this paper, we introduce Large Multi-view Gaussian Model (LGM), a novel framework designed to generate high-resolution 3D models from text prompts or single-view images. Our key insights are two-fold: (1) 3D Representation: We propose multi-view Gaussian features as an efficient yet powerful representation, which can then be fused together for differentiable rendering. (2) 3D Backbone: We present an asymmetric U-Net as a high-throughput backbone operating on multi-view images, which can be produced from text or single-view image input by leveraging multi-view diffusion models. Extensive experiments demonstrate the high fidelity and efficiency of our approach. Notably, we maintain the fast speed to generate 3D objects within 5 seconds while boosting the training resolution to 512, thereby achieving high-resolution 3D content generation.
三维内容创作在质量与速度层面均取得了显著进展。尽管当前的前馈模型（feed-forward models）可在数秒内生成三维物体，但训练过程中所需的密集计算限制了其分辨率上限。本文提出大型多视图高斯模型（Large Multi-view Gaussian Model, LGM），这是一种可根据文本提示或单视图图像生成高分辨率三维模型的全新框架。本文的核心思路分为两点：（1）三维表征：我们提出多视图高斯特征作为一种高效且强大的表征方式，可将其融合以实现可微渲染（differentiable rendering）；（2）三维骨干网络：我们提出将不对称U-Net作为处理多视图图像的高吞吐量骨干网络，所需的多视图图像可通过利用多视图扩散模型（multi-view diffusion models），从文本或单视图图像输入中生成。大量实验证明了本方法的高保真度与高效性。值得注意的是，本方法在将训练分辨率提升至512的同时，仍保持了5秒内生成三维物体的快速速度，从而实现了高分辨率三维内容生成。
