Feed-Forward Novel View Synthesis from Sparse Observations

Novel view synthesis (NVS) aims to create images of a 3D scene from angles not originally captured, using images taken from different viewpoints as input. To make NVS more practical for real-world use, this research focuses on two key challenges: ensuring the system can handle new scenes without retraining and working with limited input images (1 to 7 views). This research contains four closely related projects: Sem2NeRF, MatchNeRF, MVSplat, and MVSplat360. The core idea is that combining knowledge of 3D geometry and patterns learned from large-scale data improves results. Extensive experiments verified the superiority of this research.