Dynamic street scene 3D reconstruction with self-supervised Gaussian Splatting using spatiotemporal deformation field

Accurate 3D reconstruction of dynamic street scenes is crucial for autonomous driving, yet existing methods either require costly 3D annotation boxes or fail to capture fine object motion. To overcome these limitations, we propose SSTD-GS, a self-supervised Gaussian Splatting framework for annotation-free dynamic scene reconstruction and novel view synthesis. Specifically, we design a spatiotemporal deformation field to model the detailed motion of dynamic objects, and develop an uncertainty dynamic mask guided self-supervised strategy to enable joint optimization of dynamic and static scene components. To further improve the quality of novel view synthesis, with the help of the powerful priors of the depth completion model and diffusion model, we design a confidence dense depth prior module and a diffusion model virtual view prior module to provide additional geometric and appearance constraints. Moreover, a geometry aware Gaussian adaptive control mechanism is employed to suppress inaccurate densification in 3DGS caused by rendering errors. Experimental results on the Waymo and KITTI datasets show that SSTD-GS outperforms existing NeRF and 3DGS-based methods in 4D scene reconstruction and novel view synthesis. In the novel view synthesis task, the PSNR reaches 29.83 and 28.59 dB, respectively, which are 1.72 and 1.36 dB higher than the suboptimal PVG.