Panoramic multi-view image datasets for bullet-time video synthesis

Bullet-time video, a significant application of free-viewpoint video technology, offers an immersive viewing experience by allowing the camera viewpoint to switch along the camera array orbit. This technology is widely used in film, television, and broadcasting. However, current bullet-time videos are generated by perspective cameras with a limited field of view (FoV), which narrows the common FoV of the camera array. This limitation can reduce target selection interactivity and result in low immersion for the produced bullet-time video. To address these limitations, we propose using multiple 360\u00b0 panoramic cameras to construct a panoramic camera array for bullet-time video synthesis. The panoramic camera array enables complete coverage of the scene and introduces additional rotational degrees of freedom (DoF). Using the panoramic camera array, we propose a bullet-time video synthesis method based on the target axis to enhance the smoothness of the bullet-time video. This is achieved by maintaining the position, direction, and size of the target axis stable in the output bullet-time video frame. In addition, we propose a novel interactive high-dimensional bullet-time video synthesis system. The system incorporates a comprehensive pipeline that includes modules for panoramic video capture, panoramic camera array calibration, panoramic video codec, and bullet-time video synthesis. In our system, users can interactively select any target in the full scene and enjoy higher-DoF immersive bullet-time videos in real-time on lightweight devices like laptops. Qualitative and quantitative evaluations on simulated and real datasets demonstrate that our method outperforms state-of-the-art methods. In addition, our system was demonstrated at the 2023 Hangzhou Asian Games.