"Fisheye-Supervised Pinhole Video Stitching Dataset"

"Although video stitching has achieved great progress, existing methods can still perform poorly in real-life scenarios by presenting distortions such as ghosting, misalignment, missing parts, distortions, and shaking. The main reason hindering the progression of video stitching research is the lack of ground-truth to evaluate how constituent views are warped and aligned as a unified whole. On the one hand, most existing methods evaluate the accuracy of video stitching by mainly considering the alignment accuracy within overlapping areas, ignoring the fact that the overlapping area between two adjacent views only occupies a small proportion of the entire scene, while many stitching distortions can happen outside of overlapping regions. On the other hand, existing real datasets for image or video stitching do not provide ground-truths for the entire scene covered by candidate cameras. In this work, we introduce a new dataset supporting fully-supervised model training and full-reference quality evaluation for video stitching. While it is straight-forward to generate ground-truth images manually, this approach often fails to ensure distortion-free results, and is extremely costly and cannot guarantee temporal smoothness in the case of videos. Instead, we design a tri-camera system, with a fisheye camera installed in the middle of two pinhole cameras, so that the fisheye camera can provide video stitching ground-truth for the entire scene covered by the two pinhole cameras. With this special tri-camera system, we collect a dataset called Fisheye-Supervised Pinhole Video Stitching (FSPVS) dataset containing 151,152 frames covering diverse scenes, including but not limited to crowd, traffic flow, indoor, outdoor, eye-level view, bird's eye view scenes. With the proposed dataset, we further propose solutions to how existing representative stitching methods can be adapted to be evaluated in a full-reference manner or to be trained in a fully-supervised manner. We show that our dataset providing fisheye-based ground-truth can effectively improve performance of existing deep learning approaches by finetuning them in a fully-supervised manner."