Subjective Test Dataset and Meta-data-based Models for 360° Streaming Video Quality

During the last years, the number of 360° videos available for streaming has rapidly increased, leading to the<br> need for 360° streaming video quality assessment. In this paper, we report and publish results of three subjective 360° video<br> quality tests, with conditions used to reflect real-world bitrates and resolutions including 4K, 6K and 8K, resulting in 64 stimuli<br> each for the first two tests and 63 for the third. As playout device we used the HTC Vive for the first and HTC Vive Pro<br> for the remaining two tests. Video-quality ratings were collected using the 5-point Absolute Category Rating scale. The 360°<br> dataset provided with the paper contains the links of the used source videos, the raw subjective scores, video-related meta-data,<br> head rotation data and Simulator Sickness Questionnaire results per stimulus and per subject to enable reproducibility of the<br> provided results. Moreover, we use our dataset to compare the performance of state-of-the-art full-reference quality metrics such<br> as VMAF, PSNR, SSIM, ADM2, WS-PSNR and WS-SSIM. Out of all metrics, VMAF was found to show the highest correlation<br> with the subjective scores. Further, we evaluated a center-cropped version of VMAF ("VMAF-cc") that showed to provide a similar<br> performance as the full VMAF. In addition to the dataset and the objective metric evaluation, we propose two new video-quality<br> prediction models, a bitstream meta-data-based model and a hybrid no-reference model using bitrate, resolution and pixel<br> information of the video as input. The new lightweight models provide similar performance as the full-reference models while<br> enabling fast calculations.