A data-driven approach to compressed video quality assessment using just noticeable difference

The problem of human-centric compressed video quality assessment (VQA) is studied in this research. Our studies include three major topics: 1) proposing a new methodology for compressed video quality measurement and assessment based on the just-noticeable-difference (JND) notion and building a large-scale dataset accordingly, 2) measuring the JND-based video quality using the satisfied user ratio (SUR) curve and designing an SUR prediction method with video quality degradation features and masking features, and 3) proposing a probabilistic JND-based video quality model to quantify the influence of subject variabilities as well as content variabilities and building a user model based on viewers' capability to address inter-group difference. ❧ For the first topic, the process of building a large-scale coded H.264/AVC video quality dataset, which measures human subjective experience based on the just-noticeable-difference (JND), is described in Chapter 3. The dataset, called the VideoSet, measures the first three JND points of 220 5-second sequences, each at four resolution (i.e., 1920 × 1080, 1280 × 720, 960 × 540 and 640 × 360). Each of these 880 video clips was encoded by the H.264/AVC standard with QP = 1, ..., 51. An improved bisection search algorithm was adopted to speed up subjective test without loss of robustness. We present the subjective test procedure, detection and removal of outlying measured data, and the properties of collected JND data. ❧ For the second topic, we propose a machine learning method to predict the satisfied-user-ratio (SUR) curves based on the VideoSet and then derive the JND points accordingly. Our method consists of the following steps. First, we partition a video clip into local spatial-temporal segments and evaluate the quality of each segment using the VMAF quality index. Next, we aggregate these local VMAF measures to derive a global index. Then, significant segments are selected based on the slope of quality scores between neighboring coded clips. After that, we incorporate the masking effect that reflects the unique characteristics of each video clip. Finally, we use the support vector regression (SVR) to minimize the L₂ distance of the SUR curves, and derive the JND point accordingly. ❧ For the third topic, we propose a JND-based VQA model that takes subject variabilities and content variabilities into account. The model parameters used to describe subject and content variabilities are jointly optimized by solving a maximum likelihood estimation (MLE) problem. We use subject inconsistency to filter out unreliable video quality scores. Moreover, we build a user model by utilizing user's capability to discern the quality difference. We study the SUR difference as it varies with user profile as well as content with variable level of difficulty. The proposed model aggregates quality ratings per user group to address inter-group difference.