Experimental design and evaluation methodology for human-centric visual quality assessment

The problem of human-centric visual quality assessment (VQA) is extensively studied in this thesis. Our study includes three major topics: 1) design of a dataset for streaming video quality assessment, 2) development of a new and effective video quality assessment index, 3) exploration of a new methodology for human visual quality assessment based on the notion of just-noticeable-differences (JND). ❧ For the first topic, we present a high-definition VQA dataset that captures two typical video distortion types in streaming video services in Chapter 3. The VQA dataset, called MCL-V, contains 12 source video clips and 96 distorted video clips with subjective assessment scores. The source video clips are selected from a large pool of public-domain video sequences with representative and diversified contents. Both distortion types are perceptually adjusted to distinguishable distortion levels. An improved pairwise comparison method is adopted for subjective evaluation to save evaluation time. Several VQA algorithms are evaluated against the MCL-V dataset. ❧ For the second topic, we propose two objective assessment indices to predict subjective video quality in Chapter 4. They are a fusion-based video quality assessment (FVQA) index and an ensemble-learning video quality assessment (EVQA) index. The FVQA index first classifies video sequences according to their content complexity so as to reduce content diversity within each group. Then, it fuses several VQA methods to provide the final video quality score, where fusion coefficients are learned from training samples in the same group. Being motivated by ensemble learning, we propose another video quality assessment index to extend FVQA furthermore, and call it the EVQA index. The basic idea is to fuse multiple VQA methods with diverse and complementary merits so that the fused outcome outperforms that of any single method. The superior performance of EVQA is demonstrated by comparing it with other video quality assessment indices with several benchmarking video quality datasets. ❧ For the third topic, we propose a new human-centric methodology for visual quality assessment based on the JND notion in Chapter 5. JND is characterized by the detectable minimum amount of two visual stimuli, and has been used to enhance perceptual visual quality in the context of image/video compression. We first argue that the perceived quality of coded image/video is a stairwise function with several discrete jump points defined by JND. Then, we present a novel bisection method in performing the JND test on JPEG-coded images. Finally, we construct a JND dataset called MCL-JCI that contains 50 source images and analyze the relationship between the source content and the number of its distinguishable quality levels. The impact of JND-based quality assessment on image/video coding is also discussed.本论文针对以人为中心的视觉质量评估（Visual Quality Assessment, VQA）问题展开了广泛研究。本研究涵盖三大核心方向：1）面向流媒体视频质量评估的数据集构建，2）新型高效视频质量评估指标的研发，3）基于恰可察觉差（Just-Noticeable-Differences, JND）理念的人类视觉质量评估新方法探索。 针对第一个研究方向，本文第三章提出了一款高清视觉质量评估数据集，用于捕捉流媒体视频服务中两类典型的视频失真类型。该VQA数据集命名为MCL-V，包含12段原始视频片段与96段带主观评估分数的失真视频片段。原始视频片段选自海量公有领域视频序列库，内容兼具代表性与多样性。两类失真均经过感知调整，以设置可区分的失真等级。主观评估环节采用改进的成对比较法以节省评估耗时。多款视觉质量评估算法已在MCL-V数据集上完成测试与评估。 针对第二个研究方向，本文第四章提出了两款可预测主观视频质量的客观评估指标：基于融合的视频质量评估（Fusion-based Video Quality Assessment, FVQA）指标与基于集成学习的视频质量评估（Ensemble-Learning Video Quality Assessment, EVQA）指标。FVQA指标首先依据内容复杂度对视频序列进行分类，以降低每个分组内的内容多样性；随后融合多款VQA方法以生成最终的视频质量评分，其中融合系数由同分组内的训练样本学习得到。受集成学习理念启发，本文在FVQA的基础上进一步拓展，提出了另一款视频质量评估指标，即EVQA指标。其核心思路是融合多个具备多样互补优势的VQA方法，使融合后的结果优于任意单一方法的表现。通过与多款基准视频质量数据集上的其他视频质量评估指标进行对比，验证了EVQA的优异性能。 针对第三个研究方向，本文第五章提出了一种基于JND理念的新型以人为中心的视觉质量评估方法。JND指的是两种视觉刺激可被检测到的最小差异量，此前已被用于图像/视频压缩场景下的感知视觉质量提升。本文首先论证：编码图像/视频的感知质量是一个阶梯状函数，其存在若干由JND定义的离散跳变点；随后提出了一种针对JPEG编码图像的新型二分法JND测试方法；最后构建了一款名为MCL-JCI的JND数据集，包含50张原始图像，并分析了原始图像内容与其可区分质量等级数量之间的关联。本文还探讨了基于JND的质量评估对图像/视频编码的影响。