Image and video enhancement through motion based interpolation and nonlocal-means denoising techniques

Unrestricted In this research, we investigate advanced image and video enhancement techniques based on motion based interpolation and nonlocal-means (NL-means) denoising. The dissertation consists of three main results. Two video processing applications; namely, video error concealment (EC) and frame rate up-conversion (FRUC), based on motion analysis have been examined. Then, an improved NL-means algorithm has been proposed for image denoising. They are detailed below.; In the first part of this study, low-complexity error concealment techniques are studied. The boundary matching algorithm (BMA) is an attractive choice for video error concealment due to its low complexity. Here, we examine a variant of BMA called the outer boundary matching algorithm (OBMA). Although BMA and OBMA are similar in their design principle, it is empirically observed that OBMA outperforms BMA by a significant margin (typically, 0.5dB or higher) while maintaining the same level of complexity. We first explain the superior performance of OBMA, and conclude that OBMA provides an excellent tradeoff between the complexity and the quality of concealed video for a wide range of test video sequences and error conditions. In addition, we present two extensions of OBMA, i.e. refined local search and multiple boundary layers. These extensions can be employed to enhance the performance of OBMA at slightly higher computational complexity. Finally, the effect of the flexible macroblock ordering (FMO) on the performance of several EC algorithms is examined.; In the second part of this work, two challenging situations for video frame rate up-conversion (FRUC) are identified and analyzed; namely, when the input video clip has abrupt illumination change and a low frame rate. Then, a low-complexity processing technique and robust FRUC algorithm are proposed to address these two issues. The proposed algorithm utilizes a translational motion vector model of the first- and the second-orders and detects the continuity of these motion vectors. Additionally, a spatial smoothness criterion is employed to improve perceptual quality of interpolated frames. The superior performance of the proposed algorithm has been extensively tested and representative examples are given in this work.; In the third part of this research, an adaptive image denoising technique based on the NL-means algorithm is proposed. The proposed method employs the singular value decomposition (SVD) method and the K-means clustering (K-means) technique to achieve robust block classification in noisy images. Then, a local window is adaptively adjusted to match the local property of a block. Finally, a rotated block matching algorithm based on the alignment of dominant orientation is adopted for similarity matching. In addition, the noise level can be accurately estimated using block classification and the Laplacian operator. Experimental results are given to demonstrate the superior denoising performance of the proposed adaptive NL-means (ANL-means) denoising technique over various image denoising benchmarks in term of both PSNR and perceptual quality comparison, where images corrupted by additive white Gaussian noise (AWGN) and Rician noise are both tested.

本研究针对基于运动插值与非局部均值（nonlocal-means, NL-means）去噪的先进图像与视频增强技术展开探究。本学位论文包含三项核心研究成果：基于运动分析的两类视频处理应用——视频错误隐藏（error concealment, EC）与帧率上转换（frame rate up-conversion, FRUC），以及针对图像去噪的改进型NL-means算法。下文将对其逐一详述。

研究第一部分聚焦低复杂度错误隐藏技术。边界匹配算法（boundary matching algorithm, BMA）因复杂度较低，是视频错误隐藏的常用方案。本文针对其变体——外边界匹配算法（outer boundary matching algorithm, OBMA）展开研究。尽管BMA与OBMA的设计原理相近，但实验结果表明，OBMA在保持相同复杂度的前提下，性能显著优于BMA（通常可提升0.5dB及以上）。本文首先阐释OBMA的优异性能来源，并得出结论：在大量测试视频序列与错误场景下，OBMA在复杂度与隐藏视频质量之间实现了极佳的平衡。此外，本文提出OBMA的两类扩展方案，即精细化局部搜索与多边界层策略，可在小幅提升计算复杂度的前提下进一步优化OBMA的性能。最后，本文探究了灵活宏块排序（flexible macroblock ordering, FMO）对多款错误隐藏算法性能的影响。

研究第二部分识别并分析了视频帧率上转换（FRUC）面临的两类挑战性场景：输入视频存在光照突变，以及输入视频帧率偏低。针对这两类问题，本文提出一种低复杂度处理方案与鲁棒性FRUC算法。所提算法利用一阶与二阶平移运动向量模型，并检测这些运动向量的连续性；同时引入空间平滑准则，以提升插值帧的主观视觉质量。本文通过大量实验验证了所提算法的优异性能，并给出了典型测试案例。

研究第三部分提出一种基于NL-means算法的自适应图像去噪技术。所提方法结合奇异值分解（singular value decomposition, SVD）与K均值聚类（K-means clustering, K-means）技术，实现噪声图像中块区域的鲁棒分类；随后自适应调整局部窗口，以匹配块区域的局部特性；最后采用基于主方向对齐的旋转块匹配算法完成相似度匹配。此外，本文可通过块分类与拉普拉斯算子（Laplacian operator）精确估计噪声水平。实验结果表明，针对加性高斯白噪声（additive white Gaussian noise, AWGN）与莱斯噪声（Rician noise）污染的图像，在所提自适应非局部均值（adaptive NL-means, ANL-means）去噪技术上，无论是峰值信噪比（PSNR）还是主观视觉质量对比，其去噪性能均优于多款现有图像去噪基准算法。