Perceptual Similarity

虽然人类几乎可以毫不费力地快速评估两个图像之间的感知相似性，但潜在的过程被认为是相当复杂的。尽管如此，当今使用最广泛的感知指标，如 PSNR 和 SSIM，都是简单、浅层的函数，无法解释人类感知的许多细微差别。最近，深度学习社区发现，在 ImageNet 分类上训练的 VGG 网络的特征作为图像合成的训练损失非常有用。但是这些所谓的“知觉损失”到底有多感性呢？哪些因素对他们的成功至关重要？为了回答这些问题，我们引入了一个新的人类感知相似性判断数据集。我们系统地评估不同架构和任务的深层特征，并将它们与经典指标进行比较。我们发现，在我们的数据集上，深度特征大大优于所有以前的指标。更令人惊讶的是，这一结果不仅限于 ImageNet 训练的 VGG 特征，而且适用于不同的深度架构和监督级别（监督、自我监督，甚至无监督）。我们的结果表明，感知相似性是一种在深度视觉表示中共享的新兴属性。While humans can almost effortlessly and rapidly assess the perceptual similarity between two images, the underlying processes are thought to be quite complex. Nonetheless, the most widely used perceptual metrics today, such as PSNR and SSIM, are simple, shallow functions that fail to account for many nuances of human perception. Recently, the deep learning community has found that features extracted from VGG networks trained on ImageNet classification serve as highly useful training losses for image synthesis. But just how perceptual are these so-called "perceptual losses"? What factors are critical to their success? To address these questions, we introduce a new dataset of human perceptual similarity judgments. We systematically evaluate deep features across different architectures and tasks, and compare them against classical metrics. We find that deep features outperform all prior metrics substantially on our dataset. More surprisingly, this finding is not limited to ImageNet-trained VGG features, but applies to diverse deep architectures and supervision levels (supervised, self-supervised, and even unsupervised). Our results demonstrate that perceptual similarity is an emergent property shared across deep visual representations.

搜集汇总

背景与挑战

背景概述

Perceptual Similarity数据集专注于人类感知相似性判断的研究，通过构建新的人类感知数据集来系统评估深度特征在感知相似性任务中的性能。研究发现，深度特征（涵盖多种架构和监督级别）在该数据集上显著超越了传统指标，揭示了感知相似性作为深度视觉表示中的一种新兴特性。