TLL (Totally-Looks-Like)

人类对图像相似度的感知判断依赖于丰富的内部表征，从低级特征到高级概念、场景属性甚至文化关联。试图解释感知相似性的现有方法和数据集使用的刺激可能无法涵盖影响人类相似性判断的全部因素，即使是那些旨在实现这一目标的因素。我们在一个流行的娱乐网站之后引入了一个名为 Totally-Looks-Like (TLL) 的新数据集，其中包含由人类配对为视觉相似的图像。该数据集包含来自野外的 6016 对图像，揭示了人类采用的丰富多样的标准。我们进行实验以尝试通过从最先进的深度卷积神经网络中提取的特征来重现配对，并进行额外的人体实验以验证所收集数据的一致性。尽管我们创造了条件来人为地使匹配任务变得越来越容易，但我们表明机器提取的表示在再现人类选择的匹配方面表现非常差。我们讨论和分析这些结果，提出改进学习图像表示的未来方向。

Human perception of image similarity relies on rich internal representations, ranging from low-level features to high-level concepts, scene attributes, and even cultural associations. Existing methods and datasets, as well as the stimuli they employ, that attempt to explain perceptual similarity may fail to cover all factors influencing human similarity judgments, even those specifically designed for this goal. Following a popular entertainment website, we introduce a new dataset named Totally-Looks-Like (TLL), which consists of images paired as visually similar by human raters. This dataset contains 6016 pairs of in-the-wild images, revealing the diverse and rich criteria humans adopt when making similarity judgments. We conduct experiments to reproduce these pairs using features extracted from state-of-the-art deep convolutional neural networks, alongside additional human experiments to verify the consistency of the collected data. Even though we create conditions to artificially make the matching task increasingly easier, we demonstrate that machine-extracted representations perform very poorly in reproducing the human-selected matches. We discuss and analyze these results, and propose future directions for improving learned image representations.