LaMem

由于小规模和缺乏各种基准数据，估计视觉记忆性的进展受到限制。在这里，我们介绍了一种新颖的实验程序来客观地测量人类记忆，使我们能够构建 LaMem，这是迄今为止最大的带注释图像记忆数据集（包含来自不同来源的 60,000 张图像）。使用卷积神经网络 (CNN)，我们表明微调后的深度特征大大优于所有其他特征，达到 0.64 的秩相关性，接近人类一致性 (0.68)。对高级 CNN 层响应的分析表明，哪些对象和区域与可记忆性正相关和负相关，这使我们能够为每个图像创建可记忆性图，并提供执行图像可记忆性操作的具体方法。这项工作表明，人们现在可以稳健地估计来自许多不同类别的图像的可记忆性，将可记忆性和深度可记忆性特征定位为估计认知系统信息效用的主要候选者。

Progress in estimating visual memorability has been limited by small scale and the scarcity of diverse benchmark datasets. Herein, we introduce a novel experimental paradigm for objectively measuring human memory, which enables us to construct LaMem, the largest annotated image memorability dataset to date, comprising 60,000 images from diverse sources. Using convolutional neural networks (CNNs), we demonstrate that fine-tuned deep features outperform all other feature types significantly, achieving a rank correlation of 0.64, which is close to human consistency (0.68). Analysis of the responses from high-level CNN layers reveals which objects and regions are positively or negatively correlated with memorability, allowing us to generate memorability maps for each image and provide concrete methods for manipulating image memorability. This work demonstrates that memorability of images across a wide range of categories can now be robustly estimated, positioning memorability and deep memorability-related features as leading candidates for estimating the information utility of cognitive systems.

搜集汇总

背景与挑战

背景概述

LaMem是一个由麻省理工学院于2015年发布的大型图像记忆数据集，包含60,000张带注释的图像，主要用于研究视觉记忆性。该数据集通过卷积神经网络（CNN）分析图像的可记忆性，并生成可记忆性图，支持图像分类、风格迁移等计算机视觉任务，旨在推动图像记忆预测和认知系统研究。