The Met Dataset

The Met Dataset是一个专为大规模实例级艺术作品识别设计的数据集，由捷克技术大学电气工程学院等多个研究机构的专家合作创建。该数据集包含约418,605张图像，涵盖超过224,408个独特的艺术作品类别，这些作品来自世界各地，时间跨度从旧石器时代至今。数据集的构建过程严谨，通过从大都会艺术博物馆的公开收藏中筛选和标注图像，确保了数据的高质量和准确性。The Met Dataset不仅为艺术领域的计算机视觉研究提供了丰富的资源，还为开发和评估新的识别技术提供了平台，特别是在解决艺术作品的实例级识别问题上具有重要价值。

The Met Dataset is a dataset specifically designed for large-scale instance-level artwork recognition, co-created by experts from multiple research institutions including the Faculty of Electrical Engineering of the Czech Technical University. This dataset contains approximately 418,605 images, covering over 224,408 unique artwork categories, with works originating from across the globe and spanning from the Paleolithic era to the present day. The dataset is constructed with rigorous procedures, ensuring high data quality and accuracy by filtering and annotating images from the public collections of the Metropolitan Museum of Art. The Met Dataset not only provides abundant resources for computer vision research in the art field, but also offers a platform for developing and evaluating novel recognition technologies, and holds significant value especially in addressing instance-level recognition challenges for artworks.

搜集汇总

数据集介绍

构建方式

The Met Dataset is meticulously constructed by leveraging the open-access collection of The Metropolitan Museum of Art (The Met) in New York. The training set comprises approximately 400,000 images, encompassing over 224,000 unique classes, each corresponding to a distinct museum exhibit. These images are captured under controlled studio conditions, ensuring high-quality representations of the artworks. The testing set includes images taken by museum visitors, introducing a distribution shift that challenges the robustness of recognition models. Additionally, a set of distractor images, unrelated to The Met exhibits, is included to simulate out-of-distribution detection scenarios. This comprehensive dataset adheres to the evaluation protocol of the Google Landmarks Dataset (GLD), fostering research on domain-independent instance-level recognition approaches.

特点

The Met Dataset stands out for its large-scale instance-level recognition challenges, including high inter-class similarity, long-tail distribution, and numerous classes. The dataset's training images are meticulously curated under studio conditions, while the testing images, captured by museum visitors, introduce a significant distribution shift. This dual setup not only tests the model's ability to recognize artworks under varying conditions but also its capability to handle out-of-distribution queries. The dataset's meticulous annotation and verification processes ensure minimal noise, making it a reliable benchmark for instance-level recognition research. Furthermore, its public availability and adherence to the GLD evaluation protocol encourage comparative studies and advancements in domain-agnostic recognition techniques.

使用方法

The Met Dataset is designed for training and evaluating models on instance-level recognition tasks within the domain of artworks. Researchers can utilize the dataset to develop and test models that can accurately classify and retrieve artworks based on images taken under diverse conditions. The dataset's structure, with a large training set of studio-captured images and a testing set of visitor-taken images, allows for the assessment of model robustness and generalization capabilities. Additionally, the inclusion of distractor images enables the evaluation of out-of-distribution detection performance. Researchers can employ various machine learning techniques, including deep learning models, to extract features and classify images. The dataset's public availability and detailed documentation facilitate reproducibility and comparative analysis, making it an invaluable resource for advancing instance-level recognition research.

背景与挑战

背景概述

The Met Dataset, introduced in 2021 by researchers from Czech Technical University in Prague, Osaka University, Columbia University, and the University of Amsterdam, represents a pioneering effort in large-scale instance-level recognition within the domain of artworks. This dataset leverages the open-access collection of The Metropolitan Museum of Art (The Met) to form a comprehensive training set comprising approximately 400,000 images from over 224,000 unique exhibits. Each exhibit defines its own class, making it a unique resource for instance-level classification tasks. The dataset's creation addresses the critical need for large-scale, accurately labeled datasets in the field of instance-level recognition, particularly in the realm of artworks, which has historically attracted less attention compared to category-level recognition tasks. The Met Dataset not only facilitates research in artwork recognition but also serves as a benchmark for domain-independent approaches, encouraging advancements in instance-level recognition across various domains.

当前挑战

The Met Dataset presents several significant challenges. Firstly, the task of instance-level recognition in artworks is inherently difficult due to the large inter-class similarity, long-tail distribution of classes, and the sheer number of classes involved. The dataset also introduces a distribution shift between training images, which are taken under studio conditions, and testing images, which are captured by museum visitors, posing additional complexities. Additionally, the inclusion of out-of-distribution (OOD) images in the test set further complicates the recognition task, resembling an out-of-distribution detection problem. The creation process of the dataset itself involved meticulous annotation and verification to ensure the accuracy of labels, a tedious process given the scale and diversity of the collection. Despite these challenges, the Met Dataset stands as a robust benchmark, pushing the boundaries of instance-level recognition research and offering a fertile ground for future comparative studies.

常用场景

经典使用场景

The Met Dataset 在艺术品实例级识别领域具有经典应用场景，主要用于训练和测试模型在大型艺术品数据库中的实例级识别能力。该数据集通过利用大都会艺术博物馆的开放访问收藏，构建了一个包含约224,000个类别的训练集，每个类别对应一个博物馆展品。测试集则主要由博物馆游客拍摄的照片组成，这些照片在训练和测试之间引入了分布偏移，使得任务更具挑战性。

衍生相关工作

The Met Dataset 的发布催生了一系列相关研究工作，特别是在艺术品实例级识别和跨领域实例级识别方面。研究者们利用该数据集开发了多种深度学习模型，如结合自监督学习和监督对比学习的模型，显著提升了识别性能。此外，该数据集还激发了对长尾分布数据处理、分布外检测等问题的深入研究，推动了计算机视觉领域在这些方向上的进展。

数据集最近研究

最新研究方向

在艺术作品领域，The Met Dataset的出现为大规模实例级识别研究提供了新的契机。该数据集通过利用大都会艺术博物馆的开放访问收藏，构建了一个包含约224k个类别的训练集，每个类别对应一个博物馆展品，并在工作室条件下拍摄照片。测试集则主要由博物馆游客拍摄的照片组成，引入了训练与测试之间的分布偏移。此外，数据集还包括一组与Met展品无关的图像，使任务类似于分布外检测问题。该数据集的提出遵循了其他领域实例级识别数据集的范式，鼓励研究领域无关的方法。最近的研究方向包括结合自监督和监督对比学习来训练骨干网络，以及探索非参数分类方法，这些方法在处理大规模实例级识别任务中显示出潜力。

相关研究论文

• The Met Dataset: Instance-level Recognition for Artworks

  捷克技术大学电气工程学院 · 2022年