Recipe1M+

在本文中，我们介绍了Recipe1M，这是一个新的大型结构化语料库，包含100万多个烹饪食谱和1300万的食物图像。作为最大的公开配方数据集合，Recipe1M具有在对齐的多模式数据上训练高容量模型的能力。使用这些数据，我们训练神经网络以学习配方和图像的联合嵌入，从而在图像配方检索任务中产生令人印象深刻的结果。此外，我们证明了通过添加高级分类目标进行正则化既可以提高检索性能，使其与人类的检索性能相媲美，又可以实现语义向量算法。我们假设这些嵌入将为进一步探索Recipe1M数据集以及一般的食物和烹饪提供基础。代码、数据和模型是公开的。

In this paper, we introduce Recipe1M, a novel large-scale structured corpus containing over one million culinary recipes and 13 million food images. As the largest publicly available recipe dataset collection, Recipe1M enables training high-capacity models on aligned multimodal data. Using this dataset, we train neural networks to learn joint embeddings for recipes and images, yielding impressive performance on the image-recipe retrieval task. Furthermore, we demonstrate that incorporating advanced classification objectives for regularization can both enhance retrieval performance to match human-level performance and enable semantic vector arithmetic. We posit that these embeddings will serve as a foundational resource for further exploration of the Recipe1M dataset, as well as food and cooking in general. Code, data, and models are publicly available.

搜集汇总

背景与挑战

背景概述

Recipe1M+是一个大型多模态烹饪数据集，包含超过100万份食谱和1300万张食物图像，是目前最大的公开配方集合。该数据集设计用于训练高容量模型，支持图像和食谱的联合嵌入学习，以提升图像-食谱检索任务的性能，并已应用于语义向量算法研究。