Im4Sketch Dataset

Im4Sketch is a large-scale dataset with shape-oriented set of classes for image-to-sketch generalization . It consists of a collection of natural images from 874 categories for training and validation, and sketches from 393 categories (a subset of natural image categories) for testing.

The images and sketches are collected from existing popular computer vision datasets. The categories are selected having shape similarity in mind, so that object with same shape belong to the same category.

The natural-image part of the dataset is based on the ILSVRC2012 version of ImageNet. The original ImageNet categories are first merged according to the shape criteria. Object categories for objects whose shape, e.g. how a human would draw the object, is the same are merged. For this step, semantic similarity of categories, obtained through WordNet and category names, is used to obtain candidate categories for merging. Based on visual inspection of these candidates, the decision to merge the original ImageNet classes is made by a human. For instance, ”Indian Elephant” and ”African Elephant”, or ”Laptop” and ”Notebook” are merged. An extreme case of merging is the new class “dog” that is a union of 121 original ImageNet classes of dog breeds.

In the second step, classes from datasets containing sketches are used. In particular, DomainNet, Sketchy, PACS, and TU-Berlin. Note that merging is not necessary for classes in these datasets, because the shape criteria are guaranteed since they are designed for sketches. In this step, a correspondence between the merged ImageNet categories and categories of the other datasets is found. As in the merging step, semantic similarity is used to guide the correspondence search. Sketch categories that are not present in the merged ImageNet are added to the overall category set, while training natural images of those categories are collected from either DomainNet or Sketchy. In the end, ImageNet is used for 690 classes, DomainNet for 183 classes, and Sketchy for 1 class, respectively.

Im4Sketch 是一个大规模的数据集，其类别设置以形状为导向，旨在实现图像到草图的一般化。该数据集包含来自 874 个类别的自然图像，用于训练和验证，以及来自 393 个类别（自然图像类别的子集）的草图，用于测试。

图像和草图均来自现有的流行计算机视觉数据集。类别选择时考虑了形状相似性，以确保形状相同的物体属于同一类别。

数据集的自然图像部分基于 ILSVRC2012 版本的 ImageNet。首先根据形状标准合并了原始 ImageNet 的类别。对于形状相同，例如人类绘制该物体时呈现的形状相同的物体类别，进行了合并。为此步骤，通过 WordNet 和类别名称获得了类别的语义相似性，以获取合并候选类别。基于对这些候选类别进行视觉检查，由人类决定合并原始 ImageNet 类别。例如，“印度象”和“非洲象”，或“笔记本电脑”和“笔记簿”被合并。合并的一个极端情况是新的类别“狗”，它是 121 个原始 ImageNet 狗品种类别的并集。

在第二步中，使用了包含草图的数据集的类别。特别是 DomainNet、Sketchy、PACS 和 TU-Berlin。请注意，对于这些数据集中的类别，合并是不必要的，因为它们是为草图设计的，形状标准得到保证。在此步骤中，找到了合并后的 ImageNet 类别与其他数据集类别的对应关系。与合并步骤一样，使用语义相似性来指导对应关系搜索。不在合并后的 ImageNet 中的草图类别被添加到整体类别集中，而那些类别的训练自然图像则从 DomainNet 或 Sketchy 中收集。最终，ImageNet 用于 690 个类别，DomainNet 用于 183 个类别，Sketchy 用于 1 个类别。