Earth on Canvas

With the advancement in sensor technology, huge amounts of data are being collected from various satellites. Hence, the task of target-based data retrieval and acquisition has become exceedingly challenging. Existing satellites essentially scan a vast overlapping region of the Earth using various sensing techniques, like multi-spectral, hyperspectral, Synthetic Aperture Radar (SAR), video, and compressed sensing, to name a few. With increasing complexity and different sensing techniques at our disposal, it has become our primary interest to design efficient algorithms to retrieve data from multiple data modalities, given the complementary information that is captured by different sensors. This type of problem is referred to as inter-modal data retrieval. In remote sensing (RS), there are primarily two important types of problems, i.e., land-cover classification and object detection. In this work, we focus on the target-based object retrieval part, which falls under the realm of object detection in RS. Object retrieval essentially requires high-resolution imagery for objects to be distinctly visible in the image. The main challenge with the conventional retrieval approach using large-scale databases is that, quite often, we do not have any query image sample of the target class at our disposal. The target of interest solely exists as a perception to the user in the form of an imprecise sketch. In such situations where a photo query is absent, it can be immensely useful if we can promptly make a quick hand-made sketch of the target. Sketches are a highly symbolic and hieroglyphic representation of data. One can exploit the notion of this minimalistic representative of sketch queries for sketch-based image retrieval (SBIR) framework. While dealing with satellite images, it is imperative to collect as many samples of images as possible for each object class for object recognition with a high success rate. However, in general, there exists a considerable number of classes for which we seldom have any training data samples. Therefore, for such classes, we can use the zero-shot learning (ZSL) strategy. The ZSL approach aims to solve a task without receiving any example of that task during the training phase. This makes the network capable of handling an unseen class (new class) sample obtained during the inference phase upon deployment of the network. In this database, we propose an aerial skecth-image database that can be useful for designing frameworks for the above-mentioned tasks.

随着传感器技术的进步，各类卫星正源源不断采集海量数据。因此，面向目标的数据检索与获取任务已极具挑战性。当前卫星大多通过多光谱、高光谱、合成孔径雷达（Synthetic Aperture Radar，SAR）、视频成像、压缩感知等多种传感技术，对地球的大面积重叠区域进行扫描。面对日益复杂多样的传感技术，结合不同传感器所捕获的互补信息，设计高效的多模态数据检索算法已成为核心研究目标，这类问题被称为跨模态数据检索。在遥感（Remote Sensing，RS）领域，核心研究问题主要分为两类：土地覆盖分类与目标检测。本研究聚焦面向目标的对象检索任务，该任务隶属于遥感领域的目标检测范畴。目标检索本质上需要高分辨率影像，以使目标在图像中清晰可见。基于大规模数据库的传统检索方法面临的主要挑战在于：通常情况下，我们无法获取目标类别的查询图像样本，用户仅能以模糊手绘草图的形式形成对目标的直观感知。在缺乏照片查询样本的场景中，若能快速手绘出目标草图，将极具应用价值。草图是一种高度符号化的象形数据表征形式，可将这种极简的草图查询形式应用于基于草图的图像检索（Sketch-based Image Retrieval，SBIR）框架。在处理卫星影像时，为实现高成功率的目标识别，需为每个目标类别收集尽可能多的图像样本。但在实际场景中，存在大量类别几乎没有可用的训练数据样本，因此针对这类类别，可采用零样本学习（Zero-shot Learning，ZSL）策略。零样本学习旨在无需在训练阶段接触目标任务的任何样本即可完成该任务，这使得模型在部署后的推理阶段，能够处理未见类别（即新类别）的样本。本数据集提出了一种航空草图-图像数据库，可用于为上述各类任务设计相关框架。