Earth on Canvas

With the advancement in sensor technology, huge amounts of data are being collected from various satellites. Hence, the task of target-based data retrieval and acquisition has become exceedingly challenging. Existing satellites essentially scan a vast overlapping region of the Earth using various sensing techniques, like multi-spectral, hyperspectral, Synthetic Aperture Radar (SAR), video, and compressed sensing, to name a few. With increasing complexity and different sensing techniques at our disposal, it has become our primary interest to design efficient algorithms to retrieve data from multiple data modalities, given the complementary information that is captured by different sensors. This type of problem is referred to as inter-modal data retrieval. In remote sensing (RS), there are primarily two important types of problems, i.e., land-cover classification and object detection. In this work, we focus on the target-based object retrieval part, which falls under the realm of object detection in RS. Object retrieval essentially requires high-resolution imagery for objects to be distinctly visible in the image. The main challenge with the conventional retrieval approach using large-scale databases is that, quite often, we do not have any query image sample of the target class at our disposal. The target of interest solely exists as a perception to the user in the form of an imprecise sketch. In such situations where a photo query is absent, it can be immensely useful if we can promptly make a quick hand-made sketch of the target. Sketches are a highly symbolic and hieroglyphic representation of data. One can exploit the notion of this minimalistic representative of sketch queries for sketch-based image retrieval (SBIR) framework. While dealing with satellite images, it is imperative to collect as many samples of images as possible for each object class for object recognition with a high success rate. However, in general, there exists a considerable number of classes for which we seldom have any training data samples. Therefore, for such classes, we can use the zero-shot learning (ZSL) strategy. The ZSL approach aims to solve a task without receiving any example of that task during the training phase. This makes the network capable of handling an unseen class (new class) sample obtained during the inference phase upon deployment of the network. In this database, we propose an aerial skecth-image database that can be useful for designing frameworks for the above-mentioned tasks.

随着传感器技术的进步，各类卫星正源源不断地采集海量数据。因此，基于目标的数据检索与获取任务变得极具挑战性。现有卫星普遍借助多光谱、高光谱、合成孔径雷达（Synthetic Aperture Radar, SAR）、视频成像与压缩感知等各类遥感技术，对地球的大范围重叠区域进行扫描。随着遥感技术复杂度持续提升、可用手段日益丰富，结合不同传感器捕获的互补信息，设计高效算法以检索多模态数据已成为核心研究方向，这类问题被称为跨模态数据检索（inter-modal data retrieval）。在遥感（Remote Sensing, RS）领域，核心研究问题主要分为两类：土地覆盖分类与目标检测。本研究聚焦于基于目标的对象检索任务，该任务隶属于遥感领域的目标检测分支。目标检索本质上依赖高分辨率影像，以确保目标在图像中清晰可辨。基于大规模数据库的传统检索方法面临的核心困境在于：多数情况下，我们无法获取目标类别的查询图像样本。用户仅能通过不精确的手绘草图，在脑海中构建目标对象的感知形象。在缺乏照片查询样本的场景下，若能快速绘制目标的手绘草图，将为检索任务提供极大便利。草图是一种高度符号化的象形数据表征形式，我们可借助这种极简的查询表征形式，构建基于草图的图像检索（Sketch-Based Image Retrieval, SBIR）框架。在处理卫星影像时，若要实现高成功率的目标识别，需为每个目标类别采集尽可能多的图像样本。但通常而言，存在大量目标类别几乎无可用训练数据样本。对此类类别，我们可采用零样本学习（Zero-Shot Learning, ZSL）策略。零样本学习的核心目标是，在训练阶段无需接触目标任务的任何示例即可完成该任务。这使得部署后的模型可在推理阶段处理未见类别（即新类别）的样本。本研究构建了一款航空草图-图像数据库，可用于为上述各类任务设计对应的算法框架。