Earth on Canvas

WITH the advancement in sensor technology, huge amounts of data are being collected from various satellites. Hence, the task of target-based data retrieval and acquisition has become exceedingly challenging. Existing satellites essentially scan a vast overlapping region of the Earth using various sensing techniques, like multi-spectral, hyperspectral, Synthetic Aperture Radar (SAR), video, and compressed sensing, to name a few. With increasing complexity and different sensing techniques at our disposal, it has become our primary interest to design efficient algorithms to retrieve data from multiple data modalities, given the complementary information that is captured by different sensors. This type of problem is referred to as inter-modal data retrieval. In remote sensing (RS), there are primarily two important types of problems, i.e., land-cover classification and object detection. In this work, we focus on the target-based object retrieval part, which falls under the realm of object detection in RS. Object retrieval essentially requires high-resolution imagery for objects to be distinctly visible in the image. The main challenge with the conventional retrieval approach using large-scale databases is that, quite often, we do not have any query image sample of the target class at our disposal. The target of interest solely exists as a perception to the user in the form of an imprecise sketch. In such situations where a photo query is absent, it can be immensely useful if we can promptly make a quick hand-made sketch of the target. Sketches are a highly symbolic and hieroglyphic representation of data. One can exploit the notion of this minimalistic representative of sketch queries for sketch-based image retrieval (SBIR) framework. While dealing with satellite images, it is imperative to collect as many samples of images as possible for each object class for object recognition with a high success rate. However, in general, there exists a considerable number of classes for which we seldom have any training data samples. Therefore, for such classes, we can use the zero-shot learning (ZSL) strategy. The ZSL approach aims to solve a task without receiving any example of that task during the training phase. This makes the network capable of handling an unseen class (new class) sample obtained during the inference phase upon deployment of the network. Hence, we propose the aerial sketch-image dataset, namely Earth on Canvas dataset.

随着传感器技术的进步，来自各种卫星的数据收集量正日益庞大。因此，基于目标的检索与数据获取任务变得极为艰巨。现有的卫星主要利用多种感知技术，例如多光谱、高光谱、合成孔径雷达（SAR）、视频和压缩感知等，对地球的广阔重叠区域进行扫描。面对日益复杂的场景和丰富的感知技术，设计高效的算法以从多种数据模态中检索数据，鉴于不同传感器捕获的互补信息，已成为我们的首要关注点。此类问题被称为跨模态数据检索。在遥感（RS）领域，主要存在两种重要的研究问题，即地表覆盖分类和目标检测。在本研究中，我们专注于基于目标的检索部分，这属于遥感目标检测的范畴。目标检索本质上需要高分辨率图像，以确保对象在图像中清晰可见。传统的基于大规模数据库的检索方法面临的主要挑战是，我们往往无法获得目标类别的查询图像样本。感兴趣的目标仅以用户感知的不精确草图的形式存在。在这种情况下，如果能够迅速绘制出目标的草图，将极为有用。草图是一种高度象征性和象形的数据表示。可以利用草图查询的最简代表概念，为基于草图图像检索（SBIR）框架提供支持。在处理卫星图像时，收集尽可能多的图像样本以供每个对象类别进行高成功率的目标识别至关重要。然而，一般来说，存在许多类别的数据，我们很少拥有任何训练数据样本。因此，对于这些类别，我们可以采用零样本学习（ZSL）策略。ZSL方法旨在在没有在训练阶段接收到该任务的任何示例的情况下解决任务。这使得网络能够在网络部署的推理阶段处理未见过的类别（新类别）样本。因此，我们提出了名为地球画布的航空草图图像数据集。