ML4Sustain/EarthEmbeddings

---
license: cc-by-sa-4.0
task_categories:
- text-to-image
- image-to-image
- other
language:
- en
tags:
- satellite-imagery
- earth-observation
- embeddings
- geospatial
- clip
- majortom
size_categories:
- 10K<n<100K
- 100K<n<1M
---

<div style="display: flex; gap: 0.2em; align-items: center; justify-content: center;">
<a href="https://www.modelscope.cn/studios/VoyagerX/EarthExplorer"><img src="https://img.shields.io/badge/Open in ModelScope.cn-xGPU-624aff"></a>
<a href="https://www.modelscope.ai/studios/VoyagerX/EarthExplorer"><img src="https://img.shields.io/badge/Open in ModelScope.ai-CPU-624aff"></a>
<a href="https://huggingface.co/spaces/ML4Sustain/EarthExplorer"><img src="https://img.shields.io/badge/Open in HF Space-CPU-FFD21E"></a>
<a href="https://modelscope.cn/studios/VoyagerX/EarthExplorer/file/view/master/Tutorial.md?status=1"> <img src="https://img.shields.io/badge/Tutorial-📖-007bff"> </a>
<a href="https://www.modelscope.cn/learn/3958"> <img src="https://img.shields.io/badge/中文教程-📖-007bff"> </a>
</div>

# EarthEmbeddings

Satellite imagery embeddings dataset for the **EarthEmbeddingExplorer**, enabling natural language and location-based search of Earth observation data.

## Overview

This repository contains pre-computed embeddings of satellite imagery using state-of-the-art vision-language models. These embeddings power the [EarthEmbeddingExplorer](https://huggingface.co/spaces/ML4Sustain/EarthExplorer) application, which allows users to search for satellite images using text queries, image uploads, or geographic locations.

**Key features:**
- Global satellite imagery from Sentinel-2 (MajorTOM Core-S2L2A)
- Multiple embedding models optimized for Earth observation
- Fast similarity search without raw image preprocessing
- Ready-to-use Parquet format for efficient data access

## Dataset Description

### Data Source
- **Base dataset**: MajorTOM Core-S2L2A (Sentinel-2 Level 2A, 2.2M+ samples)
- **Processing**: Center crop (384×384 pixels) + uniform global sampling

### Embedding Models

### Embedding Models

Four state-of-the-art vision models are used:

| Model | Description | Training Data |
| :--- | :--- | :--- |
| **SigLIP** | General-purpose vision-language model | Web-scale natural image-text pairs |
| **DINOv2** | Self-supervised vision transformer | Web-scale natural images (self-supervised) |
| **FarSLIP** | Fine-grained satellite imagery model | Satellite image-text pairs |
| **SatCLIP** | Location-based satellite model | Satellite image-location pairs |

## Dataset Splits

### 1. `uniform_sample_250k` ⚠️ Preview
```
├── uniform_sample_250k
│ ├── dinov2
│ │ ├── DINOv2_grid_sample_center_224x224_249k_MajorTOM.parquet
│ │ └── DINOv2_grid_sample_center_384x384_244k.parquet
│ ├── farslip
│ │ └── FarSLIP_grid_sample_center_384x384_244k.parquet
│ ├── satclip
│ │ └── SatCLIP_grid_sample_center_384x384_244k.parquet
│ └── siglip
│ └── SigLIP_grid_sample_center_384x384_244k.parquet
```

- **~250,000** globally distributed satellite images
- **Current status**: Preview revision with ~244k pre-computed embeddings and ~249k embeddings sampled from [Major-TOM/Core-S2RGB-DINOv2](https://huggingface.co/datasets/Major-TOM/Core-S2RGB-DINOv2) available
- **Note**: About 4-6k original image chips were lost due to network error; full version coming soon
- **Crop size**: For the 1/9 sampled grids, we crop the central bbox in each grid. To ensure the image patches are the same for each model, we chose crop size of 384x384, for pre-computed embeddings, we chose the crop size at 384x384. So these embeddings could represent the same regions on Earth surface.

| Filename | Embedding Model | Crop Size | Model Input Size | Embedding Dim | Source |
|----------|-----------------|-----------|------------------|---------------|--------|
| `DINOv2_grid_sample_center_224x224_249k_MajorTOM.parquet` | [DINOv2-large](https://huggingface.co/facebook/dinov2-large) | 224×224 | 224×224 | 1024 | [Major-TOM/Core-S2RGB-DINOv2](https://huggingface.co/datasets/Major-TOM/Core-S2RGB-DINOv2) |
| `DINOv2_grid_sample_center_384x384_244k.parquet` | [DINOv2-large](https://huggingface.co/facebook/dinov2-large) | 384×384 | 224×224 | 1024 | Pre-computed |
| `FarSLIP_grid_sample_center_384x384_244k.parquet` | [FarSLIP-ViT-B-16](https://huggingface.co/ZhenShiL/FarSLIP) | 384×384 | 224×224 | 512 | Pre-computed |
| `SatCLIP_grid_sample_center_384x384_244k.parquet` | [SatCLIP-ViT16-L40](https://github.com/microsoft/satclip) | 384×384 | 224×224 | 256 | Pre-computed |
| `SigLIP_grid_sample_center_384x384_244k.parquet` | [SigLIP-SO400M-14](https://huggingface.co/timm/ViT-SO400M-14-SigLIP-384) | 384×384 | 384×384 | 1152 | Pre-computed |

### 2. `uniform_sample_22k`
- **22,000** globally distributed satellite images
- **Files**: `grid_sample_center_22k_{FarSLIP,SatCLIP,SigLIP}_384x384.parquet`

### 3. `Zhejiang_samples`
- **2,000** samples from Zhejiang region, China
- **Files**: `zhejiang_sample_center_2k_{FarSLIP,SatCLIP,SigLIP}_384x384.parquet`
- Regional case study dataset

## Data Format

All embeddings are stored in **Parquet** format:
- Efficient columnar storage for fast download
- 384×384 pixel satellite image crops

## Related Work

- **Tutorial**: [EarthEmbeddingExplorer Tutorial](https://huggingface.co/spaces/ML4Sustain/EarthExplorer/blob/main/Tutorial.md)
- **Application**: [EarthEmbeddingExplorer Space](https://huggingface.co/spaces/ML4Sustain/EarthExplorer)
- **Base Dataset**: [MajorTOM by ESA](https://github.com/ESA-PhiLab/MajorTOM)

## License

CC-BY-SA-4.0

license: CC-BY-SA-4.0
task_categories:
- text-to-image
- image-to-image
- other
language:
- en
tags:
- satellite-imagery
- earth-observation
- embeddings
- geospatial
- clip
- majortom
size_categories:
- 10K<n<100K
- 100K<n<1M
---

<div style="display: flex; gap: 0.2em; align-items: center; justify-content: center;">
<a href="https://www.modelscope.cn/studios/VoyagerX/EarthExplorer"><img src="https://img.shields.io/badge/在ModelScope.cn中启动-xGPU-624aff"></a>
<a href="https://www.modelscope.ai/studios/VoyagerX/EarthExplorer"><img src="https://img.shields.io/badge/在ModelScope.ai中启动-CPU-624aff"></a>
<a href="https://huggingface.co/spaces/ML4Sustain/EarthExplorer"><img src="https://img.shields.io/badge/在HF Space中启动-CPU-FFD21E"></a>
<a href="https://modelscope.cn/studios/VoyagerX/EarthExplorer/file/view/master/Tutorial.md?status=1"> <img src="https://img.shields.io/badge/教程-📖-007bff"> </a>
<a href="https://www.modelscope.cn/learn/3958"> <img src="https://img.shields.io/badge/中文教程-📖-007bff"> </a>
</div>

# EarthEmbeddings

卫星影像嵌入向量数据集，专为**EarthEmbeddingExplorer**打造，支持基于自然语言与地理位置的地球观测数据检索。

## 项目概述

本仓库包含基于前沿视觉语言模型预计算得到的卫星影像嵌入向量。这些嵌入向量为[EarthEmbeddingExplorer](https://huggingface.co/spaces/ML4Sustain/EarthExplorer)应用提供算力支撑，允许用户通过文本查询、图像上传或地理位置检索卫星影像。

**核心特性：**
- 采用Sentinel-2（MajorTOM Core-S2L2A）的全球卫星影像数据集
- 多款针对地球观测任务优化的嵌入模型
- 无需对原始影像进行预处理即可实现快速相似度检索
- 采用即用型Parquet格式存储，便于高效访问数据

## 数据集说明

### 数据来源
- **基础数据集**：MajorTOM Core-S2L2A（Sentinel-2 Level 2A，含220万+样本）
- **预处理流程**：中心裁剪（384×384像素）+ 全局均匀采样

### 嵌入模型

本次数据集采用四款前沿视觉模型，详情如下：

| 模型 | 描述 | 训练数据 |
| :--- | :--- | :--- |
| **SigLIP** | 通用型视觉语言模型 | 网页级自然图像-文本配对数据 |
| **DINOv2** | 自监督视觉Transformer（Transformer） | 网页级自然图像（自监督训练） |
| **FarSLIP** | 细粒度卫星影像专用模型 | 卫星图像-文本配对数据 |
| **SatCLIP** | 基于地理位置的卫星影像模型 | 卫星图像-位置配对数据 |

## 数据集划分

### 1. `uniform_sample_250k` ⚠️ 预览版

├── uniform_sample_250k
│ ├── dinov2
│ │ ├── DINOv2_grid_sample_center_224x224_249k_MajorTOM.parquet
│ │ └── DINOv2_grid_sample_center_384x384_244k.parquet
│ ├── farslip
│ │ └── FarSLIP_grid_sample_center_384x384_244k.parquet
│ ├── satclip
│ │ └── SatCLIP_grid_sample_center_384x384_244k.parquet
│ └── siglip
│ └── SigLIP_grid_sample_center_384x384_244k.parquet

- **约250,000** 张全球分布的卫星影像
- **当前状态**：预览版本，包含约244,000个预计算嵌入向量，另有约249,000个嵌入向量源自[Major-TOM/Core-S2RGB-DINOv2](https://huggingface.co/datasets/Major-TOM/Core-S2RGB-DINOv2)数据集
- **备注**：约4000-6000张原始影像切片因网络错误丢失，完整版本即将上线
- **裁剪尺寸**：针对1/9采样网格，我们裁剪每个网格的中心边界框。为确保各模型对应的图像切片一致，我们统一采用384×384的裁剪尺寸，预计算嵌入向量也对应此裁剪尺寸，因此这些嵌入向量可对应地球表面的同一区域。

| 文件名 | 嵌入模型 | 裁剪尺寸 | 模型输入尺寸 | 嵌入向量维度 | 数据来源 |
|----------|-----------------|-----------|------------------|---------------|--------|
| `DINOv2_grid_sample_center_224x224_249k_MajorTOM.parquet` | [DINOv2-large](https://huggingface.co/facebook/dinov2-large) | 224×224 | 224×224 | 1024 | [Major-TOM/Core-S2RGB-DINOv2](https://huggingface.co/datasets/Major-TOM/Core-S2RGB-DINOv2) |
| `DINOv2_grid_sample_center_384x384_244k.parquet` | [DINOv2-large](https://huggingface.co/facebook/dinov2-large) | 384×384 | 224×224 | 1024 | 预计算生成 |
| `FarSLIP_grid_sample_center_384x384_244k.parquet` | [FarSLIP-ViT-B-16](https://huggingface.co/ZhenShiL/FarSLIP) | 384×384 | 224×224 | 512 | 预计算生成 |
| `SatCLIP_grid_sample_center_384x384_244k.parquet` | [SatCLIP-ViT16-L40](https://github.com/microsoft/satclip) | 384×384 | 224×224 | 256 | 预计算生成 |
| `SigLIP_grid_sample_center_384x384_244k.parquet` | [SigLIP-SO400M-14](https://huggingface.co/timm/ViT-SO400M-14-SigLIP-384) | 384×384 | 384×384 | 1152 | 预计算生成 |

### 2. `uniform_sample_22k`
- **22,000** 张全球分布的卫星影像
- **文件列表**：`grid_sample_center_22k_{FarSLIP,SatCLIP,SigLIP}_384x384.parquet`

### 3. `Zhejiang_samples`
- **2,000** 张来自中国浙江省的样本
- **文件列表**：`zhejiang_sample_center_2k_{FarSLIP,SatCLIP,SigLIP}_384x384.parquet`
- **用途**：区域案例研究数据集

## 数据格式

所有嵌入向量均以**Parquet**格式存储：
- 采用高效的列式存储结构，便于快速下载
- 存储384×384像素的卫星影像裁剪切片

## 相关工作

- **教程**：[EarthEmbeddingExplorer 官方教程](https://huggingface.co/spaces/ML4Sustain/EarthExplorer/blob/main/Tutorial.md)
- **应用程序**：[EarthEmbeddingExplorer 在线空间](https://huggingface.co/spaces/ML4Sustain/EarthExplorer)
- **基础数据集**：[ESA 开源MajorTOM数据集](https://github.com/ESA-PhiLab/MajorTOM)

## 许可证

CC-BY-SA-4.0