BoxClass-CN

# FG-CLIP 2: A Bilingual Fine-grained Vision-language Alignment Model
Code: https://github.com/360CVGroup/FG-CLIP

FG-CLIP 2 is the foundation model for fine-grained vision-language understanding in both English and Chinese.
Across 29 datasets and 8 diverse tasks, it consistently surpasses recent strong baselines such as SigLIP 2 and MetaCLIP 2, achieving the best reported performance to date in both languages.

**[FG-CLIP 2: A Bilingual Fine-grained Vision-language Alignment Model](https://arxiv.org/abs/2510.10921)**
</br>
Chunyu Xie*, Bin Wang*, Fanjing Kong, Jincheng Li, Dawei Liang, Ji Ao, Dawei Leng†, Yuhui Yin(*Equal Contribution, ✝Corresponding Author)
</br>
[](https://arxiv.org/abs/2510.10921)
[](https://huggingface.co/collections/qihoo360/fg-clip-2-68ecbf9c548623bb78bc7913)
[](https://huggingface.co/collections/qihoo360/fg-clip-2-68ecbf9c548623bb78bc7913)
[](https://research.360.cn/sass/index)

**[FG-CLIP: Fine-Grained Visual and Textual Alignment](https://arxiv.org/abs/2505.05071)** ([code branch: v1.0](https://github.com/360CVGroup/FG-CLIP/tree/v1.0))
</br>
Chunyu Xie*, Bin Wang*, Fanjing Kong, Jincheng Li, Dawei Liang, Gengshen Zhang, Dawei Leng†, Yuhui Yin (*Equal Contribution, ✝Corresponding Author)
</br>
[](https://arxiv.org/abs/2505.05071)
[](https://icml.cc/Conferences/2025)
[](https://huggingface.co/collections/qihoo360/fg-clip-681da45d4acfb65c240a6d08)
[](https://huggingface.co/datasets/qihoo360/FineHARD)
[](https://deepwiki.com/360CVGroup/FG-CLIP)

## Data Preparation
To run the inference code for FG-CLIP 2, please follow the following step.

### Step 1: Download the model

#### Model Zoo

|Models | ViT | Model Weights | Demo |
|:-----------|:-----------------------:|:---------------------------------------------------------:|:--------------------------------------------------------:|
| FG-CLIP-Base | vit-base-patch16-224 | [🤗Huggingface](https://huggingface.co/qihoo360/fg-clip-base) | [Retrieval](https://huggingface.co/spaces/qihoo360/FG-CLIP-Retrieval-demo) & [Dense Feature](https://huggingface.co/spaces/qihoo360/FG-CLIP-Densefeature-demo) |
| FG-CLIP-Large | vit-large-patch14-336 | 🤗[Huggingface](https://huggingface.co/qihoo360/fg-clip-large) | |
| FG-CLIP2-Base | vit-base-patch16 | [🤗Huggingface](https://huggingface.co/qihoo360/fg-clip2-base) | [Retrieval](https://huggingface.co/spaces/qihoo360/FG-CLIP2-Retrieval-demo) & [Dense Feature](https://huggingface.co/spaces/qihoo360/FG-CLIP2-Densefeature-demo) |
| FG-CLIP2-Large | vit-large-patch16 | [🤗Huggingface](https://huggingface.co/qihoo360/fg-clip2-large) | |
| FG-CLIP2-So400m | vit-so400m-patch16 | [🤗Huggingface](https://huggingface.co/qihoo360/fg-clip2-so400m) | |

### Step 2: Prepare BoxClass-CN Dataset

First, pull the dataset from the following link.
[🤗BoxClass-CN](https://huggingface.co/datasets/qihoo360/BoxClass-CN)，After downloading, unzip all compressed files, you will obtain the following file structure:

```none
BoxClass-CN
├── jsonfile
| ├── valid_category_data_total_zh.json
├── images
| ├── 000010.jpg
│ ├── 000014.jpg
│ ├── 000043.jpg
│ ├── ...
│ ├── 199195.jpg
```

Benchmarks
|Model| BackBone |top1 Accuracy|
| ---- | ---- |---- |
|SigLIP 2|ViT-B/16|57.9|
|**FG-CLIP 2(ours)**|ViT-B/16|**60.7**|
|SigLIP 2|ViT-L/16|56.6|
|**FG-CLIP 2(ours)**|ViT-L/16|**68.6**|
|MetaCLIP 2|ViT-H/14|55.2|
|SigLIP 2|ViT-So/16|63.6|
|**FG-CLIP 2(ours)**|ViT-So/16|**66.5**|

## Citation
If you find BoxClass-CN useful for your research and applications, please cite using this BibTeX:

```
@article{xie2025fg2,
title={FG-CLIP 2: A Bilingual Fine-grained Vision-language Alignment Model},
author={Xie, Chunyu and Wang, Bin and Kong, Fanjing and Li, Jincheng and Liang, Dawei and Ao, Ji and Leng, Dawei and Yin, Yuhui},
journal={arXiv preprint arXiv:2510.10921},
year={2025}
}
```
```
@article{xie2025fg,
title={FG-CLIP: Fine-Grained Visual and Textual Alignment},
author={Xie, Chunyu and Wang, Bin and Kong, Fanjing and Li, Jincheng and Liang, Dawei and Zhang, Gengshen and Leng, Dawei and Yin, Yuhui},
journal={arXiv preprint arXiv:2505.05071},
year={2025}
}
```

## License

This project utilizes certain datasets and checkpoints that are subject to their respective original licenses. Users must comply with all terms and conditions of these original licenses.
The content of this project itself is licensed under the [Apache license 2.0](./LICENSE).

# FG-CLIP 2：双语细粒度视觉-语言对齐模型
代码仓库：https://github.com/360CVGroup/FG-CLIP

FG-CLIP 2是一款面向中英双语细粒度视觉-语言理解的基础模型。在涵盖29个数据集与8类多样化任务的基准测试中，该模型始终优于SigLIP 2、MetaCLIP 2等当前顶尖基线模型，在中英双语场景下均达到了截至目前已公开的最优性能。

**[FG-CLIP 2：双语细粒度视觉-语言对齐模型](https://arxiv.org/abs/2510.10921)**
</br>
谢春宇*, 王斌*, 孔繁静, 李金成, 梁大伟, 敖骥, 冷大伟†, 尹玉辉(*同等贡献作者, ✝通讯作者)
</br>
[](https://arxiv.org/abs/2510.10921)
[](https://huggingface.co/collections/qihoo360/fg-clip-2-68ecbf9c548623bb78bc7913)
[](https://huggingface.co/collections/qihoo360/fg-clip-2-68ecbf9c548623bb78bc7913)
[](https://research.360.cn/sass/index)

**[FG-CLIP：细粒度视觉与文本对齐](https://arxiv.org/abs/2505.05071)** ([代码分支: v1.0](https://github.com/360CVGroup/FG-CLIP/tree/v1.0))
</br>
谢春宇*, 王斌*, 孔繁静, 李金成, 梁大伟, 张耿深, 冷大伟†, 尹玉辉 (*同等贡献作者, ✝通讯作者)
</br>
[](https://arxiv.org/abs/2505.05071)
[](https://icml.cc/Conferences/2025)
[](https://huggingface.co/collections/qihoo360/fg-clip-681da45d4acfb65c240a6d08)
[](https://huggingface.co/datasets/qihoo360/FineHARD)
[](https://deepwiki.com/360CVGroup/FG-CLIP)

## 数据准备
若要运行FG-CLIP 2的推理代码，请按照以下步骤操作。

### 步骤1：下载模型

#### 模型库

|模型名称 | 视觉Transformer（ViT） | 模型权重 | 演示示例 |
|:-----------|:-----------------------:|:---------------------------------------------------------:|:--------------------------------------------------------:|
| FG-CLIP-Base | vit-base-patch16-224 | [🤗Hugging Face](https://huggingface.co/qihoo360/fg-clip-base) | [检索任务](https://huggingface.co/spaces/qihoo360/FG-CLIP-Retrieval-demo) & [稠密特征提取](https://huggingface.co/spaces/qihoo360/FG-CLIP-Densefeature-demo) |
| FG-CLIP-Large | vit-large-patch14-336 | 🤗[Hugging Face](https://huggingface.co/qihoo360/fg-clip-large) | |
| FG-CLIP2-Base | vit-base-patch16 | [🤗Hugging Face](https://huggingface.co/qihoo360/fg-clip2-base) | [检索任务](https://huggingface.co/spaces/qihoo360/FG-CLIP2-Retrieval-demo) & [稠密特征提取](https://huggingface.co/spaces/qihoo360/FG-CLIP2-Densefeature-demo) |
| FG-CLIP2-Large | vit-large-patch16 | [🤗Hugging Face](https://huggingface.co/qihoo360/fg-clip2-large) | |
| FG-CLIP2-So400m | vit-so400m-patch16 | [🤗Hugging Face](https://huggingface.co/qihoo360/fg-clip2-so400m) | |

### 步骤2：准备BoxClass-CN数据集

首先，请从以下链接获取该数据集：[🤗Hugging Face BoxClass-CN](https://huggingface.co/datasets/qihoo360/BoxClass-CN)。下载完成后解压所有压缩包，将得到如下文件结构：

none
BoxClass-CN
├── jsonfile
| ├── valid_category_data_total_zh.json
├── images
| ├── 000010.jpg
│ ├── 000014.jpg
│ ├── 000043.jpg
│ ├── ...
│ ├── 199195.jpg

## 基准测试结果

|模型| 骨干网络 |Top-1 准确率|
| ---- | ---- |---- |
|SigLIP 2|ViT-B/16|57.9|
|**FG-CLIP 2(本文提出)**|ViT-B/16|**60.7**|
|SigLIP 2|ViT-L/16|56.6|
|**FG-CLIP 2(本文提出)**|ViT-L/16|**68.6**|
|MetaCLIP 2|ViT-H/14|55.2|
|SigLIP 2|ViT-So/16|63.6|
|**FG-CLIP 2(本文提出)**|ViT-So/16|**66.5**|

## 引用
若您的研究或应用中使用了BoxClass-CN数据集，请使用以下BibTeX格式进行引用：

@article{xie2025fg2,
title={FG-CLIP 2: A Bilingual Fine-grained Vision-language Alignment Model},
author={Xie, Chunyu and Wang, Bin and Kong, Fanjing and Li, Jincheng and Liang, Dawei and Ao, Ji and Leng, Dawei and Yin, Yuhui},
journal={arXiv preprint arXiv:2510.10921},
year={2025}
}

@article{xie2025fg,
title={FG-CLIP: Fine-Grained Visual and Textual Alignment},
author={Xie, Chunyu and Wang, Bin and Kong, Fanjing and Li, Jincheng and Liang, Dawei and Zhang, Gengshen and Leng, Dawei and Yin, Yuhui},
journal={arXiv preprint arXiv:2505.05071},
year={2025}
}

## 许可证
本项目使用的部分数据集与模型权重需遵循其各自的原始许可证条款，使用者需严格遵守这些原始许可的所有要求。本项目本身的内容采用 [Apache许可证2.0](./LICENSE) 进行许可。