Ferret-Bench

<p align="center" width="100%">
<img src="https://i.postimg.cc/g0QRgMVv/WX20240228-113337-2x.png" width="100%" height="80%">
</p>

# Large-scale Multi-modality Models Evaluation Suite

> Accelerating the development of large-scale multi-modality models (LMMs) with `lmms-eval`

🏠 [Homepage](https://lmms-lab.github.io/) | 📚 [Documentation](docs/README.md) | 🤗 [Huggingface Datasets](https://huggingface.co/lmms-lab)

# This Dataset

This is a formatted version of [FerretBench](https://github.com/apple/ml-ferret). It is used in our `lmms-eval` pipeline to allow for one-click evaluations of large multi-modality models.

```
@article{you2023ferret,
title={Ferret: Refer and Ground Anything Anywhere at Any Granularity},
author={You, Haoxuan and Zhang, Haotian and Gan, Zhe and Du, Xianzhi and Zhang, Bowen and Wang, Zirui and Cao, Liangliang and Chang, Shih-Fu and Yang, Yinfei},
journal={arXiv preprint arXiv:2310.07704},
year={2023}
}
```

<p align="center" width="100%">
<img src="https://i.postimg.cc/g0QRgMVv/WX20240228-113337-2x.png" width="100%" height="80%">
</p>

# 大规模多模态模型评测套件（Large-scale Multi-modality Models Evaluation Suite）

> 依托`lmms-eval`加速大规模多模态模型（Large-scale Multi-modality Models，以下简称LMMs）的研发进程

🏠 [项目主页](https://lmms-lab.github.io/) | 📚 [使用文档](docs/README.md) | 🤗 [Huggingface数据集仓库](https://huggingface.co/lmms-lab)

## 本数据集

本数据集是[FerretBench](https://github.com/apple/ml-ferret)的格式化版本，被集成至我们的`lmms-eval`评测流水线中，可实现大规模多模态模型的一键式评测。

@article{you2023ferret,
title={Ferret：任意粒度下的全域指代与定位（Ferret: Refer and Ground Anything Anywhere at Any Granularity）},
author={You, Haoxuan、Zhang, Haotian、Gan, Zhe、Du, Xianzhi、Zhang, Bowen、Wang, Zirui、Cao, Liangliang、Chang, Shih-Fu 和 Yang, Yinfei},
journal={arXiv预印本 arXiv:2310.07704},
year={2023}
}