llava-bench-in-the-wild

<p align="center" width="100%">
<img src="https://i.postimg.cc/g0QRgMVv/WX20240228-113337-2x.png" width="100%" height="80%">
</p>

# Large-scale Multi-modality Models Evaluation Suite

> Accelerating the development of large-scale multi-modality models (LMMs) with `lmms-eval`

🏠 [Homepage](https://lmms-lab.github.io/) | 📚 [Documentation](docs/README.md) | 🤗 [Huggingface Datasets](https://huggingface.co/lmms-lab)

# This Dataset

This is a formatted version of [LLaVA-Bench(wild)](https://llava-vl.github.io/) that is used in LLaVA. It is used in our `lmms-eval` pipeline to allow for one-click evaluations of large multi-modality models.

```
@misc{liu2023improvedllava,
author={Liu, Haotian and Li, Chunyuan and Li, Yuheng and Lee, Yong Jae},
title={Improved Baselines with Visual Instruction Tuning},
publisher={arXiv:2310.03744},
year={2023},
}

@inproceedings{liu2023llava,
author = {Liu, Haotian and Li, Chunyuan and Wu, Qingyang and Lee, Yong Jae},
title = {Visual Instruction Tuning},
booktitle = {NeurIPS},
year = {2023}
}
```

<p align="center" width="100%"><img src="https://i.postimg.cc/g0QRgMVv/WX20240228-113337-2x.png" width="100%" height="80%"></p>

# 大规模多模态模型评测套件（Large-scale Multi-modality Models Evaluation Suite）

> 借助`lmms-eval`加速大规模多模态模型（Large-scale Multi-modality Models，LMMs）的研发

🏠 [主页](https://lmms-lab.github.io/) | 📚 [文档](docs/README.md) | 🤗 [Huggingface数据集](https://huggingface.co/lmms-lab)

# 本数据集

本数据集是LLaVA中使用的[LLaVA-Bench（野生版）](https://llava-vl.github.io/)的格式化版本，可接入我们的`lmms-eval`流程，实现大规模多模态模型的一键评测。

@misc{liu2023improvedllava,
author={Liu, Haotian and Li, Chunyuan and Li, Yuheng and Lee, Yong Jae},
title={基于视觉指令微调的改进基线模型},
publisher={arXiv:2310.03744},
year={2023},
}

@inproceedings{liu2023llava,
author = {Liu, Haotian and Li, Chunyuan and Wu, Qingyang and Lee, Yong Jae},
title = {视觉指令微调},
booktitle = {NeurIPS},
year = {2023}
}