POPE

<p align="center" width="100%"> <img src="https://i.postimg.cc/g0QRgMVv/WX20240228-113337-2x.png" width="100%" height="80%"> </p> # Large-scale Multi-modality Models Evaluation Suite > Accelerating the development of large-scale multi-modality models (LMMs) with `lmms-eval` 🏠 [Homepage](https://lmms-lab.github.io/) | 📚 [Documentation](docs/README.md) | 🤗 [Huggingface Datasets](https://huggingface.co/lmms-lab) # This Dataset This is a formatted version of [POPE](https://github.com/RUCAIBox/POPE). It is used in our `lmms-eval` pipeline to allow for one-click evaluations of large multi-modality models. ``` @article{li2023evaluating, title={Evaluating object hallucination in large vision-language models}, author={Li, Yifan and Du, Yifan and Zhou, Kun and Wang, Jinpeng and Zhao, Wayne Xin and Wen, Ji-Rong}, journal={arXiv preprint arXiv:2305.10355}, year={2023} } ```<p align="center" width="100%"> <img src="https://i.postimg.cc/g0QRgMVv/WX20240228-113337-2x.png" width="100%" height="80%"> </p> # 大规模多模态模型评测套件 > 借助`lmms-eval`加速大规模多模态模型（Large-scale Multi-modality Models, LMMs）的研发 🏠 [项目主页](https://lmms-lab.github.io/) | 📚 [文档](docs/README.md) | 🤗 [Huggingface数据集仓库](https://huggingface.co/lmms-lab) # 本数据集 本数据集为[POPE](https://github.com/RUCAIBox/POPE)的格式化版本，可集成至我们的`lmms-eval`评测流水线中，支持大规模多模态模型的一键式评测。 @article{li2023evaluating, title={大规模视觉语言模型中的对象幻觉问题评测}, author={Li, Yifan and Du, Yifan and Zhou, Kun and Wang, Jinpeng and Zhao, Wayne Xin and Wen, Ji-Rong}, journal={arXiv预印本 arXiv:2305.10355}, year={2023} }

搜集汇总

背景与挑战

背景概述

POPE是一个用于评估大规模多模态模型（LMMs）的数据集，特别关注检测视觉语言模型中的物体幻觉问题。它是'lmms-eval'评估流水线的格式化版本，支持一键式模型评估，旨在加速多模态模型的开发进程。数据集基于相关研究论文构建，适用于模型性能基准测试和分析。