HallusionBench

<p align="center" width="100%">
<img src="https://i.postimg.cc/g0QRgMVv/WX20240228-113337-2x.png" width="100%" height="80%">
</p>

# Large-scale Multi-modality Models Evaluation Suite

> Accelerating the development of large-scale multi-modality models (LMMs) with `lmms-eval`

🏠 [Homepage](https://lmms-lab.github.io/) | 📚 [Documentation](docs/README.md) | 🤗 [Huggingface Datasets](https://huggingface.co/lmms-lab)

# This Dataset

This is a formatted version of [HallusionBench](https://github.com/tianyi-lab/HallusionBench). It is used in our `lmms-eval` pipeline to allow for one-click evaluations of large multi-modality models.

```
@misc{guan2023hallusionbench,
title={HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination & Visual Illusion in Large Vision-Language Models},
author={Tianrui Guan and Fuxiao Liu and Xiyang Wu and Ruiqi Xian and Zongxia Li and Xiaoyu Liu and Xijun Wang and Lichang Chen and Furong Huang and Yaser Yacoob and Dinesh Manocha and Tianyi Zhou},
year={2023},
eprint={2310.14566},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{liu2023mitigating,
title={Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning},
author={Fuxiao Liu and Kevin Lin and Linjie Li and Jianfeng Wang and Yaser Yacoob and Lijuan Wang},
year={2023},
eprint={2306.14565},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{liu2023mmc,
title={MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuning},
author={Fuxiao Liu and Xiaoyang Wang and Wenlin Yao and Jianshu Chen and Kaiqiang Song and Sangwoo Cho and Yaser Yacoob and Dong Yu},
year={2023},
eprint={2311.10774},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```

<p align="center" width="100%">
<img src="https://i.postimg.cc/g0QRgMVv/WX20240228-113337-2x.png" width="100%" height="80%">
</p>

# 大规模多模态模型评测套件
> 借助`lmms-eval`加速大规模多模态模型（Large-scale Multi-modality Models, LMMs）的研发进程
🏠 [项目主页](https://lmms-lab.github.io/) | 📚 [文档](docs/README.md) | 🤗 [Huggingface 数据集仓库](https://huggingface.co/lmms-lab)

# 本数据集
本数据集是[HallusionBench](https://github.com/tianyi-lab/HallusionBench)的格式化版本，可集成于我们的`lmms-eval`评测流水线中，实现大规模多模态模型的一键评测。

@misc{guan2023hallusionbench,
title={HallusionBench：大型视觉语言模型中纠缠式语言幻觉与视觉错觉的高级诊断套件},
author={Tianrui Guan and Fuxiao Liu and Xiyang Wu and Ruiqi Xian and Zongxia Li and Xiaoyu Liu and Xijun Wang and Lichang Chen and Furong Huang and Yaser Yacoob and Dinesh Manocha and Tianyi Zhou},
year={2023},
eprint={2310.14566},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{liu2023mitigating,
title={通过鲁棒指令微调缓解大型多模态模型中的幻觉问题},
author={Fuxiao Liu and Kevin Lin and Linjie Li and Jianfeng Wang and Yaser Yacoob and Lijuan Wang},
year={2023},
eprint={2306.14565},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{liu2023mmc,
title={MMC：借助大规模指令微调推进多模态图表理解任务},
author={Fuxiao Liu and Xiaoyang Wang and Wenlin Yao and Jianshu Chen and Kaiqiang Song and Sangwoo Cho and Yaser Yacoob and Dong Yu},
year={2023},
eprint={2311.10774},
archivePrefix={arXiv},
primaryClass={cs.CL}
}

搜集汇总

背景与挑战

背景概述

HallusionBench是一个用于评估大规模多模态模型（LMMs）性能的数据集，特别关注语言幻觉和视觉错觉的诊断。该数据集采用Apache License 2.0许可证，已更新至2024年11月21日，下载量达2,910次。