JA-VLM-Bench-In-the-Wild

# JA-VLM-Bench-In-the-Wild

## Dataset Description
**JA-VLM-Bench-In-the-Wild** is Japanese version of [LLaVA-Bench-In-the-Wild](https://huggingface.co/datasets/liuhaotian/llava-bench-in-the-wild).
We carefully collected a diverse set of 42 images with 50 questions in total. (For LLaVA-Bench-In-the-Wild, 24 images with 60 questions)

The images contain Japanese culture and objects in Japan. The Japanese questions and answers were generated with assistance from GPT-4V (gpt-4-vision-preview), OpenAI’s large-scale language-generation model and removed nonsense data by humans.
Compared to [JA-VG-VQA-500](https://huggingface.co/datasets/SakanaAI/JA-VG-VQA-500), it contains more challenging questions and requires richer responses.

To evaluate Japanese VLMs, please go to [our Github repository](https://github.com/SakanaAI/evolutionary-model-merge).

## Usage

```python
from datasets import load_dataset

dataset = load_dataset("SakanaAI/JA-VLM-Bench-In-the-Wild", split="test")
```

## Uses
The images in this dataset are sourced from Unsplash and are free to use under the Unsplash License.
They cannot be sold without significant modification and cannot be used to replicate similar or competing services.

## Citation

```bibtex
@misc{akiba2024evomodelmerge,
title = {Evolutionary Optimization of Model Merging Recipes},
author. = {Takuya Akiba and Makoto Shing and Yujin Tang and Qi Sun and David Ha},
year = {2024},
eprint = {2403.13187},
archivePrefix = {arXiv},
primaryClass = {cs.NE}
}
```

# JA-VLM-Bench-In-the-Wild

## 数据集说明
**JA-VLM-Bench-In-the-Wild** 是 [LLaVA-Bench-In-the-Wild](https://huggingface.co/datasets/liuhaotian/llava-bench-in-the-wild) 的日语版本。
我们精心收集了总计包含50个问题的42张多样化图像。（原LLaVA-Bench-In-the-Wild数据集包含24张图像与60个问题）

本数据集的图像涵盖日本文化与日本本土物品。日语问答由OpenAI的大语言模型（Large Language Model，LLM）GPT-4V（gpt-4-vision-preview）生成，并经人工剔除无意义数据。
相较于 [JA-VG-VQA-500](https://huggingface.co/datasets/SakanaAI/JA-VG-VQA-500)，本数据集的问题更具挑战性，且要求更丰富的回答形式。

若需评估日语视觉语言模型（Vision-Language Model，VLM），请访问[我们的GitHub仓库](https://github.com/SakanaAI/evolutionary-model-merge)。

## 使用方法
python
from datasets import load_dataset

dataset = load_dataset("SakanaAI/JA-VLM-Bench-In-the-Wild", split="test")

## 使用条款
本数据集的图像均源自Unsplash平台，可依据Unsplash许可协议免费使用。未经大幅修改，不得用于商业售卖，亦不得用于复现同类或竞争性服务。

## 引用格式
bibtex
@misc{akiba2024evomodelmerge,
title = {Evolutionary Optimization of Model Merging Recipes},
author. = {Takuya Akiba and Makoto Shing and Yujin Tang and Qi Sun and David Ha},
year = {2024},
eprint = {2403.13187},
archivePrefix = {arXiv},
primaryClass = {cs.NE}
}