NASK-PIB/MMBench_V11_PL

---
license: cc-by-nc-4.0
configs:
- config_name: pl
data_files:
- split: dev
path: data/mmbench_v11_pl_dev.parquet
- config_name: en
data_files:
- split: dev
path: data/mmbench_v11_en_dev.parquet
dataset_info:
features:
- name: index
dtype: int64
- name: split
dtype: string
- name: category
dtype: string
- name: l2-category
dtype: string
- name: image
dtype: image
- name: question
dtype: string
- name: hint
dtype: string
- name: A
dtype: string
- name: B
dtype: string
- name: C
dtype: string
- name: D
dtype: string
- name: answer
dtype: string
task_categories:
- visual-question-answering
language:
- pl
size_categories:
- 1K<n<10K
---

# Polish MMBench V1.1 (Dev)

## Overview
This dataset is a Polish translation of the English MMBench V1.1 Dev set.
It serves as a comprehensive multiple-choice benchmark to systematically evaluate vision-language models across diverse capabilities,
including fine-grained perception and logical reasoning.

## Dataset Creation
The dataset was created using an automated translation followed by manual corrections:
* **Translation:** The English MMBench Dev set was initially translated into Polish using the [Tower+ 72B](https://huggingface.co/Unbabel/Tower-Plus-72B) model.
* **Manual Correction:** Professional native Polish linguists reviewed the translations and corrected linguistic and content issues.

## Dataset Structure
The dataset provides two configurations: `pl` (Polish translation) and `en` (original English). Each sample contains:
* `index`: Unique identifier.
* `split`: Dataset split (dev).
* `category` & `l2-category`: Task categories.
* `image`: The visual input.
* `question`: The multiple-choice question.
* `hint`: An optional contextual hint.
* `A`, `B`, `C`, `D`: Answer choices.
* `answer`: The correct choice.

## Usage
You can load the dataset using the `datasets` library. Specify the configuration name (`pl` or `en`) to load the desired language version.

```python
from datasets import load_dataset

# Load the translated Polish version
mmbenchv11_pl = load_dataset("NASK-PIB/MMBench_V11_PL", "pl")

# Load the original English version
mmbenchv11_en = load_dataset("NASK-PIB/MMBench_V11_PL", "en")
```

## License
This dataset is distributed under the [Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)](https://creativecommons.org/licenses/by-nc/4.0/) license.

## Citation
If you use this dataset, please cite the following paper:

```bibtex
@inproceedings{statkiewicz2026annotation,
title = {Annotation-Efficient Vision-Language Model Adaptation to the Polish Language Using the LLaVA Framework},
author = {Statkiewicz, Grzegorz and
Dobrzeniecka, Alicja and
Seweryn, Karolina and
Krasnod{\k e}bska, Aleksandra and
Piosek, Karolina and
Bogusz, Katarzyna and
Cygert, Sebastian and
Kusa, Wojciech},
booktitle = {Proceedings of the Student Workshop at the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2026)},
year = {2026},
publisher = {Association for Computational Linguistics}
}
```