scb10x/typhoon-vision-preview-data

---
task_categories:
- visual-question-answering
- image-to-text
language:
- th
- en
pretty_name: Typhoon Vision Preview Data
size_categories:
- 100K<n<1M
---
# Typhoon Vision Preview Data

## Dataset Overview

This dataset is designed for visual question-answering and image-to-text tasks, supporting both Thai (th) and English (en) languages.

## Data Source

The dataset is based on the Bunny image dataset. You can download the original images from [here](https://huggingface.co/datasets/BoyaWu10/Bunny-v1_0-data).

## Dataset Splits

The dataset is organized into multiple splits, available as Hugging Face datasets:

1. **Pretrain**: Used for pretraining the adapter in LLaVA format.
2. **Finetune**: Used for finetuning in LLaVA format.
3. **Finetune_translated_stats**: Contains original texts, their Thai translations, and COMET scores (translation quality estimation).

### Pretraining Set

- Comprises the original Bunny data.
- Includes an additional 10% of translated data appended to the original set.
- COMET QE scores were not computed for this set.

### Finetuning Set

- Based on the same structure as the pretraining set.
- The appended 10% data consists of top-performing translations, as determined by COMET scores.

## File Descriptions

- `pretrain.json`: Dataset for pretraining the adapter in LLaVA format.
- `finetune.json`: Dataset for finetuning in LLaVA format.
- `finetune_translated_stats.json`: Contains original texts, Thai translations, and COMET scores.

## Usage Notes

- The dataset is designed for use with the LLaVA (Large Language and Vision Assistant) format.
- When using the finetuning set, be aware that it includes high-quality translations based on COMET scores.
- The pretraining set can be used for initial model adaptation, while the finetuning set is optimized for final model tuning.

## Language Support

This dataset supports bilingual tasks:
- Thai (th)
- English (en)

Researchers and developers can use this dataset for tasks involving both languages, especially for cross-lingual visual question-answering and image-to-text generation.