remyxai/SpaceJudgeDataset
收藏Hugging Face2024-11-14 更新2025-04-12 收录
下载链接：
https://hf-mirror.com/datasets/remyxai/SpaceJudgeDataset
下载链接
链接失效反馈官方服务：
资源简介：
---
dataset_info:
features:
- name: images
dtype: image
- name: messages
list:
- name: content
dtype: string
- name: role
dtype: string
splits:
- name: train
num_bytes: 60681467494.556
num_examples: 62201
download_size: 17937830623
dataset_size: 60681467494.556
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
task_categories:
- visual-question-answering
tags:
- remyx
- vqasynth
pretty_name: SpaceJudgeDataset
size_categories:
- 1K<n<10K
---
# SpaceJudge Dataset
The SpaceJudge Dataset uses [prometheus-vision](https://github.com/prometheus-eval/prometheus-vision) to apply
a rubric assessing the quality of response to spatial VQA inquiries on a 1-5 likert scale by prompting
[SpaceLLaVA](https://huggingface.co/remyxai/SpaceLLaVA) to perform VLM-as-a-Judge.
[](https://colab.research.google.com/drive/1zOxSpMIjfWM6desF5Ai-iIk1szhlurUW?usp=sharing)
The assessment is made for images in the [OpenSpaces](https://huggingface.co/datasets/remyxai/OpenSpaces) dataset in order to
distill the 13B VLM judge into smaller models like [Florence-2](https://huggingface.co/collections/microsoft/florence-6669f44df0d87d9c3bfb76de)
by introducing a new `<JUDGE>` task.
## Citations
```
@misc{lee2024prometheusvision,
title={Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluation},
author={Seongyun Lee and Seungone Kim and Sue Hyun Park and Geewook Kim and Minjoon Seo},
year={2024},
eprint={2401.06591},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```
提供机构：
remyxai
