SenseNova-Vision-Benchmark

# Vision as Unified Multimodal Generation <p align="center"> <strong>English</strong> | <a href="https://huggingface.co/datasets/sensenova/SenseNova-Vision-Benchmark/blob/main/README_CN.md">简体中文</a> </p> <p align="center"> <a href="https://github.com/OpenSenseNova/SenseNova-Vision"><img alt="GitHub Stars" src="https://img.shields.io/github/stars/OpenSenseNova/SenseNova-Vision?style=social"></a> <a href="https://arxiv.org/abs/2607.06560"><img alt="arXiv" src="https://img.shields.io/badge/arXiv-2607.06560-b31b1b.svg"></a> <a href="https://huggingface.co/sensenova/SenseNova-Vision-7B-MoT"><img src="https://img.shields.io/static/v1?label=%F0%9F%A4%97%20Hugging%20Face&message=Model&color=green"></a> <a href="https://huggingface.co/datasets/sensenova/SenseNova-Vision-Corpus-50M"><img src="https://img.shields.io/static/v1?label=%F0%9F%A4%97%20Hugging%20Face&message=Dataset&color=yellow"></a> <a href="https://huggingface.co/datasets/sensenova/SenseNova-Vision-Benchmark"><img src="https://img.shields.io/static/v1?label=%F0%9F%A4%97%20Hugging%20Face&message=Benchmark&color=orange"></a> <a href="https://creativecommons.org/licenses/by-nc/4.0/" target="_blank"><img src="https://img.shields.io/badge/License-CC%20BY--NC%204.0-blue.svg" alt="License"></a> </p> # SenseNova OpenSource Benchmark JSONL ## Introduction This repository provides a collection of benchmark JSONL files for evaluating open-source vision-language models, covering detection and segmentation tasks. All files are organized in JSONL format, making them easy to plug into unified evaluation scripts or convert to the formats required by other evaluation frameworks. ## Dataset Size The current directory contains 52 JSONL files. The archive `SenseNova-Vision_benchmark_jsonl.tar.gz` provides the same set of JSONL files in a packaged format. The number of samples for each task file is listed in the directory structure below. ## Directory Structure ```bash SenseNova-Vision-Benchmark/ ├── detection/ # Detection, OCR, document, and grounding evaluation │ ├── COCO.jsonl │ ├── Dense200.jsonl │ ├── DocLayNet.jsonl │ ├── HierText.jsonl │ ├── HumanRef.jsonl │ ├── IC15.jsonl │ ├── LVIS.jsonl │ ├── RefCOCOg_test.jsonl │ ├── RefCOCOg_val.jsonl │ ├── SROIE.jsonl │ ├── TotalText.jsonl │ ├── VisDrone.jsonl │ ├── gui_eval/ # GUI icon/text localization evaluation │ │ ├── screenspot_desktop_v2_icon.jsonl │ │ ├── screenspot_desktop_v2_text.jsonl │ │ ├── screenspot_mobile_v2_icon.jsonl │ │ ├── screenspot_mobile_v2_text.jsonl │ │ ├── screenspot_web_v2_icon.jsonl │ │ ├── screenspot_web_v2_text.jsonl │ │ ├── ScreenSpotPro_cad_icon.jsonl │ │ ├── ScreenSpotPro_cad_text.jsonl │ │ ├── ScreenSpotPro_creative_icon.jsonl │ │ ├── ScreenSpotPro_creative_text.jsonl │ │ ├── ScreenSpotPro_dev_icon.jsonl │ │ ├── ScreenSpotPro_dev_text.jsonl │ │ ├── ScreenSpotPro_office_icon.jsonl │ │ ├── ScreenSpotPro_office_text.jsonl │ │ ├── ScreenSpotPro_os_icon.jsonl │ │ ├── ScreenSpotPro_os_text.jsonl │ │ ├── ScreenSpotPro_sci_icon.jsonl │ │ └── ScreenSpotPro_sci_text.jsonl │ ├── keypoint_eval/ # Keypoint detection evaluation │ │ ├── ap-10k.jsonl │ │ └── coco.jsonl │ ├── point_eval/ # Pointing / point grounding evaluation │ │ ├── COCO.jsonl │ │ ├── Dense200.jsonl │ │ ├── HumanRef.jsonl │ │ ├── LVIS.jsonl │ │ ├── RefCOCOg_test.jsonl │ │ ├── RefCOCOg_val.jsonl │ │ └── VisDrone.jsonl │ └── visual_prompt_eval/ # Visual prompt evaluation │ ├── COCO.jsonl │ ├── Dense200.jsonl │ ├── FSCD_test.jsonl │ └── LVIS.jsonl ├── seg_ade20k_panoptic_val.jsonl # ADE20K panoptic segmentation ├── seg_gcg_test_gcgseg.jsonl # GCG segmentation test ├── seg_gcg_val_gcgseg.jsonl # GCG segmentation val ├── seg_panoptic_coco_val.jsonl # COCO panoptic segmentation val ├── seg_reason_test.jsonl # Reasoning segmentation test ├── seg_reason_val.jsonl # Reasoning segmentation val ├── seg_refcoco_val_binary.jsonl # RefCOCO binary segmentation val ├── seg_refcocog_val_binary.jsonl # RefCOCOg binary segmentation val └── seg_refcocop_val_binary.jsonl # RefCOCO+ binary segmentation val ```