danielsteinigen/StructVis

---
dataset_info:
features:
- name: id
dtype: string
- name: category_name
dtype: string
- name: user
dtype: string
- name: assistant
dtype: string
- name: image
dtype: image
- name: code
dtype: string
- name: lang_name
dtype: string
- name: type
dtype: string
- name: domain
dtype: string
- name: category_key
dtype: string
- name: lang_key
dtype: string
splits:
- name: train
num_bytes: 15337913863
num_examples: 216343
- name: val
num_bytes: 387361130
num_examples: 5550
- name: test
num_bytes: 179109704
num_examples: 2401
download_size: 15522444547
dataset_size: 15904384697
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: val
path: data/val-*
- split: test
path: data/test-*
license: cc-by-4.0
task_categories:
- visual-question-answering
language:
- en
tags:
- code
- structure
- diagram
- vision
size_categories:
- 100K<n<1M
---

# StructVis Dataset

StructVis is a large-scale dataset of structured diagrams and paired question-answer tasks, generated with the StructVis framework and rendered with the Structivize toolkit. It targets multi-domain vision-language understanding using formal representation languages (FRLs) and code-to-diagram grounding.

## Overview
- **Domains (7):** electrical & computer engineering, computer science & AI, software engineering & system modeling, biology, chemistry, business & process management, games & music
- **Scale:** 218K samples
- **Diagram categories (21):** structured diagrams such as circuit schematics, molecular structures, musical notation, business process flow charts, class diagrams, and more
- **Question types (8):**
- **Open-ended:** captioning, describing functionality, generative questions, image-code translation
- **Closed-ended:** detail questions, structural problems, association problems, consistency problems

## How It Was Built
The dataset is generated by the StructVis pipeline, which creates domain-specific FRL code, renders diagrams with Structivize, and applies multi-stage filtering. It includes explicit code-image mappings and problem-solution pairs to support training and evaluation of VLMs and LLMs on structured diagram understanding.

## Related Repositories
- **StructVis framework:** https://github.com/danielsteinigen/StructVis
- **Structivize rendering toolkit:** https://github.com/danielsteinigen/structivize

## Paper
- **Code-Guided Reasoning in Vision-Language Models for Complex Diagram Understanding** — ESANN 2026. https://doi.org/10.14428/esann/2026.ES2026-372

## Intended Use
- Training and evaluation of VLMs on structured diagram understanding
- Code-to-image grounding and FRL-based reasoning tasks
- Benchmarking model performance across diverse domains and diagram types

## Dataset Fields
- `id` (string)
- `category_name` (string)
- `user` (string)
- `assistant` (string)
- `image` (image)
- `code` (string)
- `lang_name` (string)
- `type` (string)
- `domain` (string)
- `category_key` (string)
- `lang_key` (string)

## Splits
- `train`: 216,343 examples
- `val`: 5,550 examples
- `test`: 2,401 examples

## License
[CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)