Fhrozen/relaion-synthetic

---
license: apache-2.0
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
dataset_info:
features:
- name: image
dtype: image
- name: filename
dtype: string
- name: url
dtype: string
- name: text
dtype: string
- name: width
dtype: int64
- name: height
dtype: int64
- name: top_caption
dtype: string
- name: all_captions
list: string
- name: dense_caption
dtype: string
- name: vqa
dtype: string
- name: objects
dtype: string
- name: text_content
dtype: string
splits:
- name: train
num_bytes: 344524450516
num_examples: 5144102
download_size: 330644002120
dataset_size: 344524450516
---

# Relaion Synthetic - LLM-Annotated

[Original Source](https://huggingface.co/datasets/laion/relaion-synthetic-115m)

## 📌 Introduction

This dataset comprises images and annotations from the original Relaion Synthetic Dataset.

Out of the 115M images, a subset of **5.1M images** has been annotated with automatic methods (Image-text-to-text models).

## Captions

The annotations include four annotation columns:
- `dense_caption`: A dense annotation about the image
- `vqa`: Visual Question-Answers related to the image. JSON dictionary embedded as a string.
- `objects`: Object found in the image. JSON dictionary embedded as a string.
- `text_content`: OCRed text found in the image. JSON dictionary embedded as a string.

obtained from a Qwen3 VLM (https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Thinking-FP8).

**System Prompt:**

```python
sys_prompt = """You are a professional JSON data generator. Your responses must ALWAYS be valid, parseable JSON.
CRITICAL RULES:
- Output ONLY valid JSON, no additional text before or after
- Use double quotes for all strings
- Escape special characters properly (\\n, \\", \\\\)
- Boolean values must be lowercase: true, false
- Null values must be lowercase: null
- Do not use trailing commas
- Ensure all brackets and braces are properly closed"""
```

**User Prompt:**

```python
prompt = """Analyze this image and provide a detailed annotation in VALID JSON format.

STEP 1: CHECK FOR WATERMARKS
If you detect significant watermarks (Getty Images, shutterstock logos, large copyright overlays), respond with:
{"watermark_detected": true, "status": "rejected"}

Otherwise, proceed to STEP 2.

STEP 2: GENERATE COMPREHENSIVE ANNOTATION
Return a JSON object with these exact fields:

{
"watermark_detected": false,
"dense_caption": "<Write a detailed 3-5 sentence paragraph describing the scene. Include: overall atmosphere, main objects and their spatial locations (left/right/center, foreground/background), colors, textures, lighting, relationships between objects, and any actions or emotions conveyed.>",
"objects": [
{"object_name": "<name>", "attributes": "<color, material, condition>", "location_hint": "<position in frame>"}
],
"text_content": {
"has_text": <true or false>,
"transcription": "<actual text from signs, labels, books, etc. or null>",
"context": "<what the text is on or null>"
},
"vqa_dataset": [
{"question": "<perception/counting/reasoning question>", "answer": "<answer>", "type": "<Perception|Counting|Reasoning|OCR>"}
]
}

REQUIREMENTS:
- Generate 5-10 VQA pairs covering different question types
- Do NOT ask about watermarks, timestamps, or camera metadata
- List 3-10 key objects with their attributes
- Keep all text in a single line (no literal newlines in strings)
- Ensure the response is ONLY the JSON object, nothing else

OUTPUT ONLY VALID JSON - NO MARKDOWN, NO EXPLANATIONS."""
```

The request JSON is:

```python
data = {
"model": "llm-model",
"messages": [
{"role": "system", "content": [{"type": "text", "text": sys_prompt}]},
{"role": "user", "content": [
{"type": "text", "text": prompt},
{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{encoded_image}"} }
]}
],
"stream": False,
"temperature": 0.7,
"max_completion_tokens": 8192,
}
```

## Licensing

The generated prompts and descriptions are licensed under the Apache 2.0 license.
The images obtained from the original repository remain under their respective licenses.
In the event of any license issue, an image will be removed without prior notice.

## 🙏 Acknowledgement

All credits to the original Laion team.

许可证：Apache-2.0
配置项：
- 配置名称：default
数据文件：
- 划分集：train
文件路径：data/train-*
数据集信息：
特征字段：
- 字段名：图像（image），数据类型：图像
- 字段名：文件名（filename），数据类型：字符串（string）
- 字段名：链接（url），数据类型：字符串（string）
- 字段名：文本（text），数据类型：字符串（string）
- 字段名：宽度（width），数据类型：int64
- 字段名：高度（height），数据类型：int64
- 字段名：顶部标题（top_caption），数据类型：字符串（string）
- 字段名：全部标题（all_captions），数据类型：字符串列表（list[string]）
- 字段名：密集标注（dense_caption），数据类型：字符串（string）
- 字段名：视觉问答（VQA），数据类型：字符串（string）
- 字段名：目标对象（objects），数据类型：字符串（string）
- 字段名：文本内容（text_content），数据类型：字符串（string）
划分集：
- 划分名称：train，占用字节数：344524450516，样本数量：5144102
下载总大小：330644002120，数据集总大小：344524450516

# Relaion Synthetic - LLM-Annotated
[原始来源](https://huggingface.co/datasets/laion/relaion-synthetic-115m)

## 📌 数据集简介
本数据集涵盖源自原始Relaion Synthetic数据集的图像与标注。在1.15亿张原始图像中，有510万张图像子集通过自动方法（图像-文本转文本模型）完成标注。

## 标注字段说明
本数据集包含四类标注列：
- `dense_caption`：针对图像的密集场景标注，用于详细描述图像整体内容
- `vqa`：与图像相关的视觉问答（Visual Question-Answering, VQA）内容，以字符串形式封装的JSON字典
- `objects`：图像中识别出的目标对象信息，以字符串形式封装的JSON字典
- `text_content`：图像中的光学字符识别（Optical Character Recognition, OCR）文本内容，以字符串形式封装的JSON字典

上述标注均由Qwen3 VLM模型（https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Thinking-FP8）生成。

**系统提示词：**
python
sys_prompt = """You are a professional JSON data generator. Your responses must ALWAYS be valid, parseable JSON.
CRITICAL RULES:
- Output ONLY valid JSON, no additional text before or after
- Use double quotes for all strings
- Escape special characters properly (\n, ", \\)
- Boolean values must be lowercase: true, false
- Null values must be lowercase: null
- Do not use trailing commas
- Ensure all brackets and braces are properly closed"""

**用户提示词：**
python
prompt = """Analyze this image and provide a detailed annotation in VALID JSON format.

STEP 1: CHECK FOR WATERMARKS
If you detect significant watermarks (Getty Images, shutterstock logos, large copyright overlays), respond with:
{"watermark_detected": true, "status": "rejected"}

Otherwise, proceed to STEP 2.

STEP 2: GENERATE COMPREHENSIVE ANNOTATION
Return a JSON object with these exact fields:

{
"watermark_detected": false,
"dense_caption": "<Write a detailed 3-5 sentence paragraph describing the scene. Include: overall atmosphere, main objects and their spatial locations (left/right/center, foreground/background), colors, textures, lighting, relationships between objects, and any actions or emotions conveyed.>",
"objects": [
{"object_name": "<name>", "attributes": "<color, material, condition>", "location_hint": "<position in frame>"}
],
"text_content": {
"has_text": <true or false>,
"transcription": "<actual text from signs, labels, books, etc. or null>",
"context": "<what the text is on or null>"
},
"vqa_dataset": [
{"question": "<perception/counting/reasoning question>", "answer": "<answer>", "type": "<Perception|Counting|Reasoning|OCR>"}
]
}

REQUIREMENTS:
- Generate 5-10 VQA pairs covering different question types
- Do NOT ask about watermarks, timestamps, or camera metadata
- List 3-10 key objects with their attributes
- Keep all text in a single line (no literal newlines in strings)
- Ensure the response is ONLY the JSON object, nothing else

OUTPUT ONLY VALID JSON - NO MARKDOWN, NO EXPLANATIONS."""

请求的JSON格式如下：
python
data = {
"model": "llm-model",
"messages": [
{"role": "system", "content": [{"type": "text", "text": sys_prompt}]},
{"role": "user", "content": [
{"type": "text", "text": prompt},
{"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{encoded_image}"} }
]},
],
"stream": false,
"temperature": 0.7,
"max_completion_tokens": 8192,
}

## 许可声明
本数据集生成的提示词与标注描述采用Apache 2.0许可证授权。从原始仓库获取的原始图像仍遵循其各自的版权协议。若出现任何版权合规问题，相关图像将被立即移除且不另行通知。

## 🙏 致谢
谨向原始Laion团队致以全部敬意与感谢。