遇见数据集

llm-jp/Jagle

收藏
Hugging Face2026-05-12 更新2026-05-31 收录
官方服务:

资源简介:

--- language: - ja size_categories: - 1M<n<10M license: other --- <div align="center" style="line-height: 1;"> <h1>Jagle: Building a Large-Scale Japanese Multimodal Post-Training Dataset for Vision–Language Models</h1> | <a href="https://huggingface.co/collections/llm-jp/jagle" target="_blank">🤗 HuggingFace</a> &nbsp;| <a href="https://arxiv.org/abs/2604.02048" target="_blank">📄 Paper</a> &nbsp;| <a href="https://github.com/llm-jp/llm-jp-4-vl" target="_blank">🧑‍💻 Code</a> &nbsp;| <br/> <img src="./jagle-construction-pipeline.svg" width="50%"/> </div> ## Overview Jagle is a large-scale Japanese multimodal post-training dataset, comprising approximately 9.2 million instances across diverse tasks. Jagle was used to train [LLM-jp-4-VL 9B beta](https://huggingface.co/llm-jp/llm-jp-4-vl-9b-beta). We construct Jagle by collecting a wide range of source data, including image–text pair datasets such as [WAON](https://huggingface.co/datasets/llm-jp/WAON) and PDF corpora like [FinePDFs-Edu](https://huggingface.co/datasets/HuggingFaceFW/finepdfs-edu), and then generating VQA data through various methods such as VLM-based QA generation and translation. Below are representative VQA samples from Jagle. <figure> <img src="./samples.svg" alt="" style="width: 80%"> <figcaption>Samples in Jagle.</figcaption> </figure> Below, we present the performance trajectory of the average score over Japanese and English tasks for a 2.2B model composed of Qwen3-1.7B and the SigLIP2-so400m-patch16-512 vision encoder, trained with Jagle. <figure> <img src="./training_curve_avg.svg" alt="" style="width: 80%"> <figcaption>Training curve on Avg, JA Avg, and EN Avg.</figcaption> </figure> The results show that using Jagle significantly improves performance on Japanese tasks. ## Category distribution Below, we present the category-wise distribution of Jagle. <figure> <img src="./piechart_4metrics.svg" alt="" style="width: 80%"> <figcaption>Category distribution of Jagle.</figcaption> </figure> ## Usage Due to copyright restrictions, the Jagle dataset is hosted only on a domestic server and is not included in this mirror. To use Jagle, first download the dataset from: 👉 https://gitlab.llm-jp.nii.ac.jp/datasets/jagle ```bash git clone https://gitlab.llm-jp.nii.ac.jp/datasets/jagle.git mv jagle/data . ``` After placing the dataset directory locally (e.g., in a folder named data/Jagle), you can load each dataset using the 🤗 datasets library: ```python from datasets import load_dataset ds = load_dataset("data/Jagle/japanese-photos-VQA", split="train") print(ds) ``` ## License Jagle is provided for use within the scope permitted under Article 30-4 of the Japanese Copyright Act. The copyrights and related rights to the original source datasets remain with their respective rights holders. | Source Dataset | License | |---|---| | [WAON](https://huggingface.co/datasets/llm-jp/waon) | Apache 2.0 (use limited to information analysis under Article 30-4 of the Japanese Copyright Act) | | [japanese-photos](https://huggingface.co/datasets/ThePioneer/japanese-photos) | CC0 1.0 | | [Wikipedia-ja](https://ja.wikipedia.org/) | CC BY-SA 4.0 | | [FinePDFs-Edu](https://huggingface.co/datasets/HuggingFaceFW/finepdfs-edu) | ODC-BY | | [PlotQA](https://github.com/NiteshMethani/PlotQA) | CC BY 4.0 | | [TAT-QA](https://nextplusplus.github.io/TAT-QA/) | CC BY 4.0 | | [NDL WARP PDF](https://gitlab.llm-jp.nii.ac.jp/datasets/ndl-warp-pdf) | (use limited to information analysis under Article 30-4 of the Japanese Copyright Act) | | [e-Gov](https://www.e-gov.go.jp/) | Government Standard Terms of Use 2.0 | | [JSSODa](https://huggingface.co/datasets/llm-jp/jssoda) | CC BY 4.0 | | [Nemotron-VLM-Dataset-v2](https://huggingface.co/datasets/nvidia/Nemotron-VLM-Dataset-v2) | CC BY 4.0 | ### Models Used for QA Generation and Preprocessing | Model | License | |---|---| | [Qwen3-VL-235B-A22B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-235B-A22B-Instruct) | Apache 2.0 | | [llm-jp-3-13b-instruct](https://huggingface.co/llm-jp/llm-jp-3-13b-instruct) | Apache 2.0 | | [PaddleOCR-VL](https://github.com/PaddlePaddle/PaddleOCR) | Apache 2.0 | ## Citation ```bibtex @misc{sugiura2026jaglebuildinglargescalejapanese, title={Jagle: Building a Large-Scale Japanese Multimodal Post-Training Dataset for Vision-Language Models}, author={Issa Sugiura and Keito Sasagawa and Keisuke Nakao and Koki Maeda and Ziqi Yin and Zhishen Yang and Shuhei Kurita and Yusuke Oda and Ryoko Tokuhisa and Daisuke Kawahara and Naoaki Okazaki}, year={2026}, eprint={2604.02048}, archivePrefix={arXiv}, primaryClass={cs.CV}, url={https://arxiv.org/abs/2604.02048}, } ```

Jagle is a large-scale Japanese multimodal post-training dataset, comprising approximately 9.2 million instances across diverse tasks. Jagle was used to train LLM-jp-4-VL 9B beta. It is constructed by collecting a wide range of source data, including image–text pair datasets such as WAON and PDF corpora like FinePDFs-Edu, and then generating VQA data through various methods such as VLM-based QA generation and translation.

提供机构:
llm-jp
搜集汇总
数据集介绍
llm-jp/Jagle 数据集图片
构建方式
围绕日文视觉语言模型后训练资源匮乏的现实需求,Jagle 的构建采取多源汇聚与模型辅助生成相结合的路径。其原始材料涵盖 WAON 等图像—文本对、FinePDFs-Edu 等 PDF 语料,以及 Wikipedia-ja、PlotQA、TAT-QA、NDL WARP PDF、e-Gov、JSSODa、Nemotron-VLM-Dataset-v2 等多样化来源。在数据加工阶段,流程借助 Qwen3-VL-235B-A22B-Instruct、llm-jp-3-13b-instruct 与 PaddleOCR-VL 等模型完成视觉问答生成、翻译与预处理,最终形成约 920 万条实例,服务于视觉语言模型的后训练环节。
使用方法
面向受版权约束且需合规使用的日文多模态研究场景,Jagle 未在 Hugging Face 镜像中直接分发,而是托管于日本国内服务器。使用者需先从 https://gitlab.llm-jp.nii.ac.jp/datasets/jagle 克隆仓库,并将 jagle/data 移动至本地数据目录。完成本地放置后,可借助 Hugging Face datasets 库按子集加载,例如 load_dataset('data/Jagle/japanese-photos-VQA', split='train')。调用时应遵守日本著作权法第 30-4 条的信息分析限制,并同时尊重各源数据集的许可条款。
背景与挑战
背景概述
在视觉语言模型迅猛演进之际,日语多模态资源长期处于相对稀缺状态。Jagle于2026年由Sugiura Issa、Sasagawa Keito、Nakao Keisuke、Maeda Koki等研究者提出,并依托LLM-jp项目生态构建,旨在回应日语视觉语言模型后训练数据不足的核心问题。该数据集汇集约920万条实例,覆盖图文对、PDF文档、图表问答与翻译生成等多类任务,并用于训练LLM-jp-4-VL 9B beta。其构建整合WAON、FinePDFs-Edu等来源,通过VLM问答生成与翻译扩展日语VQA规模。实验表明,使用Jagle可显著提升模型在日语任务上的平均表现,为日语多模态理解与生成研究提供了重要基础设施,并对低资源语言的多模态后训练具有示范意义。
当前挑战
面向日语视觉语言模型的后训练,数据层面的核心矛盾在于规模、多样性与合规性之间的复杂张力。Jagle所解决的领域问题,是日语多模态指令与问答数据匮乏,难以支撑模型对日本文化语境、文档图表和视觉推理的深入理解。构建过程中,研究团队需应对多源异构数据的清洗、OCR解析、机器翻译与VLM生成问答的质量控制,避免噪声与语义漂移;同时,源数据涉及不同许可证与日本著作权法第30-4条限制,数据集仅能在国内服务器托管并使用。约920万实例的类别平衡、版权合规、生成内容可靠性与日语表达准确性,构成持续且艰巨的工程与伦理挑战。
常用场景
经典使用场景
在日语视觉语言模型的后训练范式下,Jagle以约九百二十万条多模态实例承担指令微调与跨模态对齐的核心角色。典型使用是加载japanese-photos-VQA等子集,以图像、问题与答案三元组开展监督微调,使模型习得日语视觉问答、图像描述、文档图表理解及跨语言迁移能力。研究常将Jagle与WAON、FinePDFs-Edu等源语料混合,进行多任务后训练,并在日语与英语任务平均分上检验模型表现。受日本著作权法第三十条之四约束,使用多在本地或合规环境加载,并借datasets库按子集划分训练与验证。
解决学术问题
日语多模态研究长期受制于高质量后训练语料匮乏与任务覆盖狭窄,Jagle针对此瓶颈提供规模化、任务多样的视觉问答资源。它缓解日语视觉语言模型在图像问答、文档理解、图表推理与跨语言迁移中的性能不足,并支持以统一后训练框架检验多模态对齐、指令遵循和知识注入等议题。其构建流程融合视觉语言模型生成问答与翻译增强,为低资源语言多模态数据合成提供可复现范式,也推动在版权合规前提下开展日语文档与照片语义分析研究。基于Jagle训练的LLM-jp-4-VL 9B beta显示日语任务提升,为日语视觉语言模型基准与模型比较提供实证基础。
实际应用
面向产业与公共部门的日语多模态智能需求,Jagle可用于训练和评测视觉语言模型,支撑图像问答、文档信息抽取、图表理解、教育辅助与政务PDF语义检索等应用。模型能够读取日文照片、PDF语料和图表,回答自然语言问题,协助信息分析。由于数据托管于日本国内服务器且受著作权限制,实际部署常限于合规的信息分析场景,如企业内部文档理解、公共资料问答与学术研究。借助约九百二十万实例的多任务混合,开发者可构建面向日语用户的视觉语言助手,并利用日英任务指标评估跨语言稳健性。
数据集最近研究
最新研究方向
在视觉语言模型向多语言、多模态纵深演进的背景下,Jagle以约920万日语后训练实例弥补日语视觉语言资源的结构性缺口。近期研究聚焦异质语料驱动的合成VQA规模化,利用Qwen3-VL-235B-A22B-Instruct等模型从WAON、FinePDFs-Edu生成与翻译问答,探索文档视觉问答、图表推理、行政文本理解和日语OCR等任务中的跨模态对齐。借LLM-jp-4-VL 9B beta的验证,该数据显著提升日语任务表现,并呼应日本《著作权法》第30-4条信息分析例外及生成式AI版权争议,在合规托管、许可证治理和本地化多模态评测方面形成实践样本,为日语视觉语言模型、文档智能与公共数字化服务提供了关键数据基础设施。
以上内容由遇见数据集搜集并总结生成
二维码
社区交流群
二维码
科研交流群
商业服务