SakanaAI/KamonBench

KamonBench是一个基于语法的图像到结构基准数据集，专门用于评估视觉语言模型中的组合因子恢复能力。该数据集围绕日本家纹（kamon）构建，包含合成家纹图像，每个家纹都配对了多种结构化信息：正式的家纹描述语言字符串（Kamon Description Language, KDL）、分段的日语分析、英语翻译以及基于三个生成因子（容器C、修饰符R、图案M）的非语言程序代码。数据集支持直接因子指标评估、受控因子对重组分割、在固定上下文下的反事实图案敏感性测试，以及从冻结表示中提取因子的线性探针。数据集包含54,116张PNG图像，分为合成家纹（20,000张）、基础图案组件（20,000张）和容器组件（14,116张），并划分为训练集（43,280条记录）、开发集（5,405条记录）和测试集（5,431条记录）。此外，还提供了多个Croissant元数据文件，用于不同的实验设置，如因子组合保留分割。数据集主要用于组合视觉识别、因子感知评估和表示分析的研究，但非权威文化或历史家纹目录，且图像为合成渲染，风格与传统家纹有所不同。数据集遵循CC BY-NC 4.0许可，仅用于非商业研究目的。

KamonBench is a grammar-based image-to-structure benchmark for evaluating compositional factor recovery in vision-language models, built around Japanese family crests (kamon). Each composite crest is paired with a formal kamon description language string (KDL), a segmented Japanese analysis, an English translation, and a non-linguistic program code over the generator factors (container C, modifier R, motif M). The dataset supports direct factor metrics, controlled factor-pair recombination splits, counterfactual motif-sensitivity tests under fixed contexts, and linear probes of factor accessibility from frozen representations. It includes 54,116 PNG images, comprising composite crests (20,000), base-motif components (20,000), and container components (14,116), with splits into train (43,280 records), dev (5,405 records), and test (5,431 records). Multiple Croissant metadata files are provided for various experimental setups, such as held-out factor combinations. KamonBench is intended for research in compositional visual recognition, factor-aware evaluation, and representation analysis, but is not an authoritative cultural or historical kamon catalogue; the crests are synthetically rendered and differ in style from traditional ones. The dataset is released under CC BY-NC 4.0 for non-commercial research use.