SakanaAI/KamonBench

KamonBench是一个基于语法的图像到结构基准数据集，专门用于评估视觉语言模型中的组合因子恢复能力。该数据集围绕日本家纹（kamon）构建，包含合成家纹图像，每个家纹都配对了多种结构化信息：正式的家纹描述语言字符串（Kamon Description Language, KDL）、分段的日语分析、英语翻译以及基于三个生成因子（容器C、修饰符R、图案M）的非语言程序代码。数据集支持直接因子指标评估、受控因子对重组分割、在固定上下文下的反事实图案敏感性测试，以及从冻结表示中提取因子的线性探针。数据集包含54,116张PNG图像，分为合成家纹（20,000张）、基础图案组件（20,000张）和容器组件（14,116张），并划分为训练集（43,280条记录）、开发集（5,405条记录）和测试集（5,431条记录）。此外，还提供了多个Croissant元数据文件，用于不同的实验设置，如因子组合保留分割。数据集主要用于组合视觉识别、因子感知评估和表示分析的研究，但非权威文化或历史家纹目录，且图像为合成渲染，风格与传统家纹有所不同。数据集遵循CC BY-NC 4.0许可，仅用于非商业研究目的。KamonBench is a grammar-based image-to-structure benchmark dataset specifically designed to evaluate the compositional factor recovery capability of vision-language models. Built around Japanese kamon, it contains synthetic kamon images, each paired with multiple types of structured information: formal Kamon Description Language (KDL) strings, segmented Japanese linguistic analyses, English translations, and non-linguistic procedural code based on three generative factors: Container (C), Modifier (R), and Pattern (M). The dataset supports direct factor metric evaluation, controlled factor-pair recombination splits, counterfactual pattern sensitivity tests under fixed contexts, and linear probing for factor extraction from frozen representations. It contains 54,116 PNG images, categorized into three groups: synthetic kamon (20,000 samples), basic pattern components (20,000 samples), and container components (14,116 samples), and is split into training set (43,280 records), development set (5,405 records), and test set (5,431 records). Additionally, multiple Croissant metadata files are provided for different experimental setups, such as factor combination-preserving splits. The dataset is primarily intended for research on compositional visual recognition, factor-aware evaluation, and representation analysis, but it is not an authoritative cultural or historical kamon catalog. Moreover, the synthetically rendered images differ stylistically from traditional kamon. The dataset is licensed under CC BY-NC 4.0 and is solely for non-commercial research use.