Japanese Heron-Bench

Japanese Heron-Bench是一个专为评估日语视觉语言模型(VLMs)设计的基准数据集。该数据集由图灵公司创建，包含102个独特的图像-问题-答案对，这些内容均针对日本文化背景定制。数据集的创建过程涉及收集与日本相关的公共领域或CC BY 2.0许可的图像，并为每个图像设置三个类别：对话、细节和复杂，每个类别包含一到两个问题。数据集的应用领域主要在于评估VLMs在理解和回答日语环境下的视觉场景问题方面的能力，旨在解决当前VLMs在非英语语言环境中评估不足的问题。

Japanese Heron-Bench is a benchmark dataset specifically designed for evaluating Japanese visual language models (VLMs). Developed by Turing Inc., it contains 102 unique image-question-answer pairs, all customized for the Japanese cultural context. The dataset construction process involves collecting Japan-related public domain or CC BY 2.0 licensed images, and establishing three categories of questions for each image: Conversation, Detail, and Complex, with one or two questions under each category. Its main application is to evaluate the ability of VLMs to understand and answer visual scene questions in Japanese contexts, aiming to address the current insufficient evaluation of VLMs in non-English language environments.