SciGraphQA Dataset

SciGraphQA is a large-scale, open-domain dataset focused on generating multi-turn conversational question-answering dialogues centered around understanding and describing scientific graphs and figures. It contains over 300,000 samples derived from academic research papers in computer science and machine learning domains.

Each sample in ScFiGraphQA consists of a scientific graph image sourced from papers on ArXiv, accompanied by rich textual context including the paper's title, abstract, figure caption, and a paragraph from the paper referencing the figure. Using this comprehensive context, the dataset employs a to produce multi-turn question-answer dialogues aimed at explaining the given graph in an interactive, conversational format. On average, each sample contains 2-3 turns of question-answer exchange.

The key motivation behind SciGraphQA is providing a large-scale resource to support research and development of multi-modal AI systems that can engage in informative, open-ended conversations about graphs and data visualizations. The multi-turn dialogue format presents a more natural and interactive setting compared to standard visual question answering datasets that use fixed sets of standalone questions.

Potential use cases of SciGraphQA include pre-training and benchmarking multi-modal conversational models for scientific graph comprehension, building AI assistants that can discuss data insights, and developing aids to help individuals understand complex figures and diagrams interactively. The academic source material also provides a way to evaluate model capabilities on expert-level graphs spanning diverse topics and complex visual encodings.

SciGraphQA 是一个大规模、开放域数据集，专注于生成围绕理解和描述科学图表与图形的多轮对话式问答对话。该数据集包含超过 30 万个样本，源自计算机科学和机器学习领域的学术研究论文。ScFiGraphQA 中的每个样本都包含一幅来自 ArXiv 论文的科学图表图像，并附带丰富的文本背景，包括论文标题、摘要、图注以及论文中引用该图的段落。利用这一全面背景，该数据集旨在生成多轮问答对话，以交互式、对话式的形式解释所提供的图表。平均而言，每个样本包含 2-3 轮的问答交流。

SciGraphQA 的核心动机在于提供一个大规模资源，以支持多模态 AI 系统的研究与开发，使其能够参与关于图表和数据可视化的信息丰富、开放式对话。多轮对话格式相较于使用固定独立问题集的标准视觉问答数据集，提供了一个更加自然和互动的环境。

SciGraphQA 的潜在应用案例包括：对多模态对话模型进行预训练和基准测试，以理解科学图表；构建能够讨论数据洞察的 AI 助手；以及开发辅助工具，帮助个人以交互式方式理解复杂的图表和图形。学术源材料还提供了一种评估模型在涵盖多样主题和复杂视觉编码的专家级图表上的能力的方法。