UniParser/RxnBench-Doc

RxnBench-Doc是一个文档级问答(DocQA)基准，包含540个多项选择题，旨在评估博士级别的有机化学反应在文本和多模态上下文中的理解能力。所有问题都经过多轮专家评审，以确保清晰性、一致性和科学严谨性。RxnBench-Doc发布了英文和中文两个版本。基准涵盖两种关键任务类型：上下文推理，需要整合来自多模态源（如反应图像、表格和文本）的信息以回答问题；以及结构推理，专注于与分子和Markush结构、反应组分和机制推理相关的问题。评估方法包括将PDF格式文档渲染为144 dpi的图像序列，并将其作为多模态大语言模型(MLLM)的输入，通过模型输出提取最终的多项选择答案并与标准答案进行比较。

RxnBench (FD-QA) is a document-level question answering (DocQA) benchmark comprising 540 multiple-select questions designed to assess PhD-level understanding of organic chemistry reactions in textual and multimodal contexts. All questions underwent multiple rounds of expert review to ensure clarity, consistency, and scientific rigor. RxnBench-Doc is released in both English and Chinese versions. The benchmark covers two key task types: Context Reasoning, which requires integrating information from multimodal sources such as reaction images, tables, and text to answer questions; and Structure Reasoning, which focuses on reasoning questions related to molecular and Markush structures, reaction components, and mechanistic inference. The evaluation involves rendering each PDF format document into images sequence at 144 dpi and using them as inputs to the MLLM, extracting the final multiple-select answer from the model’s raw output and comparing it against the ground truth.