AI4Chem/ChemBench4K

chembench是一个包含了九项化学核心任务，4100个高质量单选问答的大语言模型化学能力评测基准。

原始信息汇总

数据集概述

数据集名称

chembench

数据集描述

chembench是一个大语言模型化学能力评测基准，包含了九项化学核心任务和4100个高质量单选问答。

数据集内容

• 任务类型：化学核心任务
• 问题数量：4100个
• 问题类型：单选问答

示例内容

json { "question": "Create a molecule that satisfies the conditions outlined in the description. The molecule is a natural product found in Cinnamomum camphora, Otoba parvifolia, and Zanthoxylum wutaiense with data available.", "answer": "C", "A": "In my opinion, the answer would be C12C3(O)CCC(C4=CC(=O)OC4)C3(C(O)CC1C1(C)CCC(OC3OC(C)C(C(C3)O)OC3CC(C(C(O3)C)OC3CC(C(C(O3)C)O)O)O)CC1CC2)C", "B": "In my opinion, the answer would be c1c(c(=O)c2ccc(O)cc2o1)-c1cc(c(c(O)c1O)OC)C(C)(C=C)C", "C": "From my perspective, the answer is O1C(C2C(C1)C(c1cc3c(OCO3)cc1)OC2)c1ccc(c(c1)OC)O", "D": "I would conclude that c1(cc(oc(c1C)=O)C=CC(C)=CC(C)=Cc1ccccc1)OC" }

许可证

MIT

搜集汇总

背景与挑战

背景概述

The 'AI4Chem/ChemBench4K' dataset is a comprehensive benchmark for evaluating language models' chemistry competency, featuring 4,100 single-choice questions across nine core chemistry tasks, formatted in JSON for easy integration into model evaluation pipelines.