SciCUEval: A Comprehensive Dataset for Evaluating Scientific Context Understanding in Large Language Models

SciCUEval is a benchmark designed to evaluate context understanding capabilities of Large Language Models (LLMs) in scientific domains. It addresses the lack of specialized benchmarks for scientific domains by providing:

10 Diverse Datasets: Spanning biology, chemistry, physics, biomedicine, and materials science.

Multiple Data Modalities: Structured tables, knowledge graphs, and unstructured text.

Four Core Competencies: Relevant information identification, Information-absence detection, Multi-source information integration, and Context-aware inference

Comprehensive Evaluation: Assessing state-of-the-art LLMs across various scientific tasks.

SciCUEval 是一款用于评估大语言模型（Large Language Models，LLMs）在科学领域上下文理解能力的基准测试集。针对当前科学领域专用基准测试集匮乏的现状，该测试集提供了以下资源：
1. 十大多样化数据集：覆盖生物学、化学、物理学、生物医学与材料科学等学科领域。
2. 多模态数据类型：涵盖结构化表格、知识图谱及非结构化文本。
3. 四项核心考核能力：相关信息识别、信息缺失检测、多源信息融合以及上下文感知推理。
4. 全面性能评估：对当前顶尖大语言模型在各类科学任务中的表现开展测评。