ORKG Properties and LLM-Generated Research Dimensions Evaluation Dataset

This dataset contains a collection of 103 research comparisons from the Open Research Knowledge Graph (ORKG) with annotated properties and corresponding research dimensions generated by three different Large Language Models (LLMs). The dataset includes 1,317 papers from 35 diverse research fields, addressing 153 distinct research problems. Each paper is associated with human-annotated ORKG properties, as well as research dimensions generated by GPT-3.5, Llama 2, and Mistral LLMs. The dataset provides a comprehensive evaluation benchmark for comparing the performance of different LLMs in generating research dimensions that align with human-annotated properties.

## Dataset columns:
* comparison_id: Unique identifier of the research comparison in the Open Research Knowledge Graph (ORKG)
* contribution_id: Identifier of the individual research contribution (paper) within a comparison
* paper_id: Unique identifier of the research paper
* paper_title: Title of the research paper
* research_field: Field of research associated with the paper
* research_problem: Specific research problem addressed by the paper
* orkg_properties: Human-annotated properties of the paper in the ORKG, representing specific attributes or characteristics of the research contribution
* gpt_dimensions: Research dimensions generated by the GPT Large Language Model (LLM) for the paper
* mistral_dimensions: Research dimensions generated by the Mistral LLM for the paper
* llama2_dimensions: Research dimensions generated by the Llama2 LLM for the paper
* mappings: Mapping of ORKG properties to LLM-generated research dimensions
* alignments: Alignment scores between ORKG properties and LLM-generated research dimensions
* deviations: Deviation scores between ORKG properties and LLM-generated research dimensions
* orkg_gpt_similarity: Cosine similarity score between the embeddings of ORKG properties and GPT-generated research dimensions
* orkg_llama2_similarity: Cosine similarity score between the embeddings of ORKG properties and Llama2-generated research dimensions
* orkg_mistral_similarity: Cosine similarity score between the embeddings of ORKG properties and Mistral-generated research dimensions
* gpt_llama2_similarity: Cosine similarity score between the embeddings of GPT-generated and Llama2-generated research dimensions
* gpt_mistral_similarity: Cosine similarity score between the embeddings of GPT-generated and Mistral-generated research dimensions
* llama2_mistral_similarity: Cosine similarity score between the embeddings of Llama2-generated and Mistral-generated research dimensions

本数据集收录了来自开放研究知识图谱（Open Research Knowledge Graph，ORKG）的103项研究对比数据，包含标注属性以及由三款不同大语言模型（Large Language Model，LLM）生成的对应研究维度。该数据集涵盖来自35个不同研究领域的1317篇学术论文，涉及153个明确的研究问题。每篇论文均关联有人类标注的ORKG属性，同时附带由GPT-3.5、Llama 2与Mistral三款大语言模型生成的研究维度。本数据集为对比不同大语言模型生成贴合人类标注属性的研究维度的性能，提供了一套全面的评估基准。

## 数据集字段：
* 对比唯一标识符（comparison_id）：开放研究知识图谱（Open Research Knowledge Graph，ORKG）内研究对比的唯一标识
* 研究贡献标识符（contribution_id）：某一对比中对应单篇学术研究贡献的标识符
* 论文唯一标识符（paper_id）：学术论文的唯一标识
* 论文标题（paper_title）：学术论文的标题
* 研究领域（research_field）：该学术论文所属的研究领域
* 研究问题（research_problem）：该学术论文所针对的具体研究问题
* ORKG标注属性（orkg_properties）：人类在开放研究知识图谱中为该研究贡献标注的属性，代表该研究贡献的特定属性或特征
* GPT生成研究维度（gpt_dimensions）：由GPT大语言模型为该论文生成的研究维度
* Mistral生成研究维度（mistral_dimensions）：由Mistral大语言模型为该论文生成的研究维度
* Llama2生成研究维度（llama2_dimensions）：由Llama2大语言模型为该论文生成的研究维度
* 映射关系（mappings）：ORKG标注属性与大语言模型生成的研究维度之间的对应映射
* 对齐得分（alignments）：ORKG标注属性与大语言模型生成的研究维度之间的对齐评分
* 偏差得分（deviations）：ORKG标注属性与大语言模型生成的研究维度之间的偏差评分
* ORKG与GPT相似度（orkg_gpt_similarity）：ORKG标注属性与GPT生成的研究维度的嵌入向量之间的余弦相似度评分
* ORKG与Llama2相似度（orkg_llama2_similarity）：ORKG标注属性与Llama2生成的研究维度的嵌入向量之间的余弦相似度评分
* ORKG与Mistral相似度（orkg_mistral_similarity）：ORKG标注属性与Mistral生成的研究维度的嵌入向量之间的余弦相似度评分
* GPT与Llama2相似度（gpt_llama2_similarity）：GPT生成的研究维度与Llama2生成的研究维度的嵌入向量之间的余弦相似度评分
* GPT与Mistral相似度（gpt_mistral_similarity）：GPT生成的研究维度与Mistral生成的研究维度的嵌入向量之间的余弦相似度评分
* Llama2与Mistral相似度（llama2_mistral_similarity）：Llama2生成的研究维度与Mistral生成的研究维度的嵌入向量之间的余弦相似度评分