KCs4EDU

Welcome to the dataset of Evaluating Large Language Models with Educational Knowledge Graphs: Challenges with Prerequisite Relationships and Multi-Hop Reasoning. Large Language Models (LLMs) have transformed educational technology, enhancing personalized learning and curriculum delivery. However, their capabilities in tasks such as relationship verification, multi-hop reasoning, and relationship extraction within educational knowledge graphs (KGs) are under-explored. These tasks are crucial for developing intelligent educational systems that accurately model and navigate structured knowledge components (KCs) and their prerequisite relationships. This paper presents a comprehensive evaluation of pre-trained LLMs on these complex tasks. We assess their performance in one-hop relationship verification, multi-hop chain reasoning, and relationship extraction across diverse datasets. Our findings highlight significant variability in LLM performance, particularly as task complexity increases. While models like GPT-4 and Claude-3 show robustness in simpler tasks, their accuracy declines with more complex multi-hop reasoning. Our contributions include an evaluation framework for educational tasks, detailed analyses of LLM performance, and recommendations for enhancing model training and application. This work advances our understanding of LLM capabilities in education and provides practical insights for their effective integration, aiming to improve learning efficiency and accessibility.