EduMKG: A Multimodal Knowledge Graph for Education with Text, Video, Image and Audio

EduMKG is a multimodal educational knowledge graph dataset that covers natural sciences (biology, physics, and chemistry) in middle and high school education. It includes multimodal concepts covering text, images, videos, and audio, as well as knowledge points and exercises extracted from curriculum standards and MOOCs. EduMKG comprises 34,630 multimodal concepts and 403,400 triples, making it a vital resource for research in multimodal educational applications.Zip DescriptionRawData: This folder comprises the publicly available raw data, including images derived from course materials, subtitles extracted from MOOC videos, audio files associated with specific concepts, and supplementary files generated through multimodal concept extraction and alignment. EduMKG: This folder contains the entire EduMKG, including the triple files of EduMKG as well as the cross-indexing files between multimodal concepts, knowledge points, and imagesExample: The examples of the Case Study for Multimodal Alignment Section in the paper.Groundtruth: The groundtruth for Evaluation on Concept Extraction Section in the paper. Document Description Documents in EduMKG {subject}Image.json: This is a cross-indexing file between images and concepts in the multimodal discipline. Each image has a unique ID and is associated with a varying number of concepts.{subject}Knowledge.json: This is a cross-reference file for knowledge points. Each knowledge point has a unique ID, along with URLs for corresponding exercises and IDs of related knowledge points. {subject}.json: This is a multimodal concept cross-indexing file involving text, images, videos, audio, and knowledge points. Each multimodal concept has a unique ID.{subjectTriples}.json: This file contains all the triples of EduMKG. Documents in Example Each folder contains text, image, video, and audio knowledge of multimodal concepts. To avoid copyright issues, we only provide video URLs and the time intervals where the concepts appear. Documents in Groundtruth Each `groundtruth.txt` contains the concepts corresponding to the course with the same name as the folder. The contents were manually annotated by three PhD students and seven master's students, with cross-validation performed to ensure accuracy. The `groundtruth.txt` files are used in the "Evaluation on Concept Extraction" section of the paper to validate the effectiveness of the proposed concept extraction method. Documents in RawDataIncludes images corresponding to multimodal concepts, as well as intermediate files generated during the construction of the knowledge graph, facilitating the reproducibility of the EduMKG construction process.EduMKG是一款多模态教育知识图谱（multimodal educational knowledge graph）数据集，覆盖初高中教育阶段的自然科学领域，包含生物学、物理学与化学三门学科。该数据集涵盖文本、图像、视频及音频四类多模态概念，同时包含从课程标准与慕课（MOOC）中提取的知识点与习题。EduMKG共包含34630个多模态概念与403400条三元组（triple），是支撑多模态教育应用研究的关键资源。 ### 压缩包文件说明 RawData文件夹：该文件夹包含公开可用的原始数据，包括源自课程素材的图像、从慕课视频中提取的字幕、与特定概念关联的音频文件，以及通过多模态概念提取与对齐流程生成的辅助文件。 EduMKG文件夹：该文件夹包含完整的EduMKG数据集，涵盖EduMKG的三元组文件，以及多模态概念、知识点与图像之间的交叉索引文件。 Example：对应论文中“多模态对齐章节案例研究”的示例文件。 Groundtruth：对应论文中“概念提取评估章节”的基准真值（Groundtruth）数据。 ### 数据集文档说明 #### EduMKG内置文档 {subject}Image.json：该文件为多模态学科内图像与概念的交叉索引文件。每张图像拥有唯一ID，并与数量不等的概念相关联。 {subject}Knowledge.json：该文件为知识点的交叉引用文件。每个知识点拥有唯一ID，同时附带对应习题的URL以及相关知识点的ID。 {subject}.json：该文件为涵盖文本、图像、视频、音频与知识点的多模态概念交叉索引文件。每个多模态概念拥有唯一ID。 {subjectTriples}.json：该文件包含EduMKG的全部三元组数据。 #### 示例文件夹内文档 每个文件夹包含多模态概念对应的文本、图像、视频及音频知识。为规避版权问题，本数据集仅提供视频URL以及概念出现的时间区间。 #### 基准真值文件夹内文档 每个`groundtruth.txt`文件包含与同名文件夹对应课程的关联概念。其内容由3名博士研究生与7名硕士研究生手动标注，并通过交叉验证确保数据准确性。此类`groundtruth.txt`文件将用于论文的“概念提取评估”章节，以验证所提出的概念提取方法的有效性。 #### 原始数据文件夹内文档 该文件夹包含与多模态概念对应的图像，以及知识图谱构建过程中生成的中间文件，便于复现EduMKG的完整构建流程。