The Knowledge Component Attribution Problem for Programming: Methods and Tradeoffs with Limited Labeled Data

Understanding students’ learning of knowledge components (KCs) is an important educational data miningtask and enables many educational applications. However, in the domain of computing education,where program exercises require students to practice many KCs simultaneously, it is a challenge to attributetheir errors to specific KCs and, therefore, to model student knowledge of these KCs. In this paper,we define this task as the KC attribution problem. We first demonstrate a novel approach to addressing thistask using deep neural networks and explore its performance in identifying expert-defined KCs (RQ1).Because the labeling process takes costly expert resources, we further evaluate the effectiveness of transferlearning for KC attribution, using more easily acquired labels, such as problem correctness (RQ2).Finally, because prior research indicates the incorporation of educational theory in deep learning modelscould potentially enhance model performance, we investigated how to incorporate learning curves in themodel design and evaluated their performance (RQ3). Our results show that in a supervised learning scenario,we can use a deep learning model, code2vec, to attribute KCs with a relatively high performance(AUC > 75% in two of the three examined KCs). Further using transfer learning, we achieve reasonableperformance on the task without any costly expert labeling. However, the incorporation of learning curvesshows limited effectiveness in this task. Our research lays important groundwork for personalized feedbackfor students based on which KCs they applied correctly, as well as more interpretable and accuratestudent models.理解学生对知识组件（Knowledge Components，KCs）的学习情况是一项重要的教育数据挖掘任务，可支撑诸多教育应用场景。然而在计算教育领域，编程练习要求学生同时运用多个KCs，此时将学生的错误归因于具体的KCs并据此构建学生的知识掌握模型极具挑战。本文将该任务定义为KCs归因问题。 首先，我们提出了一种基于深度神经网络的全新方法以解决该任务，并探究其在识别专家定义的KCs上的性能表现（RQ1）。由于标注流程需耗费大量专家资源，我们进一步评估了迁移学习（Transfer Learning）应用于KCs归因任务的有效性，采用了更易获取的标注标签，例如习题作答正确性（RQ2）。最后，鉴于已有研究表明，在深度学习模型中融入教育理论有望提升模型性能，我们探究了如何在模型设计中融入学习曲线，并评估了其效果（RQ3）。 我们的研究结果表明，在监督学习场景下，可借助深度学习模型code2vec对KCs进行归因，且性能表现较为优异——在三个测试KCs中的两个上，受试者工作特征曲线下面积（Area Under the Receiver Operating Characteristic Curve，AUC）超过75%。进一步结合迁移学习后，即便无需耗费大量专家资源进行标注，我们也能在该任务上取得较为可观的性能表现。但学习曲线的融入在该任务中的效果较为有限。本研究为基于学生正确应用的KCs为其提供个性化反馈，以及构建更具可解释性与准确性的学生知识模型奠定了重要基础。