Evaluating Knowledge Tracing Models for Student Competency Assessment in Software Engineering Education

Knowledge Tracing (KT) tracks students' learning progress to adapt learning content and feedback accordingly. Hence, it is a crucial part of educational technologies, particularly intelligent tutoring systems (ITSs). Which KT model is most suitable depends on contextual factors, particularly the learning domain. While substantial prior work has covered KT in computer science education, few studies have provided open datasets and comprehensively compared KT models in the context of software engineering (SE) education. We provide a novel dataset from a first-semester programming course with about 600 students and compare a wide range of KT models to identify suitable candidate models for an ITS for SE education. Our results show that, among other models, an Elo-based model (M-Elo) has a good performance rate while also fulfilling most of our predefined requirements. Furthermore, our findings indicate limited success in predicting final exam performance. The results provide valuable insights into KT in SE education and highlight models that potentially perform well in ITSs for SE education.