Open-Ended Scrum Framework Questions in a Software Engineering Course

This dataset is used for the automated assessment of open-ended exam questions in the online course Introduction to Software Engineering at Constantine the Philosopher University in Nitra. The dataset originates from the Moodle Learning Management System (LMS) and includes responses to eight open-ended questions centered on fundamental terminology related to the Scrum framework, a key methodology in agile software development. The dataset comprises 528 responses from 66 university students, alongside 800 responses generated by ten Large Language Models (LLMs) under controlled conditions. AI-generated responses were compared against students and reference answers extracted from course study materials, which were designed to serve as benchmarks for evaluating the quality and accuracy of both student and AI-generated responses. The open-ended exam questions included: a) Explain the term sprint. b) Explain the term product backlog. c) Explain the term sprint backlog. d) Explain the term user story. e) What phases does a software product go through in development? f) How are these phases implemented in Scrum? g) What do you see as the advantages of using Scrum? h) What do you see as the disadvantages of using Scrum? To collect AI-generated data, the questions were combined with five distinct introduction prompts and input into each LLM across two iterations per session. The prompts simulated various student input styles, such as "Provide the most accurate answers" and "Write correct answers." This process resulted in a dataset of 800 AI-generated entries for analysis.  The dataset, gathered in autumn 2024, offers a comprehensive resource for studying the accuracy, consistency, and application of LLMs in educational settings. These findings contribute to improving the automated assessment of open-ended student responses in blended learning environments.