Data for Analysis: GUI, Robot, Digital Twin - Drawing Tasks and User Evaluation.

In this study, 69 first-year undergraduate Computer Science students were assigned to three groups of 23 participants, corresponding to the three experimental conditions. The study was conducted at the end of a 24-week academic year after students had successfully completed their first year of study and the same introductory programming curriculum. Participation in the study was voluntary and all responses were anonymised prior to analysis.<br>To reduce performance imbalance between groups, participant allocation was performed randomly within existing performance bands derived from prior assessment data. Students were assigned anonymised numerical identifiers, which were then randomly distributed across the three groups using software for random allocation. Participants were not informed of the study hypothesis, and all groups received equivalent instructional support throughout the activity.<br>Time on task was not controlled across conditions due to inherent differences between the programming environments. In particular, the physical robot condition required additional setup, positioning, calibration, and hardware execution time, while both the physical robot and Digital Twin conditions required program execution through motor-control sequences. In contrast, the GUI-only condition allowed substantially faster execution and iteration without physical or simulated movement constraints. Consequently, direct comparison of task completion time across conditions was not considered methodologically reliable. \\During Year 1 of undergraduate Computer Science, students are evaluated through a competency-based approach to teaching. These competencies are measured by being threshold, typical, and excellent. To pass the year, students would need to be observed on 36 threshold competencies. However, students gain additional credits by getting as many typical and excellent credits as possible. The total amount of competencies is 136.<br>For the purposes of our study, the 69 students are separated into uniform groups, meaning that each group has an equivalent proportion of low, medium and high performing students in terms of observed competencies. All of the students are studying an undergraduate Computer Science program full-time; 11 of the students were female, 58 male. All students had previous experience with the hardware robotic platform, with the GUI, but no experience with the Digital Twin.<br>For the assessment of the results, all student submissions were evaluated based on the observable correctness of the produced outputs. In the GUI-only condition, programs were assessed by visually inspecting whether the expected graphical output was correctly displayed within the programming environment. In the physical robot condition, assessment was based on whether the robot successfully produced the required drawing or movement pattern in the physical environment. In the Digital Twin condition, correctness was evaluated using the line-tracing representation generated by the simulated robot, allowing the produced paths and shapes to be visually inspected and compared against the expected task outcomes. This approach ensured that assessment across all three conditions focused consistently on the observable behaviour generated by student programs.Solutions were classified as correct only when all required task behaviours and expected output shapes were successfully produced.