Supplementary Package - Socratic User Study & Tool

This is the data analysis done for the paper titled "Socratic LLM Framework (Extended Abstract): Fostering Critical Thinking in Software Engineering Students" (submitted to FSE - Software Engineering Education track). Abstract -- Large language models continue to attract the attention of the software industry due to their potential to boost developers' productivity. However, this focus often sidelines the development of critical thinking skills, which are essential for novice developers. In university settings, educators seek to understand how to make students properly engage with these tools, mitigating the impact on their skills' growth. Our goal is to develop a tool that stimulates novice software developers to think critically rather than simply copying and pasting code. We propose the Socratic LLM framework focused on code generation, code explanation, and debugging. Our framework and system are designed to benefit novice developers and SE educators by primarily encouraging critical thinking and independent problem-solving, as well as providing personalised guidance to students. This repository contains: Qualtrics surveys: pre_task_survey.pdf and post_task_survey.pdf Task instruction: User Study Instructions.pdf Socratic LLM tool User interface: UI_dashboard.png UI_chatbox.png Socratic LLM Tool (Code): SocraticLLM.zip