Enhancing Proof Assistant Error Messages with Hints: A User Study

This repository contains the user study data accompanying the master thesis by Maria Khakimova titled "Enhancing Proof Assistant Error Messages with Hints: A User Study". The goal of the research was to investigate the impacts of hint-based error message enhancements in Agda on novice programmers. To do this, we enhanced three error messages with hints, and conducted a user study.<br>In the user study, we asked participants to resolve errors in pre-written Agda code, and rate the helpfulness of the error message. We collected the following data:code compilation status (success/fail),compilation timestamps, andresponses to the "Did you find the error message helpful?" question (on a Likert scale).<br>This repository contains the programming questions created for the user study, with the accompanying error messages (both original and enhanced) in <code>programming_exercises.zip</code>. We also provide the (anonymised) collected data in <code>JSON</code> format in <code>response-data.json</code>.<br>For more details, please read the provided <code>README</code>.