Enhancing Proof Assistant Error Messages with Hints: A User Study

This repository contains the user study data accompanying the master thesis by Maria Khakimova titled "Enhancing Proof Assistant Error Messages with Hints: A User Study". The goal of the research was to investigate the impacts of hint-based error message enhancements in Agda on novice programmers. To do this, we enhanced three error messages with hints, and conducted a user study.<br>In the user study, we asked participants to resolve errors in pre-written Agda code, and rate the helpfulness of the error message. We collected the following data:code compilation status (success/fail),compilation timestamps, andresponses to the "Did you find the error message helpful?" question (on a Likert scale).<br>This repository contains the programming questions created for the user study, with the accompanying error messages (both original and enhanced) in <code>programming_exercises.zip</code>. We also provide the (anonymised) collected data in <code>JSON</code> format in <code>response-data.json</code>.<br>For more details, please read the provided <code>README</code>.

本代码仓库包含玛丽亚·哈基莫娃（Maria Khakimova）题为"基于提示优化证明助手错误信息：一项用户研究"的硕士学位论文的配套用户研究数据。本研究旨在探究Agda环境下基于提示的错误信息优化对新手程序员的影响。为此，我们为三条错误信息添加了提示内容，并开展了一项用户研究。
在本次用户研究中，我们要求参与者修复预先编写的Agda代码中的错误，并对错误信息的有用性进行评分。我们采集了以下数据：代码编译状态（成功/失败）、编译时间戳，以及针对"您认为本次错误信息是否有帮助？"这一问题的李克特量表（Likert scale）作答结果。
本仓库同时包含为本次用户研究设计的编程练习题，配套的错误信息（含原始版本与优化版本）存放于<code>programming_exercises.zip</code>压缩包中。此外，我们将经过匿名化处理的采集数据以JSON格式存储于<code>response-data.json</code>文件内。
如需了解更多细节，请参阅附带的<code>README</code>文件。