Image2Test: Using ChatGPT to Build Manual Tests from Screenshots

The <b>both_test_forms.zip</b> file provides a summarized visual comparison of both test versions as images. The <b>screenshots.zip</b> archive contains the screenshots that were used as input for ChatGPT; these images have been slightly censored to preserve anonymity.<br><br><b>Background</b>: Website layouts often change with new design trends and front-end frameworks. Quality assurance is necessary during these changes, but manual testing takes much time and money. Manual tests are the standard way to maintain quality, but they are slow and expensive. Changes in graphical interfaces can cause errors or break features, which affects quality. Writing manual tests is the most time-consuming part of the process. Aims: This paper presents a tool that uses ChatGPT to create Natural Language Tests from screenshots and operator instructions. The goal is to reduce the time spent on manual test creation and to maintain quality in both the tests and the application. <b>Method</b>: We used two evaluation methods. First, we conducted a survey with 18 software testing professionals and students to compare tests made by ChatGPT and by humans. Second, we used Natural Language Processing techniques to measure the similarity between ChatGPT-generated tests and human-made tests. <b>Results</b>: The qualitative analysis showed that ChatGPT tests exceed human tests in completeness by a difference of 5.56%, achieving 36.11% acceptance rate. Human tests exceeded ChatGPT tests in clarity by 6.95%, reaching 41.67% acceptance rate. The quantitative analysis found that 66.7% of ChatGPT tests shared over 50% similarity with human tests. <b>Conclusions</b>: Our tool can help automate the creation of software tests. The similarity between AI-generated and human-made tests shows that this approach can save time and reduce costs, while keeping test quality at an acceptable level. This framework can help maintain quality during changes in website layouts and application development.

<b>both_test_forms.zip</b> 文件以图像形式提供了两种测试版本的汇总可视化对比。<b>screenshots.zip</b> 归档文件包含了用作 ChatGPT 输入的截图，为保护匿名性，这些图像已进行了轻度打码处理。

<b>背景</b>：网站布局往往会随着全新设计趋势与前端框架的更新而发生变化。在此类更新过程中，质量保障工作必不可少，但人工测试需耗费大量时间与资金。尽管人工测试是保障质量的常规手段，但其效率低下且成本高昂。图形界面的变更可能引发错误或功能故障，进而影响产品质量，而编写人工测试用例则是整个流程中最耗时的环节。

<b>研究目标</b>：本论文提出一款可基于截图与操作指令，借助ChatGPT生成自然语言测试用例的工具。其核心目标在于缩减人工测试用例编写的耗时，同时保障测试用例与应用程序的整体质量。

<b>研究方法</b>：我们采用了两种评估方案。其一，招募18名软件测试专业人士与学生开展问卷调查，对比ChatGPT生成的测试用例与人工编写的测试用例；其二，运用自然语言处理（Natural Language Processing）技术，量化衡量ChatGPT生成的测试用例与人工编写的测试用例之间的相似度。

<b>研究结果</b>：定性分析显示，ChatGPT生成的测试用例在完整性上优于人工测试用例，高出5.56个百分点，整体接受率达36.11%。人工测试用例则在清晰度上领先ChatGPT生成的测试用例6.95个百分点，接受率达到41.67%。定量分析发现，66.7%的ChatGPT生成测试用例与人工测试用例的相似度超过50%。

<b>研究结论</b>：我们开发的工具可助力自动化生成软件测试用例。人工智能生成的测试用例与人工编写的测试用例之间的相似度表明，该方案能够在节省时间、降低成本的同时，将测试质量维持在可接受的水平。此框架可用于保障网站布局更新与应用程序开发过程中的质量水准。