遇见数据集

Evaluation of LLM-Generated Functional Requirements: A Preliminary Study Based on Quality Attributes and Human Perception

收藏
Zenodo2026-05-22 更新2026-05-26 收录
官方服务:

资源简介:

Large Language Models (LLMs) have increasingly been explored in Requirements Engineering (RE), especially to support the generation and refinement of software requirements. However, empirical evidence regarding the quality of AI-generated requirements remains limited, particularly in educational settings involving novice users. This paper presents an exploratory empirical study conducted with 27 undergraduate Software Engineering students who used Google Gemini 2.5 Pro to support the generation of functional requirements across seven software projects. A total of 106 functional requirements were independently evaluated by three RE specialists using classical quality attributes, including clarity, completeness, consistency, correctness, verifiability, and feasibility. The results indicate moderate overall quality, with stronger performance in correctness, consistency, and feasibility. In contrast, recurrent limitations were identified in completeness and verifiability, mainly associated with insufficient detail and difficulties in defining objective validation criteria. The findings also suggest that prompt detail and contextualization directly influenced the quality of the generated requirements, reinforcing the importance of Prompt Engineering during interactions with LLMs. In addition, students reported positive perceptions regarding productivity and learning support, although they also highlighted the constant need for human review. As a contribution, this study provides preliminary empirical evidence on the use of LLMs in educational RE activities and discusses opportunities for future investigations involving human-AI collaboration and prompt engineering in Requirements Engineering.

提供机构:
Zenodo
创建时间:
2026-05-22
二维码
社区交流群
二维码
科研交流群
商业服务