Quality Evaluation of Software Functional Requirements Generated by LLMs: A Systematic Mapping Study

Abstract. Research Context: Large Language Models (LLMs) have beenincreasingly applied in software engineering, especially in the generation offunctional requirements for information systems. Scientific and/or PracticalProblem: However, the quality of these requirements still raises concerns,such as ambiguities, incompleteness, and dependency on prompts. ProposedSolution and/or Analysis: This study conducts a systematic mapping toanalyze how the literature has evaluated requirements generated by LLMs.Related IS Theory: The work is aligned with the sociotechnical perspective,considering information systems requirements as both technical and socialartifacts. Research Method: A systematic mapping was conducted, reviewing1,875 studies and selecting 51 primary studies published between 2020 and2025. Summary of Results: The findings indicate a diversity of evaluationpractices, combining traditional criteria and NLP metrics, with advantagesin automation and standardization, but limitations regarding the absence ofbenchmarks and validation in industrial contexts, especially in the developmentof information systems. Contributions and Impact to IS area: The study con-tributes to academia by consolidating evaluation approaches and identifyinggaps, and to industry by supporting the understanding of risks and opportu-nities in the use of LLMs in the requirements engineering of information systems.