Test Generation Tools: A Multivocal Systematic Literature Review

Context: Software testing tools are essential during the processes of developing and maintaining quality digital applications in a scenario where they become increasingly complex. Problem: The market offers hundreds of test generation tools with different approaches and functionalities, it is important that developers and designers have knowledge of which tools are best suited to their work in order to avoid unnecessary expenses and easily avoidable frustrations. Solution: to identify the main tools available on the market in order to highlight their advantages and disadvantages, thus presenting a reference guide that will help developers choose the most appropriate tool for each test situation. Method In this work: a multivocal literature review has been carried out to identify works related to test generation tools, also identifying the most cited tools in white and gray literature. Furthermore, a survey was developed and conducted with professionals working in different areas of software development, in order to ascertain their familiarity with the testing tools identified in the white and gray literature, as well as the advantages and challenges that have more weight when deciding which tool to adopt. Summarization of Results: The results showed that most of the tools found in the white literature, such as Monkey and Dynodroid, are not very well known and used by the participants and that they were more familiar with tools more mentioned in the gray literature, such as Postman and Selenium. In addition, a list of characteristics most desired and avoided by participants when needing to choose a new tool for a project was obtained. Impact on the IS field: This work contributes to IS field by providing a reference guide where practitioners can identify the most recommended tools according to a specific scenario, and then use a comparative table in order to evaluate the most appropriate tool for the project among those recommended. The Guide can be used by any testing practitioner.