Industry-Derived Requirements for an LLM-Based Conversational Agent to Assist Software Staging: Protocol, Raw Data, Scripts, and Demographics

This repository contains the protocol, raw data, literal quotes, and clusters related to the focus group in the paper "Industry-Derived Requirements for an LLM-BasedConversational Agent to Assist Software Staging," currently under review at the RE Industry Innovation Track. The repository contains the following files

Focus-Group-Protocol.docx

It contains the detailed protocol, variables, and motivation of the focus group sessions.

Demographics.xlsx

It contains the anonymised demographics of the industry experts who joined the focus group sessions. We collect data about: 

Session

What is your current role in your organization?

How much experience do you have in your current role? (in years)    

What is your organization's size?    

Do you have any experience as SRE/QA/Performance engineer? If yes, how much? (in years)   

Which of the following profiles do you feel identify with?    

Which tools, if used, do you have experience with when taking staging decisions?  

Do you have any hands-on experience in other fields/roles/Industry? If yes, in which field/role/industry and how many years?    

Do you have experience implementing Machine Learning models? If yes, how much experience (in years)?    

Do you use chatbots/conversational agents based on Large Language Models (LLMs) on your work? If yes, which ones and how often do you use them?

When taking a staging decision/assessing the quality of the software, do you use any chatbots/conversational agents to support you? If yes, how do you use them?

Clustering-anonym.xlsx

It contains the raw data and clustering of all documents collected via the focus group method. It contains four pages:

QV1

Contains the clusters related to questions a conversational must/should/could/will not answer, together with the documents that compose the cluster and the priority given by the subjects.

QV2

Contains the clusters related to behaviours and non-functional requirements that the conversational agent must/should/could/will not have, together with the documents that compose the cluster and the priority given by the subjects.

QV3

Contains the clusters related to situations where a conversational agent would be useful/not useful, together with the documents that compose the cluster and the priority given by the subjects

Traceability matrix

Contains the traceability matrix among the clusters, the proposed i* goal model, and the agentic architecture. 

CA_focus_group_topic_modelling.ipynb

Contains the BERTopic script for running topic modelling before human intervention.