Manually Annotated Event Log of Users Prompts in LLMs for Conceptual Modeling

This repository contains the supplementary material for our paper at ER 2024 Conference. 

The data contains the results of an empirical study with 76 undergraduate information systems students. The students submitted the course assignments in 39 groups (of one or two students). The assignment used for the study required use case modeling with UML use case diagrams and domain modeling with UML class diagrams. The groups were first expected to interact with an LLM and then, if needed, to manually improve their models. Groups were randomly assigned to interact with either GPT 4.0 or Code Llama 34B Instruct in one of three application domains. 

The participants were instructed to engage with the LLM until they were satisfied with the results or opted to skip further refinement. The interaction log contains the following fields: User ID, Input (the user prompt), Response (the modeling artifacts), and the Prompt Number (within user ID).