Using the SOCIO Chatbot for UML Modelling: A Second Family of Experiments

CONTENTS OF THIS FILE

 * Data Analysis

 * Experimental Design

Data Analysis

This folder consists of data obtained from the experiments and the materials we use for analysis. Here we provide the following documents:

01-The folder "Class diagrams"

      This folder contains class diagrams realized by participants in three experiments.

02-Code.R 

      Here we provide the code we use to analyze the raw data of three experiments. With this code, we generate profile-plots, violin-plots, the summaries of descriptive statistics. 

03-Familiarity Result.xlsx

      Here we provide the data we obtained from the familiarity questionnaire.

04-Raw Data.xlsx

      Here we provide the aggregation of the raw data of the three experiments. 

05-Task 1_Ideal Class diagram.png and 06-Task 2_Ideal Class diagram.png

      Here we provide two ideal class diagrams corresponding to task 1 and task 2. To analyze the response variable effectiveness and the quality of class diagrams, we compare these two ideal class diagrams with each of the class diagrams realized by participants.

07-How to calculate the quality.pdf

      Here we provide two examples of SOCIO and Creately to illustrate how we compare the ideal class diagram and the class diagram obtained in the experiment.

08-The folder "Thematic Analysis"

      This folder contains the code and data that we use to generate the thematic results graphs.

Experimental Design

This folder contains the original materials used in the experimental design process. Here we provide the following documents:

01-Experiment Procedure.pdf

      Here we provide the experiment procedure in detail. The experiment procedure is identical among the three experiments. 

02-Informed Consent.pdf

      Here we provide the informed consent in detail. We require participants to sign the informed consent before the experiment, stating that they granted us permission to record their data via Telegram. We ensured that the participation of all subjects in this research was completely anonymous, and no information they shared could be electronically traced to them

03-The folder "Questionnaires"

      This folder contains the familiarity questionnaire we used before the execution of each experiment (i.e., Familiarity Questionnaire.pdf), the SUS questionnaire for Task 1 (i.e., SUS Questionnaire_Task1.pdf) and the SUS questionnaire for Task 2 (i.e., SUS Questionnaire_Task2.pdf).

04-The folder “Tasks”

      This folder contains the specific content of Task 1 (i.e., Task1.pdf) and Task 2 (i.e., Task2.pdf).

05-The folder “Tutorial Slides”

      This folder contains the slides we use to give the tutorials of the chatbot SOCIO (i.e., Tutorial_SOCIO V1.pdf) and the web-based online tool Creately (i.e., Tutorial_Creately.pdf).

本文件包含内容
* 数据分析
* 实验设计

数据分析
本文件夹包含实验所得数据与本次分析所用的相关材料，现将提供的文件清单如下：
01- 「类图」文件夹
本文件夹收录了三项实验中参与者所绘制的各类类图。
02- Code.R
本次提供用于分析三项实验原始数据的代码，通过该代码可生成轮廓图、小提琴图以及描述性统计量汇总结果。
03- 熟悉度调查结果.xlsx
本文件存储了从熟悉度调查问卷中获取的实验数据。
04- 原始数据.xlsx
本文件汇总了三项实验的全部原始数据。
05- 任务1_理想类图.png 与 06- 任务2_理想类图.png
本次提供对应任务1与任务2的两份理想类图。为分析类图的有效性响应变量与绘制质量，我们将这两份理想类图分别与参与者所绘制的各类类图进行比对。
07- 质量计算方法.pdf
本次提供SOCIO与Creately的两个示例，用以说明我们如何比对理想类图与实验中获取的参与者绘制类图。
08- 主题分析文件夹
本文件夹包含用于生成主题分析结果图表的代码与相关数据。

实验设计
本文件夹存储了实验设计流程中使用的原始材料，现将提供的文件清单如下：
01- 实验流程.pdf
本次提供详细的实验流程说明，三项实验所采用的实验流程完全一致。
02- 知情同意书.pdf
本次提供详细的知情同意书说明。我们要求参与者在实验前签署知情同意书，确认其允许我们通过Telegram记录其实验数据。本研究确保所有受试对象的参与完全匿名，其共享的任何信息均无法通过电子途径追溯至本人。
03- 调查问卷文件夹
本文件夹包含各实验开展前使用的熟悉度调查问卷（即「熟悉度调查问卷.pdf」）、任务1所用的系统可用性量表（System Usability Scale，简称SUS）问卷（SUS Questionnaire_Task1.pdf）以及任务2所用的系统可用性量表（System Usability Scale，简称SUS）问卷（SUS Questionnaire_Task2.pdf）。
04- 任务文件夹
本文件夹包含任务1（即「Task1.pdf」）与任务2（即「Task2.pdf」）的具体内容说明。
05- 教程幻灯片文件夹
本文件夹包含我们用于讲解聊天机器人SOCIO与基于网页的在线工具Creately的教程幻灯片，分别为「Tutorial_SOCIO V1.pdf」与「Tutorial_Creately.pdf」。