Empirical data on the impact of eyenalyzer

How did the data collection work?The goal of the study is to compare the performance on data analysis tasks when using the data analysis tool eyenalyzer or others. Hence, participants are randomly assigned to one of two groups (i.e., with and without eyenalyzer).  All participants work on the same six data analysis tasks - according to their group assignment either with the tool eyenalyzer or with any other tool apart from it. After a maximum of two hours, the participants hand in their solutions to the tasks as well as a self-report of the time taken to complete each task (in minutes), the tooling used for each task (as list), and the estimated difficulty of each task (on a 7-point Likert scale). Finally, they fill out a questionnaire about their demographic data (i.e. age, gender, and previous experience). The submitted solutions are later graded in terms of correctness. Further details on the procedure can be found in this conference paper. 

What data is provided?In this repository you can find both the data obtained from 20 participants (i.e., file data.xlsx) and a german replication package (i.e., folder material.zip). 

The data file data.xlsx comprises 13 columns and 120 rows. Each row represents one combination of participant and task. The columns are:

participant the current participant as a unique identifier from {P01, ..., P20}

independent_variable the group assignment of the participant as one of {eyenalyzer, others}

task the current task as one of {T1, ..., T6}

score the grading of the current task for the current participant as one of {not completed, wrong, borderline, acceptable, correct}

time time used by the current participant for the current task in minutes

difficulty difficulty estimate of the current participant for the current task as one of {very difficult, difficult, rather difficult, indecisive, rather easy, easy, very easy}

tools_used listing of tooling used for the current task if the current participant belongs to the group without eyenalyzer

age age of the current participant in years

gender gender of the current participant as one of {male, female}

experience_programming previous experience of the current participant in programming as one of {none, little, indecisive, some, much}

experience_data_science previous experience of the current participant in data science as one of {none, little, indecisive, some, much}

experience_experimental_research previous experience of the current participant in experimental research as one of {none, little, indecisive, some, much}

experience_statistics previous experience of the current participant in statistics as one of {none, little, indecisive, some, much}

The replication folder material.zip comprises three subfolders with the following material:

group_with_eyenalyzer/primer.pdf information material on statistics provided to the group with eyenalyzer

group_with_eyenalyzer/tasks.pdf the instructions provided to the group with eyenalyzer

group_without_eyenalyzer/primer.pdf information material on statistics provided to the group without eyenalyzer

group_without_eyenalyzer/tasks.pdf the instructions provided to the group without eyenalyzer

both_groups/data/ a directory containing the 4 data files needed to complete the tasks (based on the data from this repository)

both_groups/questionnaire.pdf a transcription of the demographic questionnaire

Do you have further questions?For more information, please feel free to contact .