RAGE_Pilot2_SPL-SpaceModules

General description:

The data are gathered during an evaluation of the second version of the game Space Modules Incorporated within the RAGE project.

This evaluation investigated the quality of the ecologically valid educational setting with the game (game – embedded feedback & debriefing) for acquiring IT helpdesk communication skills. The quality of the game was examined by looking at participants’ opinions on game-experience, studyability, and perceived affordances of the game for acquiring communication skills as IT helpdesk employee (i.e., perceived usefulness for learning). In addition, more objective measurements of participants’ learning progress were gathered via in-game traces (i.e., in-game scores on five variables regarded important in IT helpdesk communication) and three virtual conversation MC-tests (before the game, one or two weeks after game completion).

The main research question targeted in this evaluation was: Does the educational setting with the game (i.e., game-embedded feedback & debriefing) lead to learning progress, and by which effort and satisfaction of the learner (task load, motivation)?

In total, 336 students installed the game, but a far lower number of 111 students completed the evaluation.

According to the results, the main research question with respect to whether learning gains occurred from the educational setting with the game could be positively answered.

Topic
ACM CSS 2012: Applied computing~Interactive learning environments   • Human-centered computing~Usability testing 
PsycINFO Classification: 3500 Educational Psychology - 3530 Curriculum & Programs & Teaching Methods; 3550 Academic Learning & Achievement

Name entitites
Organizational information: Open University of the Netherlands
Geographical information: The Netherlands
Time information: February 2018 – November 2018

Types: SPSS-file (version 24)

RAGCS target group: demand side – end-users - adults - vocational school

Evaluation dimensions
Evaluation object: Space Modules Incorporated game
Methodology/design: A mixed ANOVA with communication-scores as within-subjects factor (three scores) and feedback-formats as between-subjects factor (three groups) to measure the learning effectiveness of feedback-formats by using in-game traces (scores).
Evaluation variables: Communication skills, Task load, Motivation, Flow, Studyability

Instruments: Communication skills (in-game-traces); Task load (NASA-TLX, Hart & Staveland, 1988); Motivation (IMI, Ryan & Deci, 2000); Flow (FSS, Rheinberg, Vollmeier, & Engeser, 2003); Studyability (Nadolski & Hummel, 2017)

Knowledge/skill elements
RAGCS skills: skills - cognitive skills – applying; skills - affective skills - interpersonal
ESCO skills: communicate with customers (http://data.europa.eu/esco/skill/0da516ee-e70e-4384-be13-f5ff80be8127); ICT helpdesk agent; IT helpdesk communication

Relationships: D8.4 Second RAGE Evaluation Report
Related dataset: 10.5281/zenodo.2564750