Data Supporting "Usability as a Predictor of Gamification Success: an Exploratory Study"

Appended Files to "Usability as a Predictor of Gamification Success: an Exploratory Study" This repository contains supplementary data for the article on the usability of gamified systems. Three data files are provided: two Excel workbooks and one CSV dataset: usability_recommendations_for_gamified_system.xlsx - the knowledge base of usability recommendations used to perform the evaluation. gamified_application_adherence_results.xlsx - evaluation data and aggregated results for every analysed gamified system. gamified_application_correlation_dataset.csv - dataset used for the correlation and regression analyses, including usability recommendation adherence metrics and application popularity indicators. --- 1. usability_recommendations_for_gamified_system.xlsx This workbook contains the rule base of usability recommendations that drives the evaluation in gamified_system_adherence_results.xlsx. Each sheet defines recommendations as input/output rules: the INPUT columns describe the applicability conditions (e.g., age group, application domain, disorder, usability goal, gamification element) and the OUTPUT columns provide the recommendation itself together with an illustrative example. 1.1 Sheet: Generalised recommendations General usability recommendations applicable at the system level (GUR). INPUT: usabilityGoal, usabilityPrinciple, ageGroup, applicationDomain, disorder, gamificationGoal OUTPUT: genRecommendation, example 1.2 Sheet: Suitable gamification elements Gamification elements that are recommended for a given context, together with the usability recommendations to apply when they are implemented. Feeds the URSIGE evaluation. INPUT: ageGroup, applicationDomain, disorder, gamificationGoal, usabilityGoal OUTPUT: gamificationElement, usabilityRecommendation 1.3 Sheet: Not suitable gamification elements Gamification elements that are not recommended for a given context. Used to identify non-recommended but implemented elements whose usability is evaluated under URUIGE. INPUT: ageGroup, applicationDomain, disorder, usabilityGoal, gamificationGoal OUTPUT: notSuitableGamificationElement 1.4 Sheet: ISO Usability recommendations for gamification elements derived from the ISO 9241 family of standards. INPUT: gamificationElement OUTPUT: ISOrecommendation, elementUsabilityRecommendation, example 1.5 Sheet: WCAG Usability recommendations for gamification elements derived from WCAG 2.2, scoped by disorder (e.g., hearing impairment, visual impairment). INPUT: gamificationElement, disorder OUTPUT: WCAGrecommendation, elementUsabilityRecommendation, example ------- 2. gamified_application_adherence_results.xlsx The workbook is organised into three logical groups of sheets: a per-system evaluation sheet for each analysed application, a category-level aggregation sheet, and a final results sheet. 2.1 Sheet: System-Level Evaluation Data Sheets: Duolingo, Duolingo ABC, Khan academy, Khan academy kids, Headspace, FITBIT, CodeAcademyGo, Fortune city, Habitica, Memrise, Brilliant, Fabulous, Clarify by fabulous, Strava, SuperBetter, Lumosity, Sidekick Health, ABCmouse, CogniFit, Shape by fabulous, Habit Tracker - Habit Mate, TaskGame - Gamify your life, Habio - Simple Habit Tracker, Walken, Narn Habit Tracker & Goals, Aula Lengua, Foursquare Swarm - Check In, Roomsy Chores Tracker & Chart, Fitmint - Walk & Steps Tracker, Workout Quest - Gamified Gym, Death Reminder App, Pharmacy Crack Rx Drug Trivia, Gametize Explore Experiences. Each sheet presents detailed evaluation data for one analysed gamified system and contains structured tables used to assess the implementation of usability recommendations. 2.1.1 Evaluation of Recommended Gamification Elements (URSIGE) The first table presents the evaluation of usability recommendations for recommended gamification elements (URSIGE). Three types of usability recommendation sources are included: Recommendations by age group and application domain ISO 9241 recommendations WCAG 2.2 recommendations For each gamification element (e.g., Quests, Points, Levels), the following implementation levels are recorded: Fulfilled - the recommendation is fully implemented Partially fulfilled - the recommendation is partially implemented Not fulfilled - the recommendation is not implemented Total - weighted number of fulfilled recommendations, calculated as Fulfilled + 0.5 × Partially fulfilled The values in the table indicate the number of recommendations in each category. A summary table on the right aggregates these values and provides: Total number of recommendations Number of fulfilled recommendations Number of partially fulfilled recommendations Number of not fulfilled recommendations Overall fulfilment percentage 2.1.2 Evaluation of Not Recommended Gamification Elements (URUIGE) The second table follows the same structure but evaluates usability recommendations for gamification elements that were not recommended by the tool but were still implemented in the system (URUIGE). The same three recommendation sources are used: Recommendations by age group and application domain ISO 9241 recommendations WCAG 2.2 recommendations The evaluation logic is identical to the recommended elements: Fulfilled / Partially fulfilled / Not fulfilled counts are recorded Totals and aggregated results are calculated A summary table provides overall metrics for non-recommended elements, including: Total number of recommendations Number of fulfilled recommendations Number of partially fulfilled recommendations Number of not fulfilled recommendations Overall fulfilment percentage 2.1.3 Degree of Satisfaction (DoS) Calculation At the bottom of the sheet, the Degree of Satisfaction (DoS) is calculated for the evaluated application. The calculation is based on: P - total number of ratings H - number of high ratings C - number of low ratings n - control parameter The DoS metric is used to reflect the overall user satisfaction with the application and is later used in correlation analysis with usability recommendation adherence. 2.2 Sheet: Results by Recommendation Category This sheet presents aggregated results for each analysed gamified system. Each column represents a system. Each row represents evaluation results for a specific recommendation category. The following categories are evaluated: General Usability Recommendations (GUR) Implemented Gamification Elements (IGE) Usability Recommendations for Recommended Gamification Elements (URSIGE) Usability Recommendations for Not Recommended Gamification Elements (URUIGE) 2.2.1 General Usability Recommendations (GUR) This section shows how each system satisfies general usability recommendations. The following values are provided: Total number of recommendations Fulfilled Partially fulfilled Not fulfilled Total fulfilled (Fulfilled + 0.5 × Partially fulfilled) Percentage fulfilled 2.2.2 Implemented Gamification Elements (IGE) This section evaluates how many recommended gamification elements are implemented. The following values are provided: Total number of recommended gamification elements Number of elements implemented in the system Percentage of recommended elements implemented 2.2.3 Usability Recommendations for Recommended Elements (URSIGE) This section presents aggregated results of usability recommendation implementation for recommended gamification elements. The values are derived from the system-level evaluation sheets, where each system was analysed individually. 2.2.4 Usability Recommendations for Not Recommended Elements (URUIGE) This section presents aggregated results of usability recommendation implementation for gamification elements that were not recommended but are implemented in the system. The values are derived from the system-level evaluation sheets. 2.3 Sheet: Results This sheet presents the aggregated results of all analysed gamified systems. Each row represents a single gamified system. Each column represents a specific evaluation metric. 2.3.1 System Information Age group - target user groups of the system (e.g., children, teenagers, adults, elderly) Application domain - domain(s) in which the system is used (e.g., education, health, productivity) 2.3.2 Popularity Metrics User rating - average user rating of the system (from Google Play store) Rate number - total number of user ratings (from Google Play store) Downloads - total number of downloads Popular rate (DoS) - Degree of Satisfaction metric calculated based on rating distribution; reflects overall user satisfaction and mitigates bias caused by rating volume 2.3.3 Usability and Gamification Metrics General Usability Recommendations (GUR) - percentage of general usability recommendations fulfilled in the system Implemented Gamification Elements (IGE) - percentage of recommended gamification elements that are implemented in the system Usability Recommendations for Recommended Gamification Elements (URSIGE) - percentage of usability recommendations fulfilled for recommended gamification elements Usability Recommendations for NOT Recommended Gamification Elements (URUIGE) - percentage of usability recommendations fulfilled for gamification elements that were not recommended but are implemented Overall - combined adherence metric calculated from GUR, URSIGE, and URUIGE Elements recommendations (URGE) - overall adherence to usability recommendations for gamification elements ------- 3. gamified_application_correlation_dataset.csv This file contains the application-level dataset used for the statistical analyses reported in the article. It includes one row for each of the 33 analysed gamified applications. The values were derived from the detailed and aggregated evaluation results provided in gamified_application_adherence_results.xlsx The file is semicolon-delimited. Usability recommendation adherence metrics are expressed as proportions ranging from 0 to 1. 3.1 Application and Popularity Data The following columns describe each analysed application and its popularity indicators: System - name of the analysed gamified application. User_rating - average user rating obtained from the Google Play Store. Rating_Count - total number of user ratings recorded for the application. Downloads - approximate number of application downloads reported by the Google Play Store. DoS - Degree of Satisfaction value calculated using the application rating data. This metric accounts for the distribution and number of ratings and was used as an additional application popularity indicator. 3.2 Usability Recommendation Adherence Metrics The following columns contain the aggregated usability and gamification evaluation results: GUR - coverage of General Usability Recommendations. IGE - coverage of Implemented Gamification Elements, representing the proportion of recommended gamification elements implemented in the application. URSIGE - coverage of Usability Recommendations for Suitable Implemented Gamification Elements. URUIGE - coverage of Usability Recommendations for Unsuitable Implemented Gamification Elements. Overall_Adherence - overall adherence metric calculated as the mean of the available GUR, IGE, URSIGE, and URUIGE values. URGE - combined adherence to usability recommendations for gamification elements, calculated as the mean of the available URSIGE and URUIGE values. An empty URUIGE value indicates that the application did not implement any gamification elements classified as not recommended for its context. In such cases, URUIGE was considered not applicable and was excluded from the calculation of the aggregated metrics. 3.3 Use in Statistical Analysis The dataset was used to calculate Spearman’s rank correlations between usability recommendation adherence metrics and the two application popularity indicators: average user rating; Degree of Satisfaction (DoS)