Dataset_Kelders et al_Psychometric Evaluation of the TWente Engagement with Ehealth Technologies Scale (TWEETS): Evaluation Study_JMIR2020

This data was collected for the study: Psychometric Evaluation of the TWente Engagement with Ehealth Technologies Scale (TWEETS): Evaluation Study.Background: Engagement surfaces as a predictor for effectiveness of Digital Health Interventions. However, a shared understanding of engagement is missing. Therefore, a new scale has been developed that proposes a clear definition and creates a tool to measure it. The TWente Engagement with Ehealth Technologies Scale (TWEETS) is- based on a systematic review and interviews with engaged health app users. It defines engagement as a combination of behavior, cognition and affect.Objective: First, this paper is aimed at evaluating the psychometric properties of the TWEETS. Second, a comparison is made with the experiential part of the Digital Behavior Change Intervention Engagement Scale (DBCI-ES-Ex), a scale that showed some issues in previous psychometric analyses.Methods: Participants (n = 288) were asked to use any step-counter app on their smartphone for two weeks. They completed online questionnaires at four time points (T0 = baseline, T1 = after 1 day, T2 = 1 week and T3 = 2 weeks). At T0, demographics and personality (conscientiousness and intellect/imagination) were assessed, while at T1 - T3, engagement, involvement, enjoyment, subjective usage and perceived behavior change were included as measures that are theoretically related to our definition of engagement. Analyses focused on internal consistency, reliability, convergent, divergent and predictive validity of both engagement scales. Convergent validity was assessed by correlating the engagement scales with involvement, enjoyment and subjective usage; divergent validity by correlating the engagement scales with personality; and predictive validity by regression analyses using engagement to predict perceived behavior change at later time points.Results: Cronbach's alpha of the TWEETS was 0.86, 0.86 and 0.87 on T1 - T3. Exploratory factor analyses indicated that a one-factor structure best fitted the data. The TWEETS is moderately to strongly correlated with involvement and enjoyment (theoretically related to cognitive and affective engagement respectively) (P<.001). Correlations between the TWEETS and frequency of use were non-significant or small and differences between adherers and non-adherers on the TWEETS were significant (P<.001). Correlations between personality and the TWEETS were non-significant. The TWEETS at T1 was predictive of perceived behavior change at T3, with an explained variance of 16%.The psychometric properties of the TWEETS and the DBCI-ES-Ex seemed comparable on some aspects (e.g. internal consistency) and on other aspects the TWEETS seemed somewhat superior (divergent and predictive validity).Conclusion: The TWEETS performs quite well as an engagement measure with high internal consistency, reasonable test-retest reliability and convergent validity, good divergent validity, and reasonable predictive validity. As the psychometric quality of a scale is a reflection of how closely a scale matches the conceptualization of the concept, this paper is also an attempt to conceptualize and define engagement as a unique concept, providing a first step towards an acceptable standard of defining and measuring engagement.