Dependability coefficients for multiple ratings.

Background
The Integrated Palliative Care Outcome Scale for People with Dementia is a promising instrument for nursing home quality improvement and research in dementia care. It enables frontline staff in nursing homes to understand and rate the needs and concerns of people with dementia. We recently adapted the measure to include easy language for users from various educational backgrounds.
Objectives
In this study, we examine the inter-rating reliability of the Integrated Palliative Care Outcome Scale for People with Dementia for frontline staff in nursing homes.
Methods
In this secondary analysis of an experimental study, 317 frontline staff members in 23 Swiss nursing homes assessed 240 people with dementia from a convenience sample. Reliability for individual items was computed using Fleiss Kappa. Because of the nested nature of the primary data, a generalisability and dependability study was performed for an experimental IPOS-Dem sum score.
Results
The individual Integrated Palliative Care Outcome Scale for People with Dementia items showed kappa values between .38 (95% CI .3–.48) and .15 (95% CI .08–.22). For the experimental IPOS-Dem sum score, a dependability index of .57 was found. The different ratings and time between ratings explain less than 2% of the variance in the sum score. The different nursing homes make up 12% and the people with dementia make up 43% of the sum score variance. The dependability study indicates that an experimental IPOS-Dem sum score could be acceptable for research by averaging two ratings.
Conclusion
Limited research has been conducted on the measurement error and reliability of patient-centred outcome measures for people with dementia who are living in nursing homes. The Swiss Easy-Read IPOS-Dem is a promising instrument but requires further improvement to be reliable for research or decision making. Future studies may look at its measurement properties for different rater populations or at different stages of dementia. Furthermore, there is a need to establish the construct validity and internal consistency of the easy-read IPOS-Dem.