User participation in digital accessibility evaluations: reviewing methods and objectives

Although laws and standardization bodies promote user participation in digital accessibility evaluations, people with disabilities still consider themselves excluded from this process. One reason could be the lack of systematized knowledge about evaluation methods involving users. This article seeks to understand how and for what purpose digital accessibility evaluations with user participation were conducted in the scientific literature from 2018 to 2021. Three types of user participation emerged: 1) user-based usability testing to evaluate task accomplishment, user reactions and interface qualities; 2) interviewing users to assess the local and social factors impacting digital service accessibility; 3) using questionnaires or crowdsourcing to check the compliance of certain interfaces with accessibility standards. Participants are primarily chosen based on their functional impairments and, to a lesser degree, their project-related skills, biographical information, technology habits, among other criteria. The comprehensive user insights gained with these methods are judged to be positive whereas the lack of representativeness of the selected user samples is found to be regrettable. The article finally discusses the definitions of accessibility and disability that underpin these methodologies.