DesignSafe Annual Evaluation Report 2021-2022

This report summarizes the 2021-22 evaluation of the DesignSafe cyberinfrastructure, which was conducted by TACC’s Expanding Pathways in Computing (EPIC) STEM Evaluation Services. EPIC used a mixed methods approach to assess the implementation, outcomes, and impacts of DesignSafe on the natural hazards community. The evaluation aims to inform project improvements and report on outcomes over the project’s duration. Multiple data sources were used, including platform analytics, user profiles, annual and microsurvey responses, trainee surveys, and user and stakeholder interviews.

The evaluation report's key findings indicate that nearly all annual survey respondents were satisfied with DesignSafe and found it valuable for the natural hazards community. They considered it trustworthy for data storage and impactful for data sharing, collaboration, and accelerating their research. However, users faced challenges due to a lack of education on the platform's full potential and technical issues. Training survey respondents, most of whom had very little to no experience with DesignSafe prior to the trainings, expressed satisfaction with the trainings and planned to implement their new knowledge. User interviewees found DesignSafe valuable for its collaborative features, data storage and sharing capabilities, and analysis tools, but recommended improvements to its features, documentation, and the provision of more tutorials and webinars. Stakeholders acknowledged the project's accomplishments, such as updated resources, broader user base support, and open-access data sharing, and recommended further education and feature improvements. The evaluators recommend increasing and advertising training sessions, optimizing platform functionalities, encouraging and supporting data publication, continue building communities and capacity through Slack and earthquake researchers, and enhancing user guides, use cases, and documentation.