DesignSafe Annual Evaluation Report 2022-2023

This report summarizes the 2022-23 evaluation of the DesignSafe cyberinfrastructure, which was conducted by TACC’s Expanding Pathways in Computing (EPIC) STEM Evaluation Services. EPIC used a mixed methods approach to assess the implementation, outcomes, and impacts of DesignSafe on the natural hazards community. The evaluation aims to inform project improvements and report on outcomes over the project’s duration. Multiple data sources were used, including platform analytics, user profiles, annual and microsurvey responses, trainee surveys, and user and stakeholder interviews.

Annual user survey respondents indicated they have high satisfaction with DesignSafe and believe it to be useful to the natural hazards community. Users valued the platform for data reuse, collaboration, and research publication, although they faced challenges such as JupyterHub instability and the complexity of the site. Training participants expressed satisfaction with the trainings and indicated plans to apply their knowledge, with a preference for more in-depth and practical training opportunities. User interviews highlighted differences among high and low activity users, with high activity users taking advantage of a wider array of DesignSafe components (e.g., Jupyter, HPC, Slack, Python, MATLAB, and OpenSees). User interviewees recommended feature improvements, allowing for more customized functions and applications, and more training materials. Stakeholders said a major accomplishment included obtaining CoreTrustSeal certification; and they were pleased with how well DesignSafe promoted collaboration within the community. Stakeholders also recommended enhancements to platform functionality and user engagement. Based on these findings, the evaluators recommend both educating users and potential, developing entry points through use cases and Jupyter Notebook templates, improving site navigability, considering Data Depot requests, providing guidance on usage restrictions, and continuing current efforts to enhance the platform.