EarthCube2021 An Approach for Creating Immutable and Interoperable End-to-End Hydrological Modeling Computational Workflows

This HydroShare resource provides the Jupyter Notebooks created for the study "An Approach for Creating Immutable and Interoperable End-to-End Hydrological Modeling Computational Workflows" led by researcher Young-Don Choi submitted to the 2021 EarthCube Annual meeting, Notebook Sessions.

To find out the instructions on how to run Jupyter Notebooks, please refer to the README file provided in this resource.

For the sake of completeness, the abstract for the study submitted to the EarthCube session is mentioned below:

"Reproducibility is a fundamental requirement to advance science. Creating reproducible hydrological models that include all required data, software, and workflows, however, is often burdensome and requires significant work. Computational hydrology is a rapidly advancing field with fast-evolving technologies to support increasingly complex computational hydrologic modeling. The growing model complexity in terms of variety of software and cyberinfrastructure capabilities makes achieving computational reproducibility extremely challenging. Through recent reproducibility research, there have been efforts to integrate three components: 1) (meta)data, 2) computational environments, and 3) workflows. However, each component is still separate, and researchers must interoperate between these three components. These separations make verifying end-to-end reproducibility challenging. Sciunit was developed to assist scientists, who are not programming experts, with encapsulating these three components into a container to enable reproducibility in an immutable form. However, there were still limitations to support interoperable computational environments and apply end-to-end solutions, which are an ultimate goal of reproducible hydrological modeling. Therefore, the objective of this research is to advance the existing Sciunit capabilities to not only support immutable, but also interoperable computational environments and apply an end-to-end modeling workflow using the Regional Hydro-Ecologic Simulation System (RHESSys) hydrologic model as an example. First, we create an end-to-end workflow for RHESSys using pyRHESSys on the CyberGIS-Jupyter for Water platform. Second, we encapsulate the aforementioned three components and create configurations that include lists of encapsulated dependencies using Sciunit. Third, we create two HydroShare resources, one for immutable reproducibility evaluation using Sciunit and the other for interoperable reproducibility evaluation using library configurations created by Sciunit. Finally, we evaluate the reproducibility of Sciunit in MyBinder, which is a different computational environment, using these two resources. This research presents a detailed example of a user-centric case study demonstrating the application of an open and interoperable containerization approach from a hydrologic modeler’s perspective."

本HydroShare资源提供了由研究人员Young-Don Choi领导，提交于2021年EarthCube年度会议笔记本会议的“构建不可变与互操作端到端水文模型计算工作流的方法”研究所创建的Jupyter笔记本。欲了解如何运行Jupyter笔记本的说明，请参阅本资源提供的README文件。

为了全面性，以下为提交至EarthCube会议的该研究摘要：

“可重现性是推进科学发展的基本要求。然而，创建包含所有所需数据、软件和工作流的可重现水文模型往往是一项繁重的任务，且需要大量的工作。计算水文学是一个快速发展的领域，其技术正以惊人的速度发展，以支持日益复杂的水文计算模型。在软件和计算机网络能力方面不断增长的模型复杂性使得实现计算可重现性变得极为困难。通过最近的可重现性研究，人们已经努力整合三个关键组件：1)（元）数据，2）计算环境，和3）工作流程。然而，这些组件仍然各自独立，研究人员必须在三者之间进行交互操作。这种分割使得验证端到端的可重现性变得极具挑战。Sciunit的开发旨在协助非编程专家的科学家将这些三个组件封装进容器中，以实现不可变形式的可重现性。然而，在支持互操作的计算环境和实现端到端解决方案方面，仍存在局限性，而这正是可重现性水文模型的最终目标。因此，本研究的目标是提升现有的Sciunit功能，不仅支持不可变的计算环境，还要支持互操作的计算环境，并以区域水文生态系统模拟系统（RHESSys）水文模型为例，实现端到端建模工作流程的应用。首先，我们利用CyberGIS-Jupyter平台上的pyRHESSys为RHESSys创建一个端到端的工作流程。其次，我们利用Sciunit封装上述三个组件，并创建包含封装依赖项列表的配置。第三，我们创建了两个HydroShare资源，一个用于不可变可重现性评估，另一个用于使用Sciunit创建的库配置实现的互操作可重现性评估。最后，我们使用这两个资源在MyBinder（一个不同的计算环境）上评估Sciunit的可重现性。本研究提供了一个以用户为中心的案例研究，详细展示了从水文模型制作者的视角应用开放和互操作容器化方法的实例。”}