CyberGIS-Jupyter for Water (CJW) 2022-Q2 Release Notes

CyberGIS-Jupyter for Water Quarterly Release Announcement (2022-Q2)

Dear CJW users,

We are pleased to announce a new release of the CyberGIS-Jupyter for Water (CJW) platform at https://go.illinois.edu/cybergis-jupyter-water. This release includes several new capabilities and features summarized as follows.

1) CJW moved to a new home. Jetstream-1, an NSF-funded high-performance cloud computing resource where CJW was hosted for the past 3 years, was permanently shut down on July 31, 2022. Its successor, Jetstream-2, which offers much more powerful capabilities, has become the new home of CJW. All existing CJW user data and notebooks have been migrated to Jetstream-2. We do not expect users to experience any change in usage due to this transition but to enjoy a faster and smoother Jupyter environment backed by the latest hardware and cloud technology. In exceptional cases, the previous CJW instance on Jetstream-1 could be accessible upon user request.

2) Improved user experience in CyberGIS-Compute job submission: Have you ever had a long-running job submitted to high-performance computing (HPC) resources but found your Jupyter session died after the browser was idle for too long? The latest CyberGIS-Compute SDK now allows you to reinstate job submission sessions for all previous jobs you submitted. Just switch to the new “Your Jobs” tab page in the user environment and “Restore” the jobs you are interested in. This also gives you a chance to re-download model outputs from previous jobs.

3) WRFHydro model integration supports merging model outputs: A new option “Merge_Output” is added to the WRFHydro workflow supported by CyberGIS-Compute. If enabled, single-timestep NetCDF files can be merged on the “Time” dimension after model execution. Currently supported output types include CHANOBS, LDASOUT, GWOUT, LAKEOUT, RTOUT, and LSMOUT. This optional merging step can reduce data transfer size and speed up post-processing work on CJW. The merged files are put into a separate folder called “Outputs_Merged” alongside the original model outputs. Users can choose to download either or both. Please refer to the example notebook for more information.

4) Enhanced support for user customization to CJW kernels: While CJW has pre-installed a large collection of common libraries and tools to support a suite of hydrologic analysis and modeling workflows, users may still want to install something specific to certain use cases. CJW now allows users to directly use “!pip install XXX” in notebook cells to customize existing kernels. CJW supports flexible additions or changes on a per-kernel basis, which does not affect other existing kernels. Please refer to this example notebook for more information.

5) Updates on CJW backend (kernel, plugin, and bugfix): A new general-purpose kernel, Python3-2022-06, is added, which incorporates a rich set of new geospatial packages. The ‘StickyLand” JupyterLab plugin is installed that allows users to create customizable dashboards and linear notebooks; A bug specific to Apple Safari browser in the OpenWith operation has been fixed.

Please refer to the following resources for details and examples:

Run WRFHydro model on HPC resources using CyberGIS-Compute V2 (updated 2022-07)
https://www.hydroshare.org/resource/cc28d769943046fdac0f9b0c0fc2afc6/

Customization to CJW Kernels with Pip
https://www.hydroshare.org/resource/d18886d2aedf4a2e8c6302165b8fe10f/

CyberGIS-Compute SDK new features
https://cybergis.github.io/cybergis-compute-python-sdk/release-notes.html

CJW 2022-Q2 Release Notes on HydroShare
https://www.hydroshare.org/resource/34b04302d8b34cc6aab826f79b5e3802/

Please let us know if you have any questions or run into any problems ([e-mail]). Any feedback would be greatly appreciated.

Best regards,
CyberGIS-Hydro team

尊敬的 CJW 用户们，

我们荣幸地宣布 CyberGIS-Jupyter for Water (CJW) 平台（https://go.illinois.edu/cybergis-jupyter-water）的最新版本发布，该版本发布于 2022 年第二季度。本次发布包含多项新增功能和特性，概述如下。

1) CJW 迁移至新址。过去三年中作为 CJW 托管平台的 NSF 资助高性能云计算资源 Jetstream-1 于 2022 年 7 月 31 日永久关闭。其继任者 Jetstream-2，提供了更为强大的功能，现已成为 CJW 的新家。所有现有的 CJW 用户数据和笔记本已迁移至 Jetstream-2。我们预计用户在使用过程中不会感受到任何变化，反而将享受到由最新硬件和云计算技术支持的更快、更流畅的 Jupyter 环境。在特殊情况下，根据用户请求，Jetstream-1 上的先前 CJW 实例可能仍然可访问。

2) 改进 CyberGIS-Compute 作业提交的用户体验：您是否曾提交过长时间运行的任务至高性能计算（HPC）资源，但发现当浏览器闲置时间过长后，您的 Jupyter 会话已终止？最新的 CyberGIS-Compute SDK 现在允许您恢复所有先前提交的作业的提交会话。只需切换到用户环境中新的“您的作业”标签页，并“恢复”您感兴趣的作业。这也为您重新下载先前作业的模型输出提供了机会。

3) WRFHydro 模型集成支持合并模型输出：CyberGIS-Compute 支持的 WRFHydro 工作流程中新增了“Merge_Output”选项。如果启用，模型执行后可以在“时间”维度上合并单时间步 NetCDF 文件。目前支持的输出类型包括 CHANOBS、LDASOUT、GWOUT、LAKEOUT、RTOUT 和 LSMOUT。此可选合并步骤可以减少数据传输大小并加快 CJW 上的后处理工作。合并后的文件将被放置在名为“Outputs_Merged”的单独文件夹中，与原始模型输出并列。用户可以选择下载其中之一或两者。请参阅示例笔记本以获取更多信息。

4) 增强 CJW 内核（内核、插件和错误修复）的用户定制支持：虽然 CJW 已预装了大量的常用库和工具以支持一系列水文分析和建模工作流程，但用户可能仍希望针对特定用例安装某些特定的库。CJW 现在允许用户在笔记本单元格中直接使用“!pip install XXX”来自定义现有内核。CJW 支持对每个内核进行灵活的添加或更改，而不会影响其他现有内核。请参阅此示例笔记本以获取更多信息。

5) CJW 后端（内核、插件和错误修复）的更新：新增了一个通用内核 Python3-2022-06，其中包含了一组新的地理空间包。安装了“StickyLand” JupyterLab 插件，允许用户创建可定制的仪表板和线性笔记本；修复了 OpenWith 操作中针对 Apple Safari 浏览器的特定错误。

请参阅以下资源以获取详细信息和示例：

在 HPC 资源上运行 WRFHydro 模型使用 CyberGIS-Compute V2（更新于 2022-07）
https://www.hydroshare.org/resource/cc28d769943046fdac0f9b0c0fc2afc6/

使用 Pip 定制 CJW 内核
https://www.hydroshare.org/resource/d18886d2aedf4a2e8c6302165b8fe10f/

CyberGIS-Compute SDK 新功能
https://cybergis.github.io/cybergis-compute-python-sdk/release-notes.html

CJW 2022-Q2 发布说明在 HydroShare
https://www.hydroshare.org/resource/34b04302d8b34cc6aab826f79b5e3802/

如果您有任何疑问或遇到任何问题（[e-mail]），请告知我们。任何反馈都将受到高度重视。

此致，
CyberGIS-Hydro 团队