Spectral band-shifting of multispectral remote-sensing reflectance products: Insights for matchup and cross-mission consistency assessments (Dataset)

HICO Dataset Overview:

The dataset archived here includes unique HICO Rrs dataset shown in the paper, “Spectral band-shifting of multispectral remote-sensing reflectance products: Insights for matchup and cross-mission consistency assessments”, published in Remote Sensing of Environment Journal (Salem et al., 2023). The dataset comprise of nearly 6.2 million unique remote-sensing reflectance (Rrs) records in the spectral range of 353–719 nm, along with 9 ancillary products. The unique Rrs of HICO dataset was extracted from 8,893 Hyperspectral Imager for the Coastal Ocean (HICO) images, captured from 2009 to 2014.

HICO Dataset and Pre-processing:

The HICO sensor, mounted on the International Space Station, collected hyperspectral data specifically for coastal ocean observation. It featured 65 spectral bands between 353 and 719 nm with a 5.7 nm sample interval in the open ocean, coastal waters, estuaries, and shallow regions.

The HICO images were processed to filter out invalid or negative Rrs values across the 65 spectral bands of HICO data. An iterative process was employed to refine and extract unique Rrs data, resulting in a novel look-up table (LUT) consisting of nearly 6.2 million unique Rrs records. The LUT's Rrs spectra were further refined using cubic spline interpolation to adjust the sampling interval from 5.7 nm to 1 nm, resulting in a comprehensive LUT of 6,335,988 records spanning 367 bands (353–719 nm). Additionally, the dataset includes 9 ancillary products as follows:

'aot_868':    Aerosol optical thickness at 868 nm,

'angstrom': Aerosol Angstrom exponent from 443 to 865 nm, 

'chlor_a':     Chlorophyll Concentration (mg m^-3), OCI Algorithm,

'chl_ocx':     Chlorophyll Concentration (mg m^-3), OC4 Algorithm,

'Kd_490':     Diffuse attenuation coefficient at 490 nm (m^-1) using KD2 algorithm,

'pic':            Calcite Concentration (mol m^-3) using Balch and Gordon,

'poc':           Particulate Organic Carbon (mg m^-3) using D. Stramski, 2007, 

'longitude': Geographical longitude,

'latitude':    Geographical latitude.

Utilization of the Dataset:

One of the application of this dataset is in the field of spectral band-shifting, a technique crucial for comparing and integrating Rrs data from different multispectral sensors. These sensors often vary in their spectral bands and response functions, posing challenges for consistent data analysis across different missions. The band-shifting technique, particularly the Spectral Matching Technique with HICO data (SMTH) approach, is extensively described in the paper titled "Spectral band-shifting of multispectral remote-sensing reflectance products: Insights for matchup and cross-mission consistency assessments" from Remote Sensing of Environment (RSE) https://doi.org/10.1016/j.rse.2023.113846. This technique is pivotal in approximating Rrs in bands that are not commonly available across various sensors, enhancing data compatibility and accuracy.

Available files:

HICO_LookUpTable_Rrs1nm_Ancillary_6M.nc: netCDF file including 6,201,385 unique HICO records with 367 Rrs bands and 9 ancillary products.

Read_HICO_Dataset.py: Python script for reading the HICO unique reflectance data.

SMTH_Rrs_BandShift.py: Python script to conduct band-shifting using the SMTH approach.

Sample_Input.csv: Sample data file for testing the SMTH approach.

GitHub:

For detailed instructions on setting up the Python environment and running the script, please visit our GitHub repository at SMTH Rrs BandShifting. This link provides comprehensive guidance on environment setup, dependencies, and script execution.

Data Use Statement: 

This work is made available under the terms of the Creative Commons Attribution 4.0 International License [Link]. In cases where the HICO data product is integral to your work, or a significant outcome or conclusion relies on it, we would value the opportunity to discuss these results with you. This ensures the accurate usage and interpretation of the data. Additionally, as we are continually enhancing the Unique HICO data, engaging with us early in your project could (i) contribute to the improvement of our product, and (ii) potentially allow us to offer you a more updated version of the data. Thank you for your collaboration!

HICO数据集概述：

本存档数据集包含发表于《环境遥感》（Remote Sensing of Environment）期刊2023年论文《多光谱遥感反射率产品的光谱波段移位：匹配与跨任务一致性评估的启示》（Spectral band-shifting of multispectral remote-sensing reflectance products: Insights for matchup and cross-mission consistency assessments，Salem等人，2023）中所采用的专属HICO遥感反射率（remote-sensing reflectance, Rrs）数据集。该数据集包含近620万条唯一的遥感反射率记录，光谱覆盖范围为353–719 nm，同时附带9项辅助产品。HICO数据集的专属Rrs数据提取自2009年至2014年间采集的8893景海岸带高光谱成像仪（Hyperspectral Imager for the Coastal Ocean, HICO）影像。

HICO数据集与预处理流程：

搭载于国际空间站的HICO传感器专为海岸带海洋观测采集高光谱数据，其在公海、近岸海域、河口及浅水区设有65个光谱波段，波段采样间隔为5.7 nm，光谱范围覆盖353–719 nm。

我们对HICO影像进行了预处理，过滤了HICO数据65个光谱波段中无效或负值的Rrs值。通过迭代流程优化并提取专属Rrs数据，最终构建了包含近620万条唯一Rrs记录的新型查找表（Look-up Table, LUT）。随后，采用三次样条插值（cubic spline interpolation）对该查找表的Rrs光谱进行进一步优化，将采样间隔从5.7 nm调整至1 nm，最终得到覆盖367个波段（353–719 nm）、共计6335988条记录的完整查找表。此外，本数据集包含以下9项辅助产品：

'aot_868'：868 nm处气溶胶光学厚度（Aerosol optical thickness at 868 nm）
'angstrom'：443–865 nm波段气溶胶安斯特朗指数（Aerosol Angstrom exponent from 443 to 865 nm）
'chlor_a'：基于OCI算法的叶绿素浓度（mg·m⁻³）（Chlorophyll Concentration (mg m^-3), OCI Algorithm）
'chl_ocx'：基于OC4算法的叶绿素浓度（mg·m⁻³）（Chlorophyll Concentration (mg m^-3), OC4 Algorithm）
'Kd_490'：采用KD2算法得到的490 nm处漫衰减系数（m⁻¹）（Diffuse attenuation coefficient at 490 nm (m^-1) using KD2 algorithm）
'pic'：基于Balch与Gordon方法的方解石浓度（mol·m⁻³）（Calcite Concentration (mol m^-3) using Balch and Gordon）
'poc'：基于D. Stramski 2007年方法的颗粒有机碳浓度（mg·m⁻³）（Particulate Organic Carbon (mg m^-3) using D. Stramski, 2007）
'longitude'：地理经度
'latitude'：地理纬度

数据集应用：

本数据集的应用场景之一为光谱波段移位技术（spectral band-shifting）领域，该技术对于对比与整合来自不同多光谱传感器的Rrs数据至关重要。不同多光谱传感器的光谱波段与响应函数往往存在差异，这给跨任务的一致性数据分析带来了挑战。波段移位技术，尤其是基于HICO数据的光谱匹配技术（Spectral Matching Technique with HICO data, SMTH），在《环境遥感》期刊2023年论文《多光谱遥感反射率产品的光谱波段移位：匹配与跨任务一致性评估的启示》（DOI: 10.1016/j.rse.2023.113846）中已有详细阐述。该技术可用于近似不同传感器共通缺失波段的Rrs数据，有效提升数据兼容性与分析精度。

可用文件：

HICO_LookUpTable_Rrs1nm_Ancillary_6M.nc：netCDF格式文件，包含6201385条唯一HICO记录，涵盖367个Rrs波段与9项辅助产品。
Read_HICO_Dataset.py：用于读取HICO专属反射率数据的Python脚本。
SMTH_Rrs_BandShift.py：采用SMTH方法实现波段移位的Python脚本。
Sample_Input.csv：用于测试SMTH方法的示例数据文件。

GitHub仓库：
如需了解Python环境搭建与脚本运行的详细操作指南，请访问我们的SMTH Rrs BandShifting GitHub仓库，该页面提供了环境配置、依赖安装与脚本执行的完整指导。

数据使用声明：

本作品采用知识共享署名4.0国际许可协议（Creative Commons Attribution 4.0 International License）进行授权[链接]。若您的研究工作中使用了本HICO数据产品，或研究成果与结论依赖于该数据，我们期待与您就相关结果进行交流，以确保数据的正确使用与解读。此外，我们正在持续优化专属HICO数据集，若您能在项目初期与我们沟通，将有助于（i）改进我们的数据集产品，（ii）为您提供更新版本的数据。感谢您的合作！