Supporting data repository for "A Network Calibration Approach Improves the Accuracy and Long-Term Stability of a Low-Cost Air Quality Mesonet in New York City"

Repository Overview By: Ellie Hojeily, updated 20 December 2025. Important! If you have any issues accessing or using the data from this repository or are interested in additional data or code, please contact [e-mail] or send me a message via LinkedIn. If you do not get a response within 1-2 weeks, please contact Scott Miller ([e-mail]) as an alternative. This repository contains supporting calibration files for Hojeily et al., 2025: "A Network Calibration Approach Improves the Accuracy and Long-Term Stability of a Low-Cost Air Quality Mesonet in New York City" (https://doi.org/10.1021/acsestair.5c00205) and Miller et al. 2025: "Combining Low-Cost Sensors with the New York State Mesonet for Continuous Fine-Scale Air Quality Monitoring in the New York City Metropolitan Area" (https://doi.org/10.1021/acsestair.5c00200). This repository supports version 1.3 of the New York State Mesonet air quality network. Computing Information Python version 3.10 was used during this project via VSCode and JupyterHub. The following packages and version numbers were used during this project: Package Version Use Jupyter 1.1.1 Python intepretor joblib 1.3.2 Handling random forest files matplotlib 3.8.3 Visualization numpy 1.26.4 Data processing pandas 2.2.1 Data processing scikit-learn 1.0.2 Model development & verification scipy 1.11.4 Model development & verification statsmodels 0.14.1 Model development & verification pytz 2023.3.post1 Data processing If you run into any issues using the provided code in supporting_functions.py, please check that you are running the correct versions of these core Python packages. Available Files and Corresponding Metadata 'slr_transfer_function_stats.csv' hosts the performance of the SLR transfer functions verified on the keystone package. The model_eval function from supporting_functions.py was used to score models. Column name Information Package Package serial number, all start with AQA20 Sensor Pollutant/sensor that is being adjusted (i.e., NO, NO2, O3NO2, etc.) Calibration_Number Co-location in which package is being corrected. C1 = co-location 1, C2 = co-location 2 R2 Coefficient of determination w/ keystone package after Transfer function is applied. MAE Mean absolute error w/ keystone package after Transfer function is applied. Units are in volts from the trace gases while pm2.5 is in µg/m³. RMSE Root-mean-square error w/ keystone package after Transfer function is applied. Units are in volts from the trace gases while pm2.5 is in µg/m³. m Slope w/ keystone package after Transfer function is applied. y-int Offset w/ keystone package after Transfer function is applied. Units are in volts from the trace gases while pm2.5 is in µg/m³. mbe Mean bias error w/ keystone package after Transfer function is applied. Units are in volts from the trace gases while pm2.5 is in µg/m³. '_network_SLR_coefficients' are the Transfer Function slope and offset values trained during co-location at Queens College w/ the Keystone package. These files are organized by pollutant. Column name Information Offset y-int term. Units are in volts from the trace gases while pm2.5 is in µg/m³. Slope coefficient term. Sensor Package serial number, all start with AQA20 Co-location number Co-location in which package is being corrected. C1 = co-location 1, C2 = co-location 2 'aqa2020_keystone_data.csv' is the calibrated data from the keystone package, AQA2020. Column name Information Datetime (UTC) Hourly datetime in UTC _MLR Calibrated using MLR. Units are in ppb for the trace gases while pm2.5 is in µg/m³. _RF Calibrated using RF. Units are in ppb for the trace gases while pm2.5 is in µg/m³. _Hybrid Calibrated using Hybrid. Units are in ppb for the trace gases while pm2.5 is in µg/m³. T_ambient Ambient temperature measured by package in degrees celsius RH_ambient Ambient relative humidity measured by package as a percent Period Indicates if a row was used for training or testing 'keystone_package_raw.csv' is the raw data from the keystone package, AQA2020. The file includes meteorological variables measured by the package. No data has been qc'd from this file outside of a simple despiking algorithm and resampling from 5-second data to hourly data. Column name Information Datetime Datetime in UTC _aux Auxilary voltage from electrochemical sensors _we Working voltage from electrochemical sensors _sensor Estimated voltage concentration (working-auxilary) from electrochemical sensors in units of Volts, copy of PM2.5 as reported by the Plantower PMS5003 in units of µg/m³ T_ambient Ambient temperature measured by package in degrees celsius RH_ambient Ambient relative humidity measured by package as a percent T_board Internal sensor package temperature in degrees celsius, this is can occasionally deviate from T_ambient and T_manifold T_manifold Manifold temperature measured by package in degrees celsius, this is typically identical to T_ambient RH_manifold Manifold relative humidity measured by package as a percent, this is typically identical to RH_ambient pres_board Internal sensor package pressure in millibars Tdew_manifold Manifold dewpoint in degrees celsius pm25_env, pm25_std PM2.5 as reported by the Plantower PMS5003 in units of µg/m³ 'supporting_functions.py' are some example python functions that were used in this work, including functions to train & verify models pollutant+'_colocation_networkstats.csv' host the performance of each package, organized by colocation (1, 2, and wildfire for PM2.5) & pollutant, when verified on regulatory instruments from the NYSDEC Queens College site. The model_eval function from supporting_functions.py was used to score models. Column name Information Package ID Package serial number, all start with AQA20 r2 Coefficient of determination w/ reference instrument after calibration algorithm is applied. rmse Root-mean-square error w/ reference instrument after calibration algorithm is applied. Units are in ppb from the trace gases while pm2.5 is in µg/m³. pbias The percent bias w/ reference instrument after calibration algorithm is applied. mae Mean absolute error w/ reference instrument after calibration algorithm is applied. Units are in ppb from the trace gases while pm2.5 is in µg/m³. mbe Mean bias error w/ reference instrument after calibration algorithm is applied. Units are in ppb from the trace gases while pm2.5 is in µg/m³. nmae The normalized mean absolute error w/ reference instrument after calibration algorithm is applied. Normalized using the mean reference concentration. nrmse The normalized root-mean-square error w/ reference instrument after calibration algorithm is applied. Normalized using the mean reference concentration. nmbe The normalized mean bias error w/ reference instrument after calibration algorithm is applied. Normalized using the mean reference concentration. nhours The number of co-location hours. ref_avg The average reference concentration of the target gas during the respective co-location. pollutant+'_NCM_MLR.txt' are files containing the MLR NCM models. A custom Python function was used to generate (generate_coefficients) and apply the coefficients (calibrate) and is included in the repository in supporting_functions.py. Column name Information model_type 'net' indicating the Network Calibration Algorithm coef_const Y-intercept coef_+variable Coefficient for specific variable. For example, 'coef_pm25_sensor' represents the coefficient for the PM2.5 sensor in the PM2.5 MLR model. pollutant+'_NCM_RF.joblib' are files containing the RF NCM models. The RF models are stored as joblib files, requiring the joblib Python package to access. Other Relevant Datasets: Reference/Regulatory Instrument Data Reference instrument data should be retrieved from the EPA via their AQS repository (https://www.epa.gov/aqs/obtaining-aqs-data). The supplemental information in Hojeily et al. (2025) details the specific site ids used for nearby field evaluations. Reference data from the calibration site, the NYSDEC Queens College site, can be retrieved via the following site id: 36-081-0124. Relevant AQS parameter codes for the EPA AQS data can be found below. Pollutant AQS Parameter Code/ID PM2.5 88101 (PM2.5 Local Conditions) O3 44201 NO2 42602 NO 42603 CO 42101 New York State Mesonet Meteorological and Air Quality Data Requests for New York State Mesonet meteorological data must be made via the mesonet website (https://www2.nysmesonet.org/weather/requestdata). The air quality data from the mesonet field sites will be made available on the mesonet website. Acknowledgements: This research was supported by the New York State Energy Research and Development Authority (NYSERDA, https://www.nyserda.ny.gov/) award 156228. The opinions expressed in this report do not necessarily reflect those of NYSERDA or the State of New York.