Denoising Framework for X-ray Absorption Spectroscopy Data

================================================================================XAS DENOISING BENCHMARK - SUPPLEMENTARY MATERIALS================================================================================ This archive contains the code, datasets, and trained models used in the manuscript "Denoising Framework for X-ray Absorption Spectroscopy Data". IMPORTANT: For the most up-to-date version of the XASDenoise code, please visit:https://github.com/tomasaidukas/XASDenoise AUTHORS:--------Tomas Aidukas† ([e-mail])Ilnura Usmanova† ([e-mail])Benjamín Béjar Haro ([e-mail])Maarten Nachtegaal ([e-mail])Adam H. Clark ([e-mail]) - corresponding author † Shared first authorship AFFILIATIONS:-------------- Center for Photon Science, Paul Scherrer Institute, Villigen, Switzerland- Swiss Data Science Center, Paul Scherrer Institute, Villigen, Switzerland- Department of Chemistry and Applied Biosciences, ETH Zürich, Zürich, Switzerland ================================================================================REPOSITORY STRUCTURE================================================================================ XASDenoiseManuscriptFiles/├── BenchmarkingData/ Training/test datasets and trained models├── DenoisingCode/XASDenoise/ Python package for XAS denoising└── Scripts/ Training, benchmarking, and visualization scripts ================================================================================OVERVIEW================================================================================ This repository provides: 1. XASDenoise Package: A comprehensive Python library implementing multiple denoising methods for X-ray Absorption Spectroscopy (XAS) data, including traditional filters, Gaussian processes, and deep learning-based autoencoders. 2. Benchmark Datasets: Curated training and test datasets with both uniformly and non-uniformly sampled XAS spectra from diverse materials. 3. Trained Models: Pre-trained autoencoder models for immediate use on XAS data. 4. Reproducible Scripts: Complete workflow for reproducing all results from the manuscript. ================================================================================INSTALLATION================================================================================ REQUIREMENTS:-------------- Python >= 3.10 (tested with Python 3.12.11)- pip package manager SETUP:------1. Create a Python virtual environment: python -m venv xas_denoise_env source xas_denoise_env/bin/activate (On Windows: xas_denoise_env\Scripts\activate) Optionally, you can clone or download the latest code from GitHub: https://github.com/tomasaidukas/XASDenoise 2. Install the XASDenoise package: cd DenoisingCode/XASDenoise/ pip install -e . This will install all required dependencies including NumPy, SciPy, scikit-learn, PyTorch, and visualization libraries. ================================================================================DATASETS================================================================================ The BenchmarkingData/ folder contains HDF5 files with pre-processed XAS spectra(normalized, cropped, etc.). All datasets are split into non-overlapping training and test sets. UNIFORMLY SAMPLED DATA (~5000 points per spectrum):----------------------------------------------------- xas_denoiser_training_database_uniformly_sampled_train.h5 (Training set)- xas_denoiser_training_database_uniformly_sampled_test.h5 (Test set) NON-UNIFORMLY SAMPLED DATA (~500 points per spectrum):-------------------------------------------------------Mimics typical step-scan acquisition with higher density near absorption edges:- xas_denoiser_training_database_nonuniformly_sampled_train.h5 (Training set)- xas_denoiser_training_database_nonuniformly_sampled_test.h5 (Test set) PRE-TRAINED MODELS:-------------------- autoencoder_models/xas_denoiser_model_noise2noise_uniformly_sampled_notnormalized.pth- autoencoder_models/xas_denoiser_model_noise2noise_nonuniformly_sampled_notnormalized.pth ================================================================================REPRODUCING RESULTS================================================================================ Follow these steps to reproduce all results from the manuscript: STEP 1: Train Autoencoder Models (Optional)--------------------------------------------Pre-trained models are provided, but you can retrain them: cd Scripts/ # Train on uniformly sampled data python step1_train_autoencoder_denoiser_uniform_data.py # Train on non-uniformly sampled data python step1_train_autoencoder_denoiser_nonuniform_data.py Training typically takes several hours on a GPU (we used NVIDIA A100). STEP 2: Run Benchmarking-------------------------Apply all denoising methods to test datasets and evaluate performance: # Benchmark on uniformly sampled data python step2_benchmark_denoising_methods_uniform_data.py # Benchmark on non-uniformly sampled data python step2_benchmark_denoising_methods_nonuniform_data.py This generates pickle files containing denoising results and error metrics. STEP 3: Generate Figures-------------------------Create benchmarking figures from the manuscript using the provided Jupyter notebook: step3_plot_benchmarking_results.ipynb ================================================================================DENOISING METHODS IMPLEMENTED================================================================================ The XASDenoise package includes: TRADITIONAL FILTERS:--------------------- Butterworth filter- Gaussian filter- Savitzky-Golay filter- Median filter- Moving average- Wavelet denoising- Total variation denoising ADVANCED METHODS:------------------ Gaussian Process regression- Autoencoder (noise2noise training) DATA WARPING TECHNIQUES:------------------------- k-space warping (EXAFS region)- Smoothness-based warping- Combined XAS + Smoothness warping ================================================================================CITATION================================================================================ If you use this code or data in your research, please cite: Aidukas, T., Usmanova, I., Béjar Haro, B., Nachtegaal, M. & Clark, A. H. (2025). Denoising Framework for X-ray Absorption Spectroscopy Data. Journal of Synchrotron Radiation. In press. BibTeX format: @article{aidukas2025denoising, title={Denoising Framework for X-ray Absorption Spectroscopy Data}, author={Aidukas, Tomas and Usmanova, Ilnura and B{\'e}jar Haro, Benjam{\'i}n and Nachtegaal, Maarten and Clark, Adam H.}, journal={Journal of Synchrotron Radiation}, year={2025}, publisher={International Union of Crystallography}, note={In press}} ================================================================================LICENSE================================================================================ This code is released under the MIT License (see LICENSE file in DenoisingCode/XASDenoise/). ================================================================================CONTACT================================================================================ For questions or issues, please contact: - Tomas Aidukas: [e-mail]- Ilnura Usmanova: [e-mail]- Adam H. Clark (corresponding author): [e-mail] Or open an issue on the GitHub repository:https://github.com/tomasaidukas/XASDenoise/issues ================================================================================ACKNOWLEDGMENTS================================================================================ This work was conducted at the Paul Scherrer Institute in collaboration with the Swiss Data Science Center.