DeepMicroscopy: A Deep Learning Framework for Multiscale and Multimodal Microscopy Image Translation

DeepMicroscopy is a PyTorch implementation of Multimodal Unsupervised Image-to-Image Translation (MUNIT) specifically designed for microscopy imaging applications. This framework addresses the challenge of integrating heterogeneous microscopy datasets from different modalities and resolutions into high-fidelity digital representations of porous materials. Key Features:- Seamless integration of unpaired imaging datasets from different modalities (e.g., micro-CT and SEM)- Support for both 2D and 3D microscopy data processing- Super-resolution capability with up to 6.4× resolution enhancement- Robust many-to-many mappings that preserve inherent uncertainty and diversity- Style-based generation for multiple realizations of the same input Applications:- Low-resolution to high-resolution micro-CT translation (6.4μm → 1.0μm, 26.88μm → 4.2μm)- Cross-modality translation between micro-CT and SEM imaging (1.0μm → 200nm)- Multiscale data fusion for comprehensive material characterization- Cross-sample generalization for unseen porous media samples The framework has been validated on sandstone and carbonate rock samples, demonstrating its ability to enhance the characterization of heterogeneous materials and uncover new insights into pore-scale physical processes. This versatile and scalable approach holds broad applicability across disciplines such as geoscience, materials science, and biomedical imaging. This repository contains:- Complete source code for training and testing- Pre-configured YAML files for different translation tasks- Scripts for data preparation and model evaluation- Example notebooks demonstrating usage- Comprehensive documentation