Development of AI-assisted microscopy frameworks through realistic simulation in pySTED

The integration of artificial intelligence into microscopy systems significantly enhances performance, optimizing both the image acquisition and analysis phases. Development of artificial intelligence (AI)-assisted super-resolution microscopy is often limited by the access to large biological datasets, as well as by the difficulties to benchmark and compare approaches on heterogeneous samples. We demonstrate the benefits of a realistic STED simulation platform, pySTED, for the development and deployment of AI-strategies for super-resolution microscopy. The simulation environment provided by pySTED allows the augmentation of data for the training of deep neural networks, the development of online optimization strategies, and the training of reinforcement learning models, that can be deployed successfully on a real microscope.

The dataset contains all of the acquired images during the optimization sessions that were performed on the microscope. The dataset also includes all of the trained models that were developed in this paper.