Machine Eye for Defects: Machine Learning-Based Solution to Identify and Characterize Topological Defects in Textured Images of Nematic Materials

Our paper has been published on Phys. Rev. Res. (doi: 10.1103/PhysRevResearch.6.013259)

Our preprint paper is also avilable at arXiv(https://arxiv.org/abs/2310.06406), here is the abstract of our paper:

Topological defects play a key role in the structures and dynamics of liquid crystals (LCs) and other ordered systems. There is a recent interest in studying defects in different biological systems with distinct textures. However, a robust method to directly recognize defects and extract their structural features from various traditional and nontraditional nematic systems remains challenging to date. Here we present a machine learning solution, termed Machine Eye for Defects (MED), for automated defect analysis in images with diverse nematic textures. MED seamlessly integrates state-of-the-art object detection networks, Segment Anything Model, and vision transformer algorithms with tailored computer vision techniques. We show that MED can accurately identify the positions, winding numbers, and orientations of ±1/2 defects across distinct cellular contours, sparse vector fields of nematic directors, actin filaments, microtubules, and simulation images of Gay–Berne particles. MED performs faster than conventional defect detection method and can achieve over 90% accuracy on recognizing ±1/2 defects and their orientations from vector fields and experimental tissue images. We further demonstrate that MED can identify defect types that are not included in the training data, such as giant-core defects and defects with higher winding number. Remarkably, MED can provide correct structural information about ±1 defects. As such, MED stands poised to transform studies of diverse ordered systems by providing automated, rapid, accurate, and insightful defect analysis.

Repository Organization

Trained Models.zip

This directory is integral for model deployment and houses all relevant pre-trained models.

plus_vit_vecUV.pt: Pre-trained model for the Plus Transformer variant.

minus_vit_theR.pt: Pre-trained model for the Minus Transformer variant.

nanodet-plus-m_416-halfenhance: A sub-directory containing all files associated with the trained Nanodet-Plus model.

configs: Configuration files for training procedures.

Training Data.zip

This directory contains all datasets used for the training of Nanodet-Plus, Plus Transformer, and Minus Transformer models.

Code.zip

This directory features the implementation details and example use-cases showcased in Figure 2 and Figure 3c of our associated paper. The directory also includes code corresponding to the specific versions of Nanodet-Plus and SAM models cited in our study.

nanodet: Code in this folder is adapted from RangiLyu/nanodet (https://github.com/RangiLyu/nanodet). We have included the exact version used for compatibility.

segment_anything: Code sourced from Facebook Research's segment-anything (https://github.com/facebookresearch/segment-anything). The specific version used is included for compatibility.

Fig2: Code for predicting topological defects in tissue cell images, citing the following reference: T. B. Saw et al., Nature 544, 212 (2017).

Fig3c: Code for predicting topological defects in microtubules images, citing the following reference: M. Golden et al., Sci. Adv. 9, eabq6120 (2023).

Initialization Steps

Before executing any code, please ensure the following:

All files in the Trained Models directory must be available.

Download the checkpoint sam_vit_l_0b3195.pth from Facebook Research's segment-anything. (https://github.com/facebookresearch/segment-anything)

Acknowledgments

RangiLyu/nanodet (https://github.com/RangiLyu/nanodet)

Facebook Research's segment-anything (https://github.com/facebookresearch/segment-anything)

For further inquiries or issue reporting, you may contact us via email.

Contact Information: [e-mail]