CLTS-GAN: Color-Lighting-Texture-Specular Reflection Augmentation for Colonoscopy

This is the models as well as results for CLTS-GAN, a deep learning model that disentangles color and lighting and texture and specular information. The results for the model as well as an augmented polyp dataset are contained here.

Abstract:

Automated analysis of optical colonoscopy (OC) video frames (to assist endoscopists during OC) is challenging due to variations in color, lighting, texture, and specular reflections. Previous methods either remove some of these variations via preprocessing (making pipelines cumbersome) or add diverse training data with annotations (but expensive and time-consuming). We present CLTS-GAN, a new deep learning model that gives fine control over color, lighting, texture, and specular reflection synthesis for OC video frames. We show that adding these colonoscopy-specific augmentations to the training data can improve state-of-the-art polyp detection/segmentation methods as well as drive next generation of OC simulators for training medical students. You can find the code and additional detail about CLTS-GAN via our Computation Endoscopy Platform at https://github.com/nadeemlab/CEP