Advancing Bag-of-Visual-Words Representations for Lesion Classification in Retinal Images

Diabetic Retinopathy (DR) is a complication of diabetes that can lead to blindness if not discovered in time. Automated screening algorithms have the potential to improve identification of patients who need further medical attention. However, the algorithms must be accurate in order to be useful for clinical application. The bag-of-visual-words (BoVW) algorithm utilizes a maximum-margin binary classifier in a flexible framework that is able to detect the most common DR-related lesions such as microaneurysms, cotton-wool spots and hard exudates. BoVW was developed to bypass the need for pre- and post-processing of the retinographic images, as well as including specific ad hoc techniques for identification of each type of lesion. An extensive evaluation of the BoVW model, using three large retinograph datasets (DR1, DR2 and Messidor) with different resolution and collected by different healthcare personnel, was performed. The results demonstrate that the BoVW classification approach can identify different lesions within an image without having to utilize different algorithms for each lesion and therefore reduces processing time and provides a more flexible diagnostic system. BoVW is based on sparse detection with a Speeded-Up Robust Features (SURF) local descriptor and semi-soft coding with max pooling. The best BoVW representation for retinal image classification was an area under the receiver operating characteristic curve (AUC-ROC) of 97.8% (exudates) and 93.5% (red lesions) when applying a cross-dataset validation protocol. To assess the accuracy for detecting cases, which require referral within one year, the sparse extraction technique associated with semi-soft coding and max pooling obtained an AUC of 94.2±2.0%, outperforming current methods. These results indicate that, for retinal image classification tasks in clinical practice, BoVW is equal and, in some instances, surpasses results obtained using dense detection of the low-level descriptors, which has gained almost unanimous acceptance in recent computer vision and image processing literature.