M3-OCTA:Leveraging Multimodal Fusion for Enhanced Diagnosis of Multiple Retinal Diseases in Ultra-wide OCTA

Ultra-wide optical coherence tomography angiography (UW-OCTA) is an emerging imaging technique that offers significant advantages over traditional OCTA by providing an exceptionally wide scanning range of up to 24 x 20 mm^{2}, covering both the anterior and posterior regions of the retina. However, the currently accessible UW-OCTA datasets suffer from limited comprehensive hierarchical information and corresponding disease annotations. To address this limitation, we have curated the pioneering M3OCTA dataset, which is the first multimodal (i.e., multilayer), multi-disease, and widest field-of-view UW-OCTA dataset. Furthermore, the effective utilization of multi-layer ultra-wide ocular vasculature information from UW-OCTA remains underdeveloped. To tackle this challenge, we propose the first cross-modal fusion framework that leverages multi-modal information for diagnosing multiple diseases. Through extensive experiments conducted on our openly available M3OCTA dataset, we demonstrate the effectiveness and superior performance of our method, both in fixed and varying modalities settings. The construction of the M3OCTA dataset, the first multimodal OCTA dataset encompassing multiple diseases, aims to advance research in the ophthalmic image analysis community.

Our proposed M3OCTA is the first multi-modal based ultra-wide retinal OCTA dataset, involving 1637 scans from 1046 eyes of 620 individuals imaged in Zigong First People’s Hospital through 24×20 scan mode. Specifically, 1067 scans contains choroid large vessel image; images of 1310 scans from 496 people are labeled as six classes in multi-label setting, including healthy, diabetic retinopathy (DR), diabetic macular edema (DME), Retinal Vein Occlusion (RVO), Hypertension (HBP) and Vitreous Hemorrhage (VH), and then split into train, validation and test set as 6:2:2. The remaining unlabeled data are only used in the pretraining step. Details of our M3OCTA and other public ones are listed in Table.1. Compared with others, M3OCTA dataset demonstrates superiorities in several aspects including the number of modalities, number of patients, image resolution, and FOV.

You can request this dataset through signing the attached agreement. The download link will send to you.