A Data-set about Surface Damage Identification for Heritage Site Protection: A Mobile Crowdsensing Solution Based on Deep Learning

The dataset has been collected to address the general problem of built heritage protection against both deterioration and loss. In order to continuously monitor and update the structural health status, a crowd-sensing solution based on powerful and automatic deep learning technique is proposed. The aim of this solution is to get rid of the limitations of manual and visual damage detection methods that are costly and time consuming. Instead, automatic visual inspection for damage detection on walls is efficiently and effectively performed using an embedded Convolutional Neural Network (CNN).
This CNN detects the most frequent types of surface damage on wall photos. The study has been conducted in the Kasbah of Algiers where the four following types of damage have been considered: Efflorescence, Spall, Crack, and Mold. The CNN is designed and trained to be integrated into a mobile application for a participatory crowd-sensing solution. The application should be widely and freely deployed, so that any user can take a picture of a suspected damaged wall, and get an instant and automatic diagnosis, through the embedded CNN.
In this context, we have used MobileNetV2 with a transfer learning approach. A set of real images have been collected and manually annotated, and have been used for training, validation, and test. Extensive experiments have been conducted in order to assess the efficiency and the effectiveness of the proposed solution, using a 5 fold cross validation procedure.
Obtained results show in particular a mean weighted average precision of 0.868±0.00862 (with a 99% of confidence level) and a mean weighted average recall of 0.84±0.00729 (with a 99% of confidence level). Obtained results show that our method remains effective even when using a small network and medium to low resolution images.

In order to provide the CNN to be deployed, as well as the dataset used for the test, to the researchers, we have made them available online. Also a source code to perform a quick test on Google Collaboratry is available. The model has been saved in Keras "h5" format, and also in a TensorFlow Lite "tflite" format, ready to be deployed on a mobile application.

Files published are:
- "Test_Kasbah_damage.ipynb": python source code to perform the test, in Google Collaboratry.
- "multi_label_CV_final.h5": the CNN saved in a h5 keras format, to be used for the test, and other purposes.
- "test_multi_label.zip": the test dataset used in the paper, saved in a zip compressed file (634 images).
- "multi_label_CV_final.tflite" : the CNN saved in a tflite format, ready for deployment in a mobile application.