MonuMAI Citizen Science Project Dataset

The latest Deep Learning (DL) models for detection and classification have achieved an unprecedented performance over classical machine learning algorithms. However, DL models are black-box methods hard to debug, interpret, and certify. DL alone cannot provide explanations that can be validated by a non technical audience. In contrast, symbolic AI systems that convert concepts into rules or symbols -- such as knowledge graphs -- are easier to explain. However, they present lower generalisation and scaling capabilities. A very important challenge is to fuse DL representations with expert knowledge. One way to address this challenge, as well as the performance-explainability trade-off is by leveraging the best of both streams without obviating domain expert knowledge. We tackle such problem by considering the symbolic knowledge is expressed in form of a domain expert knowledge graph. We present the eXplainable Neural-symbolic learning (X-NeSyL) methodology, designed to learn both symbolic and deep representations, together with an explainability metric to assess the level of alignment of machine and human expert explanations. MonuMAI database contains 1.514 RGB images of monument facades of four architectural styles Hispanic-Muslim, Gothic, Renaissance, and Baroque. Where the used images for building Monumai were selected so that the monument is centered, fill most of the image, and have high quality so the monument features can be observable. MonuMAI dataset includes 6650 annotated key elements distributed into the fifteen architectural element classes. A statistic analysis of these elements is shown in the Table.Detection annotation is provided in Pascal VOC format XML. MonuMAI app Visit https://monumai.ugr.es Terms of use The annotation in this dataset along with the images are licensed under a Creative Commons Attribution 4.0 License.当前用于检测与分类任务的深度学习（Deep Learning，DL）模型，相较经典机器学习算法已取得前所未有的性能表现。然而，深度学习模型属于难以调试、解释与验证的黑盒方法，仅靠深度学习无法提供可被非技术受众验证的解释。与之相对，将概念转化为规则或符号的符号人工智能系统——例如知识图谱（knowledge graphs）——更易于解释，但此类系统的泛化与扩展能力相对较弱。将深度学习表征与领域专家知识相融合是一项极具挑战性的关键问题，而兼顾性能与可解释性权衡的解决思路，便是在不忽视领域专家知识的前提下，充分利用两者的优势。 本研究针对该问题，以领域专家知识图谱的形式表征符号知识，提出了可解释神经符号学习（eXplainable Neural-symbolic learning，X-NeSyL）方法，该方法旨在同时学习符号表征与深度学习表征，并配套提出了用于评估机器解释与人类专家解释对齐程度的可解释性度量指标。 MonuMAI数据库包含1.514张涵盖四种建筑风格——西班牙-伊斯兰式、哥特式、文艺复兴式与巴洛克式——的建筑立面RGB图像。构建MonuMAI数据集时所选用的图像均经过严格筛选，确保建筑主体居中、占据画面大部分区域且画质清晰，以便建筑特征可被清晰观测。 MonuMAI数据集包含6650个标注关键元素，分为15个建筑元素类别。相关元素的统计分析详见下表。检测标注采用Pascal VOC格式的XML文件进行存储。 MonuMAI应用 访问网址：https://monumai.ugr.es 使用条款 本数据集的标注数据与图像均采用知识共享署名4.0（Creative Commons Attribution 4.0）许可协议进行授权。