Intelligent fatigue damage tracking and prognostics of composite structures utilizing raw images via interpretable deep learning

In recent years, prognostics has emerged as a focal point across various industries, gaining substantial attention for its prowess in optimizing maintenance schedules, elevating operational efficiency, and averting costly unplanned downtime. At the heart of prognostics lies the paramount parameter of Remaining Useful Life (RUL), signifying the critical time prior to system failure. Recent advancements in deep learning have ushered in the capability to forecast RUL by extracting features from a range of data formats: time-series, images, or sequences of images, representing one-, two-, or three-dimensional data respectively. While one-dimensional data has been frequently encountered in the literature, current approaches for predicting RUL from (sequences of) images still heavily rely on techniques, such as digital image correlation. These methods bring with them substantial computational overheads and intricate data acquisition strategies. Furthermore, the challenge of predicting RUL using high-dimensional data is exacerbated by the unreliable characteristics of deep learning models. In this regard, this study introduces an innovative deep learning architecture based on Transformers designed to tackle the dual challenges posed by high-dimensional data and the black-box nature of existing models. Building upon the remarkable achievements of Transformers in natural language processing and computer vision, their distinctive attention mechanism proves instrumental in realizing RUL predictions under uncertainty with just a sparse set of raw image sequences as input. By decomposing the spatiotemporal domain and harnessing the power of attention, our model adeptly sidesteps the black-box limitations of such architectures, thus enabling transparent and interpretable predictions. The proposed architecture is evaluated on an experimental dataset acquired by a real composite structure that is under fatigue loads with visible cracks that propagate with time. Given the unprocessed sequences of raw images as inputs, our model efficiently estimates the stochastic RUL. Concurrently, by leveraging the attention mechanism, we effectively demonstrate a strong correlation between the model's spatiotemporal focus to the sequences with the RUL making it, to the best of our knowledge, the first model to provide interpretable stochastic RUL predictions directly from sequential images of that nature.近年来，故障预后（prognostics）已成为各行业的核心关注焦点，因其在优化维护计划、提升运营效率及规避代价高昂的非计划停机方面的卓越能力而广受关注。故障预后的核心参数为剩余使用寿命（Remaining Useful Life，简称RUL），即系统发生故障前的关键剩余时长。近年来深度学习领域的进展，使得通过多种数据格式提取特征以预测RUL成为可能：这些数据格式分别为时序数据、图像数据或图像序列，对应一维、二维及三维数据。尽管一维数据在相关研究中被广泛采用，但当前基于（图像序列的）RUL预测方法仍高度依赖数字图像相关（digital image correlation）等技术，此类方法往往伴随高昂的计算开销与复杂的数据采集流程。此外，深度学习模型本身可靠性不足的特性，进一步加剧了高维数据下RUL预测的挑战。有鉴于此，本研究提出一种基于Transformer（Transformer）的创新深度学习架构，以应对高维数据与现有模型黑箱特性带来的双重挑战。依托Transformer在自然语言处理与计算机视觉领域的卓越成就，其独特的注意力机制能够仅以少量稀疏原始图像序列作为输入，在不确定性条件下实现RUL预测。通过分解时空域并借助注意力机制的能力，本模型巧妙规避了此类架构的黑箱局限，从而实现透明且可解释的预测结果。所提架构在承受疲劳载荷且伴随随时间扩展的可见裂纹的真实复合材料结构的实验数据集上完成了评估。以未经预处理的原始图像序列作为输入时，本模型可高效估算随机剩余使用寿命。同时，通过利用注意力机制，我们清晰证明了模型的时空关注焦点与RUL之间存在强相关性——据我们所知，这是首个能够直接从此类序列图像中生成可解释随机RUL预测的模型。