Dataset for AI-based optical-thermal video data fusion for near real-time blade segmentation in normal wind turbine operation

Blade damage inspection without stopping the normal operation of wind turbines has significant economic value. This study proposes an AI-based method AQUADA-Seg to segment the images of blades from complex backgrounds by fusing optical and thermal videos taken from normal operating wind turbines. The method follows an encoder-decoder architecture and uses both optical and thermal videos to overcome the challenges associated with field application. A memory is designed between the encoder and decoder to improve the method’s performance by utilizing time history information in the videos to achieve temporal complementarity. The designed memory shares information between optical and thermal modalities to achieve multimodal complementarity. We collected a large-scale dataset, i.e., 100 video pairs and over 55,000 images, of optical-thermal videos of blades in operational wind turbines to train and test the method. Experimental results show that AQUADA-Seg: i) achieves near real-time thermal-optical blade video segmentation and can analyze videos with complex backgrounds in real-world field applications; ii) achieves 0.996 and 0.981 MIoU on optical and thermal videos, respectively, outperforming state-of-the-art methods, particularly in the videos with complex backgrounds. This study provides an essential step towards automated blade damage detection using computer vision without stopping the normal operation of wind turbines.

无需中断风力涡轮机正常运行即可对叶片进行损伤检测，此举具有显著的经济价值。本研究提出了一种基于人工智能的方法AQUADA-Seg，通过融合从正常运行的风力涡轮机中捕获的光学和热成像视频，对叶片图像进行分割。该方法遵循编码器-解码器架构，并利用光学和热成像视频克服了现场应用中相关的挑战。在编码器和解码器之间设计了一种记忆机制，通过利用视频中的时间历史信息实现时间互补性，从而提升方法性能。该记忆机制在光学和热成像模态之间共享信息，以实现多模态互补性。我们收集了一个大规模数据集，即100对视频和超过55,000张叶片在运行风力涡轮机中的光学-热成像视频，用于训练和测试该方法。实验结果表明，AQUADA-Seg：i) 实现了近乎实时的热光学叶片视频分割，并能在现实世界的现场应用中分析具有复杂背景的视频；ii) 在光学和热成像视频上分别实现了0.996和0.981的MIoU，超越了现有方法，特别是在具有复杂背景的视频中表现尤为出色。本研究为利用计算机视觉实现风力涡轮机叶片损伤检测的自动化提供了关键一步，而无需中断其正常运行。

搜集汇总

背景与挑战

背景概述

该数据集是一个大规模光学-热成像视频数据集，包含100对视频和超过55,000张图像，专门用于训练和测试AI方法（如AQUADA-Seg）以实现风力涡轮机叶片的近实时分割。数据集支持多模态数据融合，旨在在风力涡轮机不停机的情况下，从复杂背景中准确分割叶片图像，应用于自动化损伤检测。