Drone-based Optical and Thermal Videos of Rotor Blades Taken in Normal Wind Turbine Operation

Blade damage inspection without stopping the normal operation of wind turbines has significant economic value. This study proposes an AI-based method AQUADA-Seg to segment the images of blades from complex backgrounds by fusing optical and thermal videos taken from normal operating wind turbines. The method follows an encoder-decoder architecture and uses both optical and thermal videos to overcome the challenges associated with field application. A memory is designed between the encoder and decoder to improve the method’s performance by utilizing time history information in the videos to achieve temporal complementarity. The designed memory shares information between optical and thermal modalities to achieve multimodal complementarity. We collected a large-scale dataset, i.e., 100 video pairs and over 55,000 images, of optical-thermal videos of blades in operational wind turbines to train and test the method. Experimental results show that AQUADA-Seg: i) achieves near real-time thermal-optical blade video segmentation and can analyze videos with complex backgrounds in real-world field applications; ii) achieves 0.996 and 0.981 MIoU on optical and thermal videos, respectively, outperforming state-of-the-art methods, particularly in the videos with complex backgrounds. This study provides an essential step towards automated blade damage detection using computer vision without stopping the normal operation of wind turbines.

不中断风力发电机组正常运行的风电叶片损伤检测工作具备极高的经济价值。本研究提出一种基于人工智能的AQUADA-Seg方法，通过融合正常运行风电机组采集的光学与热成像视频，实现复杂背景下的叶片图像分割。该方法采用编码器-解码器（encoder-decoder）架构，同时利用光学与热成像视频以应对实际现场应用中的各类挑战。研究在编码器与解码器之间设计了记忆模块，通过利用视频中的时序历史信息实现时序互补，并在光学与热成像模态间共享信息以达成多模态互补，进而提升模型性能。为训练与测试该方法，我们采集了一套大规模数据集：包含运行中风电机组叶片的光学-热成像视频对共100组，对应图像超55000张。实验结果表明，AQUADA-Seg具备以下优势：其一，可实现近乎实时的光学-热成像叶片视频分割，能够在实际现场复杂背景场景下完成视频分析；其二，在光学与热成像视频上分别取得0.996与0.981的平均交并比（MIoU），性能优于现有最优方法，在复杂背景视频场景中优势尤为显著。本研究为无需中断风电机组正常运行、基于计算机视觉的风电叶片自动化损伤检测技术迈出了关键一步。

搜集汇总

背景与挑战

背景概述

该数据集是一个基于无人机拍摄的光学-热风力涡轮机叶片视频的大规模数据集，包含36对视频和超过20,778张图像，用于训练和测试AI方法AQUADA-Seg，以实现风力涡轮机叶片的实时分割和损伤检测。