BadSTR: Backdoor Attack on Scene Text Recognition in IoT

Recent researches have shown that non-sequential tasks based on deep neural networks (DNN), such as image classification and object detection, are vulnerable to backdoor attacks, leading to incorrect model predictions. As a crucial task in computer vision, Scene Text Recognition (STR) is widely used in IoT fields such as intelligent transportation systems and intelligent surveillance. Therefore, a high degree of security is needed to ensure the accuracy of the system for text recognition. However, there are currently no studies on STR backdoor attacks. We make the first attempt to launch backdoor attack on the STR models, and successfully embed a backdoor into STR models using Patch-Based Attack. However, the experimental results indicate that this attack method has the flaw of lacking attack robustness. To explore more robust backdoor attack on STR, we further propose BadSTR, a novel backdoor attack method that is more applicable to STR, in which we use character sequence in an image as the trigger. We perform extensive experiments on eight benchmark datasets to validate the feasibility of BadSTR. Furthermore, to evaluate the effectiveness and robustness of our proposed attack method, we introduce the mean attack success rate and the variance of attack success rate as metrics. The experimental results show that our proposed BadSTR achieves an attack success rate of over 80% in all model dataset combinations, and is more effective and robust than the Patch-Based Backdoor Attack.

近期研究表明，基于深度神经网络（Deep Neural Networks, DNN）的非序列任务（如图像分类、目标检测）易受后门攻击，进而导致模型预测结果出错。作为计算机视觉领域的核心任务之一，场景文本识别（Scene Text Recognition, STR）已被广泛应用于智能交通系统、智能监控等物联网（Internet of Things, IoT）场景，因此亟需构建高安全等级的系统以保障文本识别任务的准确性。然而，目前尚无针对场景文本识别模型的后门攻击相关研究。本研究首次尝试针对场景文本识别模型开展后门攻击研究，并通过基于补丁的攻击（Patch-Based Attack）方法成功将后门植入场景文本识别模型中。但实验结果显示，该攻击方法存在攻击鲁棒性不足的缺陷。为探索适用于场景文本识别任务的高鲁棒性后门攻击方法，本研究进一步提出BadSTR——一种专为场景文本识别设计的新型后门攻击方法，该方法以图像中的字符序列作为攻击触发源。我们在8个基准数据集上开展了大量实验，以验证BadSTR方法的可行性。此外，为评估所提攻击方法的有效性与鲁棒性，本研究引入攻击成功率均值与攻击成功率方差作为评估指标。实验结果表明，所提BadSTR方法在所有模型-数据集组合下的攻击成功率均超过80%，且相较于基于补丁的后门攻击方法，具备更优异的攻击效果与鲁棒性。

搜集汇总

背景与挑战

背景概述

该数据集专注于物联网场景中的场景文本识别（STR）模型后门攻击研究，首次提出BadSTR方法，使用字符序列作为触发器以提高攻击鲁棒性。实验在八个基准数据集上验证了其有效性，攻击成功率超过80%，相比基于补丁的攻击方法更稳健。