Deep-SMMT

Autonomous driving systems equipped with Deep Neural Networks (DNN) can control vehicles using sensor-acquired environmental data without human intervention. However, defects in DNNs have led to serious traffic accidents, highlighting the need for effective testing methods.  Most of the previous metamorphic testing approaches for autonomous driving systems generate new test cases by adding digital perturbations to original images. However, the test data generated by perturbations may differ from real-life scenes, which limit their defect detection abilities. To address this problem, we propose a metamorphic testing method Deep-SMMT (Deep Scene Mutation based Metamorphic Testing) for autonomous driving systems based on scene mutation. Firstly, we design a set of metamorphic relations for driving scenes, then mutate the scenes according to the metamorphic relations to generate derivative scenes. After that, it renders the corresponding images according to the derivative scenes subsequently. Finally, by checking out whether the output of the autonomous driving system for the generated images conforms to the metamorphic relations, we can detect defects. As an important part of the perception module in the autonomous driving system, the instance segmentation model is crucial to the reliability of the autonomous driving system. Experiments on Yolact (a fully convolutional model for instance segmentation) show that the proportion of images that can detect defects in the images generated by Deep-SMMT are 40.4% higher than DeepTest, and the KMNC (K-Multisection Neuron Coverage) are 0.1140(16.2%)and 0.0577(26.0%) higher than DeepTest.