Fault Identification using Fault Injection with Domain Knowledge-Guided Reinforcement Learning

Safety assessment of Cyber-Physical Systems (CPS) requires a tremendous amount of effort as the complexity of cyber-physical systems is increasing. A well-known approach for the safety assessment of CPS is Fault Injection (FI). The goal of fault injection is to find a catastrophic fault that can fail the system by injecting faults into it. These catastrophic faults are less likely to happen, and finding it requires tremendous labor and cost. In this paper, we propose a Reinforcement Learning (RL)-based method to configure faults in the system under test automatically and find unknown faults. Reinforcement learning is a machine learning algorithm that learns by integrating dynamically with the environment. The proposed method provides a guideline to utilize high-level domain knowledge about the system under test for constructing the reinforcement learning agent and fault injection campaign. We mainly use the system (safety) specification to shape the reward function in the reinforcement learning agent. The reinforcement learning agent interacts with the system under test to find unknown faults. We test the proposed method on two use cases: adaptive cruise control and autonomous emergency braking in MATLAB/Simulink. We compare the proposed method with random-based fault injection. Our proposed method outperforms random-based FI in terms of severeness and number of found faults.