Testing Dynamically Adaptive Systems: A Preliminary Literature Review

Context. Dynamically Adaptive Systems (DAS) encompasses a large class of systems that adapt their behaviours in response to changes in their context and/or predefined optimisation goals. Advanced software architectures support these adaptations robustly and efficiently at runtime. Yet, testing such architectures remains challenging as tests also need to cope with contextual and behavioural changes (addition, modification or removal) to be useful and prevent undesired behaviours to be deployed.
Objective. The objective of this study is to identify what strategies are used to test DAS and raise evidence on techniques and tools that achieve high defect detection even in unpredictable contexts.
Method. We performed a literature review of 389 studies published from the year 2001 until 2019. These studies cover the testing of Dynamic Software Product Lines, Context-Aware Systems and Dynamically Adaptive Systems.
Results. Our analysis comprised three fundamental aspects of testing strategies for DAS: executing the test cases, the test methods, and the test tools. Our preliminary findings indicate that the literature offers a large number of strategies to cope with such aspects. However, the studies analyzed do not show how to design or execute the test cases in practice.