A User Study to Evaluate Game Engine Architecture Visualisation

Game engines are software systems divided into subsystems which provide functionality that is common in most games, such as graphics, audio and input device processing. Due to their complexity and high coupling, game engines are difficult to understand and maintain. To mitigate this issue in software systems in general, researchers have applied various techniques, among them, software architecture recovery, along with the use of metamodels and visualisation of the recovered models. However, it is not clear how much the use of models can actually help developers. In the work, we conduct a user study with 16 developers to measure whether the use of game engine architectural models can help them perform nine software architecture tasks. Seven tasks were related to architecture understanding, and the remaining two were related to impact analysis. We found out the use of models has a small, yet positive influence on task completion time for both kinds of tasks. We did not find a significant difference in task correctness.