"GUI(Graphical User Interface) Elements in Video Games-Shooter Games"

"\u2014The design and testing of user interfaces (UIs) in video games is an iterative and time-intensive process, often requiring large development teams\u2014a challenge for smaller studios operating under tight deadlines. With many games recently missing their original release, time is already a luxury for many game studios. Therefore, a new, automated method of testing UI design in video games is needed, with deep learning being a promising potential solution. This research aims to introduce the use of deep learning image detection for testing the UI elements of games for recognisability. A dataset of annotated screenshots from shooter video games was created, covering twelve games. The dataset was analysed with exploratory data analysis (EDA). A pre-trained Faster R-CNN model was trained on the dataset and hyperparameter tuning was carried out, improving mAP over the initial model by 0.094. The model achieved a mean average precision (mAP) at IoUs between 0.50 and 0.95 of 0.815 which is reasonably accurate. A real-time system utilising this model was created which detects and identifies UI elements in real time and displays them in a larger format which can aid visually impaired users. The implications of this research are that object detection techniques utilising CNNs can be used to test video game UIs during the design process with a sufficiently robust annotated dataset. This research also shows that object detection can be used to design systems which could improve the accessibility of video games for visually impaired users."