AccessFixer: Enhancing GUI Accessibility for Low Vision Users with R-GCN model

Here is the relevant dataset and open-source code for the article titled "AccessFixer: Enhancing GUI Accessibility for Low Vision Users with RGCN-based Method"

Introduction

In this work, we designed and implemented a tool, named AccessFixer, capable of fixing GUI accessibility issues in terms of small size, narrow interval, and low color contrast. It can ensure the consistency of GUI visual effects in the repair process, and will not produce inconsistent components.

Functions

Convert the GUIs screenshots to GUI-graphs. AccessFixer converts the GUIs into their corresponding GUI-graphs by parsing the XML file (in Graph_Presentation()), and meanwhile, our tool could remove all edges that connected the problematic nodes detected by Google Accessibility Scanner.

Pre-trained model based on Relational-Graph Convolutional Neural Network (R-GCN). Along with Capture_Signal(model), users can obtain the spectral signals of predicted links. Encoder and Decoder are responsible for generating new edges and scoring edges.

AccessFixer builds a mapping relationship between signal values and attribute values of GUI components in Attribute_Mapp(). Based on this mapping, it is possible to input a GUI to be repaired and generate repair strategies for its various components.

Environment

This tool can be run on an Android emulator based on Android 11.0 on a typical development machine, using Windows 11 with 2.4GHz core i7 CPU and 16 GB memory. The pre-trained RGCN model is run in PyCharm 4.5.4 with the packages of tensorflow, GraphConvolution, pandas, time, numpy, argparse, optimization, Parameter, and Module.

Installation

Using pip install package-name to install the required packages.

Configure GCN model training in the same directory.

Click the run button in PyCharm or use the command run-train.sh [configuration]

Frequently Asked Questions

Unable to install packages using pip

Answer: It could be the version of the pip install command. Detailed solution could be found in https://stackoverflow.com/questions/17869101/unable-to-install-pygame-using-pip/74229901#74229901

Some GUIs fail to build GUI-graphs

Answer: To create GII-graphs, the user is required to provide the GUI screenshot and the layout file parsed by UIAutomator.

Contact Information

If you have any questions about this tool, you can contact the author of this work at [e-mail]

Copyright

All copyright of the tool is owned by the author of the paper.