ICSE 2025 - Artifact

<b>Mobile Application Coverage: The 30% Curse and Ways Forward</b><br><b>Purpose</b><br> In this artifact, we provide the information about our benchmarks used for manual and tool exploration. We include coverage results achieved by tools and human analysts as well as plots of the coverage progression over time for analysts. We further provide manual analysis results for our case study, more specifically extracted reasons for unreachability for the case study apps and extracted code-level properties, which constitute a ground truth for future work in coverage explainability. Finally, we identify a list of beyond-GUI exploration tools and categorize them for future work to take inspiration from.<br><br> We are claiming <i>available</i><i> </i>and <i>reusable </i>badges; the artifact is publicly available, fully aligned with the results described in our paper and comprehensively documented.<br><b>Provenance</b><br>The artifact can be obtained from this figshare archival link: https://figshare.com/articles/dataset/ICSE_2025_-_Artifact/28194605/1<br><br>The paper preprint is available here: https://people.ece.ubc.ca/mjulia/publications/Mobile_Application_Coverage_ICSE2025.pdf<br><b>Data</b><br>The artifact submission is organized into five parts under the Data folder:<br>- 'BenchInfo' excel sheet describing our experiment dataset<br>- 'Coverage' folder containing coverage results for tools and analysts (RQ1) <br>- 'Reasons' excel sheet describing our manually extracted reasons for unreachability (RQ2)<br>- 'ActivationProperties' excel sheet describing our manually extracted code properties of unreached activities (RQ3)<br>- 'ActivationProperties-Graph' pdf which presents combinations of the extracted code properties in a graph format.<br>- 'BeyondGUI' folder containing information about identified techniques which go beyond GUI exploration.<br><br>The artifact requires about 15MB of storage.<br><b>Dataset: 'BenchInfo.xlsx':</b><br>This file list the full application dataset used for experiments into three tabs: 'BenchNotGP' (apps from AndroTest dataset which are not on Google Play), 'BenchGP' (apps from AndroTest which are also on Google Play) and 'TopGP' (top ranked free apps from Google Play). Each tab contains the following information:<br>- Application Name<br>- Package Name<br>- Version Used (Latest)<br>- Original Version<br>- # Activities<br>- Minimum SDK<br>- Target SDK<br>- # Permissions (in Manifest)<br>- List of Permissions (in Manifest)<br>- # Features (in Manifest)<br>- List of Features (in Manifest)<br><br>The 'TopGP' sheet also includes Google-Play-specific information, namely:<br>- Category (one of 32 app categories)<br>- Downloads<br>- Popularity Rank<br><br>The 'BenchGP' and 'BenchNotGP' sheets also include the original version (included in the AndroTest benchmark) and the source (one of F-Droid, Github or Google Code Archives).<br><br><b>RQ1: 'Coverage'</b><br>The 'Coverage' folder includes coverage results for tools and analysts, and is structured as follows:<br>- 'CoverageResults.xlsx": An excel sheet containing the coverage results achieved by each human analysts and tool. <br>    - The first tab described the results over all apps for analysts combined, tools combined, and analysts + tools, which map to Table II in the paper. <br>    - Each of the following 42 tab, one per app in TopGP, marks the activities reached by Analyst 1, Analyst 2, Tool 1 (ape) and Tool 2 (fastbot), with an 'x' in the corresponding column to indicate that the activity was reached by the given agent.<br>- 'Plots': A folder containing plots of the progressive coverage over time of analysts, split into one folder for 'Analyst1' and one for 'Analyst2'. <br>    - Each of the analysts' folder includes a subfolder per benchmark ('BenchNotGP', 'BenchGP' and 'TopGP'), containing as many png files as applications in the benchmark (respectively 47, 14 and 42 image files) named 'ANALYST_[X]_[APP_PACKAGE_NAME]'.png.<br><br><br><b>RQ2: 'Reasons.xslx'</b><br>This file contains the extracted reasons for unreachability for the 11 apps manually analyzed. <br>- The 'Summary' tab provides an overview of unreached activities per reasons over all apps and per app, which corresponds to Table III in the paper. <br>- The following 11 tabs, each corresponding to and named after a single application, describe the reasons associated with each activity of that application. Each column corresponds to a single reason and 'x' indicates that the activity is unreached due to the reason in that column. The top row sums up the total number of activities unreached due to a given reason in each column.<br>- The activities at the bottom which are greyed out correspond to activities that were reached during exploration, and are thus excluded from the reason extraction.<br><br><br><b>RQ3: 'ActivationProperties.xslx'</b><br>This file contains the full list of activation properties extracted for each of the 185 activities analyzed for RQ2.<br>The first half of the columns (columns C-M) correspond to the reasons (excluding Transitive, Inconclusive and No Caller) and the second half (columns O-AE) correspond to properties described in Figure 5 in the paper, namely:<br>- Exported<br>- Activation Location:<br>    - Code: GUI/lifecycle, Other Android or App-specific<br>    - Manifest<br>- Activation Guards:<br>    - Enforcement: In Code or In Resources<br>    - Restriction: Mandatory or Discretionary<br>- Data: <br>    - Type: Parameters, Execution Dependencies<br>    - Format: Primitive, Strings, Objects<br>The rows are grouped by applications, and each row correspond to an activity of that application. 'x' in a given column indicates the presence of the property in that column within the analyzed path to the activity. <br>The third and fourth rows sums up the numbers and percentages for each property, as reported in Figure 5.<br><b>RQ3: 'ActivationProperties-Graph.pdf'</b><br>This file shows combinations of the individual properties listed in 'ActivationProperties.xlsx' in a graph format, extending the combinations described in Table IV with data (types and format) and reasons for unreachability.<br><br><br><b>BeyondGUI</b><br>This folder includes:<br>- 'ToolInfo.xlsx': an excel sheet listing the identified 22 beyond-GUI papers, the date of publication, availability, invasiveness (Source code, Bytecode, framework, OS) and their targeting strategy (None, Manual or Automated).<br>- ToolClassification.pdf': a pdf file describing our paper selection methodology as well as a classication of the techniques in terms of Invocation Strategy, Navigation Strategy, Value Generation Strategy, and Value Generation Types. We fully introduce these categories in the pdf file.<br><br><br><b>Requirements &amp; technology skills assumed by the reviewer evaluating the artifact</b><br>The artifact entirely consists of Excel sheets which can be opened with common Excel visualization software, i.e., Microsoft Excel, coverage plots as PNG files and PDF files. It requires about 15MB of storage in total.<br>No other specific technology skills are required of the reviewer evaluating the artifact.<br>