The dataset of the Web Visual Flakiness in the Wild: An Empirical Study of Causes and Fixes

Dataset Overview

Dataset file

projects.zip

Dataset file

The data in this file can be used to further analyze flaky behaviour issues in web apps and support improvements in test stability. Our dataset file contains 8 sheets, each categorized as follows:

Projects-statistics: this sheet records statistical information for different projects, such as project names, description, the number of stars, flaky numbers, visual-related flaky numbers, etc. It provides an overview of all projects we selected.

All-data: this sheet contains the complete dataset of flaky behaviour cases, including the link, commit info, flaky description, causes of flakiness, observed fix strategies and framework, status, priority, etc. It serves as the foundation for our analysis.

Top5-code-distribution-web: this sheet presents the top five code distributions related to web projects, calculates the code line percentage. It helps analyze the prevalence of style-related code.

Causes-statistics: this sheet analyzes common causes of Style Resource Latency, Style Interpretation Issues, Timing-sensitive Visual Triggers, DOM Structure Issues, and Layout Instability. It serves as a basis for root cause analysis.

Fixed-cases and type statistics: the flaky behaviour cases that have been fixed and categorized by type. It provides insights into common fixes and their effectiveness.

Priority-statistic: it shows that developers addressing priorities (P1 (high), P2 (medium), and P3 (low)) under different flaky causes, which collect the priority of flaky cases in Chromium.

Framework-causes: it shows that different frameworks can have an impact on the nature and frequency of flaky behaviour, underscoring the need for tailored mitigation strategies.

Projects.zip

It contains all the projects we selected from the open-source projects, including source files used to calculate style-related code in the project, calculation results and statistics.