Performance Exploration Dataset

The dataset is created to benchmark performance exploration approaches. We have injected synthetic performance bottlenecks into the source code of the RUBiS web application as delays that are triggered on certain input combinations. The total number of input combinations is 3,100,000, and the bottlenecks roughly cover 9% of the total input combinations.

We have created two variants of the RUBiS web application. In RUBiS(uni), the bottlenecks are uniformly distributed, and the bottlenecks are widely spread over the entire input space. Meanwhile, in RUBiS(poi), we have used Poisson distribution. In the case of RUBiS(poi), as opposed to RUBiS(uni), the bottlenecks are packed together in the input space.

The dataset folder contains bottleneck distributions of RUBiS(poi) and RUBiS(uni). The first four columns of the *_exhaustive_data_dump.csv CSV files contain the values of the input variables: category id, region id, item id, and user id. The fifth column, named bottleneck, indicates whether a certain combination is a bottleneck. If the value is 1, then the corresponding combination is a bottleneck.

The CSV files in the experiment folder contain the cumulative number of bottlenecks identified by iPerfXRL after executing the 775,000 input combinations for RUBiS(poi) and RUBiS(uni).

This dataset has been used in the following publications:

Using Deep Reinforcement Learning for Exploratory Performance Testing of Software Systems With Multi-Dimensional Input Spaces (10.1109/ACCESS.2020.3033888)

Automatic exploratory performance testing using a discriminator neural network (10.1109/ICSTW50294.2020.00030)

Exploratory Performance Testing Using Reinforcement Learning (10.1109/SEAA.2019.00032)