Approximation-guided Fairness Testing through Discriminatory Space Analysis

Description:

This dataset contains the experimental results from the paper titled "Approximation-guided Fairness Testing through Discriminatory Space Analysis".

In this paper, we conducted 24 fairness testing tasks using 7 different fairness testing algorithms: AFT (our proposed algorithm), VBT-X, VBT, THEMIS, ExpGA, SG and LIMI. Each execution was repeated 30 times, with a runtime of 1 hour.

Files Included:

log.txt: This file contains the logs of each execution. Each log is labeled with an identifier, such as "'aft-LogReg-Adult-sex-0", which represents the 1st execution of the fairness testing task on (LogReg, Adult, sex) using AFT.

discriminatory_instances.zip: This archive includes all the IDIs (individual discriminatory instances) identified during the experiments.

res.txt: This file contains the averaged results of the 30 repetitions. It includes four metrics: #IDIs/sec (the number of indentified IDIs per second), #Tests/sec (the number of generated test cases per second), SuccessRatio (the success ratio of test cases), Diversity (the diversity of IDIs) and Naturalness (the naturalness of IDIs).