Public datasets.

The rapid growth of digital payments exacerbates the challenges in Financial Transaction Fraud Detection (FTFD). These challenges stem primarily from an extreme class imbalance, where legitimate transactions greatly outnumber fraudulent ones. This imbalance significantly hampers the ability of FTFD models to accurately learn fraud patterns. Although existing data augmentation techniques have shown effectiveness in alleviating this problem, they are often negatively influenced by anomalous samples that diverge from the true fraud distribution due to fraudsters’ concealment strategies and the inherent complexity of fraudulent patterns. This divergence makes it challenging to accurately model the distribution of fraudulent activities. In this work, we propose a Boundary-Aware Dual-discriminator Generative Adversarial Network (BADGAN) to address the class imbalance issue in FTFD. BADGAN integrates a boundary sample classifier with a dual-constraint mechanism based on distance adversarial learning, allowing the generator to produce synthetic samples that both adhere to the distribution of real fraud data and maintain a distance from the decision boundary. This boundary-aware design emphasizes the optimization of sample quality near classification boundaries, thereby improving the downstream classifier’s ability to distinguish fraudulent behavior. Extensive experiments on both real-world and public datasets demonstrate that BADGAN outperforms its competitive peers in addressing the class imbalance issue, thereby enhancing the detection performance of FTFD models.