GradDA – A novel dataset for investigating domain shifts in image classification

A domain shift occurs when the testing data is drawn from a distribution different from that of the training dataset. This shift presents a significant challenge and may compromise the performance of machine learning models, which leads to poor generalization. Over the past years, various models have been developed and evaluated on benchmark datasets such as VisDA, Office-Home and DomainNet. These datasets consist of discrete domains with different object classes. However, a notable limitation when addressing the domain shift is the absence of data samples where the exact same object exists in both domains. 

We propose a new dataset designed to address this challenge. In particular, we introduce a domain shift from a purely synthetic style (grey object on white background) to a more realistic appearance (object with texture against a realistic background) with differential modifications, which enables the representation of the same object in both synthetic and real domains, consequently facilitating the analysis of a transition between the two domains. The dataset comprises five distinct classes (Airplane, Bicycle, Bus, Car, Train), with multiple objects per class. Additionally, each object is depicted from 20 different perspectives, resulting in a total of 101 images per perspective that captures the transition from pure synthetic to a more real-world-like domain. This dataset offers a unique opportunity to investigate the impact of domain shift on model performance in classification tasks, as it focuses solely on domain changes without other interfering effects. It is the objective of our work to trigger new discussions about the domain shift problem, and how it can be tackled with alternative data driven model designs.