Exploring the Relationship Between the Geometry of a Fixed Embedding of Image Data and Its Underlying Cluster Structure

Standard self-supervised clustering algorithms transform input data via embedding models that are trained to fit the data and then cluster the embedded vectors. Despite its flexibility, a data driven embedding model may not be applicable when the raw data are unavailable due to privacy or security concerns, and it cannot be adapted to the increasingly common framework of transfer learning. We previously proposed a Merge & Expand (ME) framework for clustering images using a fixed embedding model. In this study, we have substantially modified our ME framework and conducted a series of experimental analysis to explore the relationship between the geometry of a fixed embedding space and the underlying cluster structure. We demonstrate that the clustering outcomes are robust against varying hyperparameter values. We assessed the heterogeneity of predicted labels in each region, revealing that it is a strong indicator of the quality of clustering outcomes. We further exploited the heterogeneity information to modify the ME framework, improving clustering accuracy by introducing a second embedding. Moreover, we provide intuitive explanations for sources of confusion in merged seed regions. Comparisons with numerous other clustering methods on five datasets indicate that our ME framework performs competitively despite employing a fixed embedding, a simple CNN architecture, and a common loss function. Thus, our ME framework enables users to better understand the relationship between the geometry of the embedding space and the underlying cluster structure. Supplementary materials for this article are available online.