Multi-Label Image Classification Based on Label Visual Prototype Learning

Multi-label image classification studies tend to use label semantic information and label co-occurrence probability as prior knowledge to guide the learning of multi-label classification models. However, most of these methods rely on additional semantic information, which makes it difficult to handle the information mismatch problem between different modalities. The calculation of label co-occurrence probability is also susceptible to data imbalance and noise. To address these issues, this study proposes a multi-label image classification method based on label visual prototype learning, which utilizes only the visual information of an image and constructs a multi-label classifier by generating label visual prototypes. This method reduces the reliance on prior knowledge and fully utilizes the visual information, effectively improving classification performance. First, an attention module based on class-specific activation maps is designed to guide the model to focus on image regions that are more relevant to the class and generate class-specific feature representations. Second, by capturing the visual prototype representation of each label, a label visual prototype dictionary is constructed to fully leverage the adaptability of visual feature information to image classification tasks. Finally, using this dictionary as a multi-label classifier, the visual features of the input image are reconstructed to obtain the predicted probability of the labels. Experimental results show that this method improves classification accuracy compared with similar methods on three standard multi-label image classification datasets.