AUROC results achieved by different methods.

Zero-shot image classification enables the recognition of new categories without requiring additional training data, thereby enhancing the model’s generalization capability when specific training are unavailable. This paper introduces a zero-shot image classification framework to recognize new categories that are unseen during training by distilling knowledge from foundation models. Specifically, we first employ ChatGPT and DALL-E to synthesize reference images of unseen categories from text prompts. Then, the test image is aligned with text and reference images using CLIP and DINO to calculate the logits. Finally, the predicted logits are aggregated according to their confidence to produce the final prediction. Experiments are conducted on multiple datasets, including MNIST, SVHN, CIFAR-10, CIFAR-100, and TinyImageNet. The results demonstrate that our method can significantly improve classification accuracy compared to previous approaches, achieving AUROC scores of over 96% across all test datasets. Our code is available at https://github.com/1134112149/MICW-ZIC.

零样本（Zero-shot）图像分类无需额外训练数据即可实现新类别的识别，在无法获取特定训练数据的场景下，可有效提升模型的泛化能力。本文提出一种零样本图像分类框架，通过从基础模型（foundation models）中蒸馏知识，实现对训练阶段未见过的新类别的识别。具体而言，我们首先借助ChatGPT与DALL-E，根据文本提示（text prompts）合成未见类别的参考图像。随后，我们利用CLIP与DINO将测试图像与文本及参考图像进行对齐，以计算logits（对数几率）。最后，我们根据各预测logits的置信度进行聚合，以得到最终的分类结果。本研究在多个公开数据集上开展了实验，包括MNIST、SVHN、CIFAR-10、CIFAR-100以及TinyImageNet。实验结果表明，相较于现有方法，本文提出的方法可显著提升分类准确率，在全部测试数据集上的AUROC得分均超过96%。本研究的代码已开源至：https://github.com/1134112149/MICW-ZIC。