Memory-based diverse-category single-view 3D reconstruction

Single-view object reconstruction aims to recover the geometric structure from a single image, which has wide applications in 3D modeling and virtual reality. The existing methods are limited to complex annotations or single-category models, which affect their generalizability and practical applications. To tackle this problem, we propose a memory-based single-view reconstruction network called M-SRN. Given a collection of images, M-SRN can generate high-fidelity reconstructions across diverse categories. Our main contributions here are three approaches to leverage memory representations. First, a foreground perceptron module was developed through memory-representation-based contrastive learning, enabling M-SRN to reconstruct raw image collections. Second, a purified memory-based cross-category feature compensation module was proposed to enhance dataset-level instance consistency. Finally, a dynamic neighbor consistency enhancement module based on intra-class memory prototypes was proposed to mitigate the inherent ambiguity of single-view supervision. M-SRN was validated using synthetic and real-world datasets. Experiments demonstrate that M-SRN outperforms state-of-the-art weakly supervised methods and achieves results comparable to 2D-supervised and 3D-supervised methods.