Lightweight Photorealistic Image Style Transfer with Collaborative Optimization of Shuffle Gate Attention and Channel Alignment Whitening and Coloring Transform

Existing photorealistic image style transfer algorithms typically do not fully consider the issues of algorithm model size and computational efficiency while pursuing improvements in image realism and stylization intensity. Therefore, applying these methods to low computing power devices is difficult. To address this issue, this study proposes a lightweight real image style transfer algorithm. The VGG19 is replaced with the ShuffleNet V2 lightweight network as the feature extractor, with block-wise training and skip-connection techniques introduced to significantly reduce the number of parameters and improve the speed of image style transfer. To better balance the content and style of the transferred images, the study also proposes a Shuffle Gated Channel Attention Mechanism (SGCAM) and Channel Alignment Whitening and Coloring Transform (CAWCT). SGCAM combines channel shuffling with gating mechanisms efficiently, which not only enhances the realism of generated images but also further maintains the advantage of the lightweight algorithm. CAWCT significantly boosts the stylization intensity of the generated images by introducing binary operations to match the whitened content features and style features for similarity. Experimental results show that the parameter size of the proposed algorithm is only 14.8% of that of PhotoWCT2. It takes only 4.22 s to transfer an image with a resolution of 1 000×750 pixel, which is 0.79 s faster than that achieved by PhotoWCT2. Simultaneously, the quality and stylization strength of the generated images are significantly improved. In performance evaluations, the Structural Similarity Index Measure (SSIM) and Peak Signal-to-Noise Ratio (PSNR) indicators increase by 0.031 dB and 0.066 dB, respectively, while the Content loss, Gram loss, and Style loss metrics decrease by 0.227, 0.138×10-5, and 0.116, respectively.