CReStyler: Text-Guided Single Image Style Transfer Method Based on CNN and Restormer

Text-guided image style transfer methods have gradually become a research hotspot. However, existing text-guided style transfer method suffers from content information missing and artifacts in the generated stylized images. Therefore, we propose CReStyler, a text-guided image style method based on the dual-branch structure of CNN and Restormer. In this work, in the first branch, we introduce a novel convolutional structure called FcasNet, which composes Frequency-domain Channel Attention Mechanism (FcaNet) and Cross Convolutional Block Attention Module (CRCBAM). It can generate rough style images l
fca
according to the input target text. In the second branch, the use of pixel-based Restormer constrains the phenomenon of fake images and content information missing due to excessive convolution. It can generate stylized images l
res
with complete content information based on the target text. Finally, we combine l
fca
and l
res
through weighted fusion to obtain refined stylized images. During the training process, we utilize directional CLIP loss to constrain text-image alignment. Experimental results show that our method produces better results compared with existing methods such as CLIPStyler, LDAST, Text2LIVE, InstructPix2Pix.