Biphasic Face Photo-Sketch Synthesis via Face Semantic-Aware CycleGAN

With the introduction of CycleGAN, there have been significant advancements in the task of biphasic face photo-sketch synthesis (FPSS). To further enhance the results in this task, most methods have focused on optimizing the model without attempting to exploit the additional information inherent in the face itself. To address this issue, this paper proposes a novel Face Semantic-Aware CycleGAN (FSACycleGAN) for bidirectional face photo-to-sketch synthesis. Firstly, a method is introduced to inject facial semantic distribution images into the generator, providing additional spatial and identity supervision for the synthesized face photos and sketches. Furthermore, facial semantic information is incorporated into the loss function along with identity differences in generated images, aiding in preserving facial details and identity information in the generated face photos and sketches. Lastly, a novel cyclic adversarial training strategy is proposed. It involves introducing an additional recognition model, enabling a cycle of adversarial training between the recognition and generation models to optimize the generated results. Through extensive training on the CUFS dataset, the results demonstrate that the proposed method has achieved significant improvements compared to CycleGAN. Specifically, it has shown a 16.3% improvement in SSIM and a 5.5% improvement in FSIM. The accuracy in sketch matching and photo matching tasks reached 99.42% and 98.64%, respectively, showcasing excellent performance.