Perceptual Video Coding Based on Semantic-Guided Texture Detection and Synthesis

Visually insensitive texture regions consume a large number of bitrate in hybrid video coding, leading to the waste of bandwidth resources. For this, we propose a semantic-guided texture synthesis framework (STSF). At encoder, high-level semantic information is adopted as texture features to detect texture regions and is sent to the decoder. Detected texture regions are coarsely encoded by hybrid codec. To generate realistic texture patterns, we design a multi-model semantic-guided texture synthesis generative adversarial network (STSGAN) at decoder, which works in a divide-and-conquer manner that semantically different texture regions are synthesized by different submodels in it. Experimental results show that STSF can achieve a −17.2% MOS BD-rate under the lowdelay_P configuration, compared with VVC.