Disentangled Cycle Consistency for Highly-realistic Virtual Try-On

Image virtual try-on replaces the clothes on a person image with a desired in-shop clothes image. It is challenging because the person and the in-shop clothes are unpaired. Existing methods formulate virtual try-on as either in-painting or cycle consistency. Both of these two formulations encourage the generation networks to reconstruct the input image in a self-supervised manner. However, existing methods do not differentiate clothing and non-clothing regions. A straightforward generation impedes the virtual try-on quality because of the heavily coupled image contents. In this paper, we propose a Disentangled Cycle-consistency Try-On Network (DCTON). The DCTON is able to produce highly-realistic try-on images by disentangling important components of virtual try-on including clothes warping, skin synthesis, and image composition. Moreover, DCTON can be naturally trained in a self-supervised manner following cycle consistency learning. Extensive experiments on challenging benchmarks show that DCTON outperforms state-of-the-art approaches favorably.

1. Introduction

Virtual try-on of fashion images aims at changing the clothes of a person with other in-shop clothes. There are wide applications including costume matching, fashion image editing, and clothes retrieval for e-commerce. Existing methods mainly focus on a direct try-on based on 2D images because of the available person images and in-shop clothes images online. However, these images are unpaired since the collection of images with multiple models, of which each model wears different and pixel-wise aligned clothes is infeasible.