PFDM: Parser-Free Virtual Try-On via Diffusion Model

Virtual try-on can significantly improve the garment shopping experiences in both online and in-store scenarios, attracting broad interest in computer vision. However, to achieve high-fidelity try-on performance, most state-of-the-art methods still rely on accurate segmentation masks, which are often produced by near-perfect parsers or manual labeling. To overcome the bottleneck, we propose a parser-free virtual try-on method based on the diffusion model (PFDM). Given two images, PFDM can "wear" garments on the target person seamlessly by implicitly warping without any other information. To learn the model effectively, we synthesize many pseudo-images and construct sample pairs by wearing various garments on persons. Supervised by the large-scale expanded dataset, we fuse the person and garment features using a proposed Garment Fusion Attention (GFA) mechanism. Experiments demonstrate that our proposed PFDM can successfully handle complex cases, synthesize high-fidelity images, and outperform both state-of-the-art parser-free and parser-based models.

1. INTRODUCTION

Given two images of a person and a garment, virtual try-on is to "wear" on the garment while keeping the person’s pose and identity unchanged. It has attracted wide research attention with the potential to enhance the shopping experiences in e-commerce and metaverse [1], [2]. For example, virtual try-on allows users to quickly browse the fitting effects and can reduce the refund possibilities.