3D Virtual Garment Modeling from RGB Images

We present a novel and efficient approach that automatically constructs 3D models for garments using only RGB images. Most of the previous methods deal with input photos in which the garment is on a human body or mannequin. Our approach can work with various types of input garment photos: photos in which the garment is worn by a model, or photos in which the garment is laid on a flat surface. To construct a complete 3D model, our approach requires minimum two images as input: one front view and one rear view. We propose a multi-task learning network called JFNet that jointly identifies the landmarks of the garment as well as parses the garment into semantic part segments. The predicted landmarks are used to estimate the garment size thus a template mesh can be deformed accordingly to construct the 3D mesh model. Color and textures of the model are extracted by exploiting the parsed semantic parts from input images. Our approach can be applied in various Virtual Reality and Mixed Reality applications involving garment modeling.