An Unsupervised Approach towards Varying Human Skin Tone Using Generative Adversarial Networks

With the increasing popularity of augmented and virtual reality, retailers are now focusing more towards customer satisfaction to increase the amount of sales. Although augmented reality is not a new concept but it has gained much needed attention over the past few years. Our present work is targeted towards this direction which may be used to enhance user experience in various virtual and augmented reality based applications. We propose a model to change skin tone of a person. Given any input image of a person or a group of persons with some value indicating the desired change of skin color towards fairness or darkness, this method can change the skin tone of the persons in the image. This is an unsupervised method and also unconstrained in terms of pose, illumination, number of persons in the image etc. The goal of this work is to reduce the time and effort which is generally required for changing the skin tone using existing applications (e.g., Photoshop) by professionals or novice. To establish the efficacy of this method we have compared our result with that of some popular photo editor and also with the result of some existing benchmark method related to human attribute manipulation. Rigorous experiments on different datasets show the effectiveness of this method in terms of synthesizing perceptually convincing outputs.

I. Introduction

Augmented reality (AR) is an interactive experience of a real-world environment where the objects that reside in the real world are enhanced by computer-generated augmentations to it, in order to enhance our experiences [1], [2]. Though the concept of AR is not new but with the advent of machine learning and deep learning in computer vision, AR got its much needed push into the mainstream. various retailers like Burberry, 1-800-Flowers.com and ASOS, Nike, Benjamin Moore etc. infuse augmented reality (AR) into their apps to help online customers make more informed purchase decisions [3]. For example Nike Fit is an app launched by the big sports retail brand Nike that scans one's foot dimensions using his smartphone's camera and suggests his size of shoe. During 2018 CES (Consumer Technology Association) a dressing room app was launched by GAP, a clothing retail company, where shoppers can select the cloth of their choice along with one of five body types to visualize what an outfit will look like on them. All these shows retailers are now focusing more towards new ways to enhance user's shopping experience. Our work is a new approach towards this direction. During buying cloths online we see the image of a model wearing the cloth and decide to buy or not based on how she looks on it. However excluding the cloth size factor, it is seen often that a confusion remains in the buyer's mind about his looks after wearing it. The reason may be, generally the models are slim, fair skinned, tall etc., which means the seller makes an ideal situation in which his product looks best. Another case may be you are buying a cloth for a friend of yours, however he is much fairer or darker than the model. There can be various such cases where the buyer wishes to see the cloth on a model with his choice of features. Not only clothes, the same concept applies to various accessories also. This work addresses the skin tone aspect of this problem, where we can vary the skin tone of a person in an image. Such an application may be helpful for fashion designers also. Fig. 1:

Illustrating the objective of the present work. The results with different skin tones is shown in the right to . The axis line above indicates the values of the skin tone control variable. The values along the negative direction of the axis indicates darkness while that along the positive direction indicates fairness. The amount of skin color change is proportional to the absolute value of the variable.