A Chatbot to Search for Similar Fashion Photos and Reference Fashion Coordinators via Body Part and Fashion Item Segmentations

In the past decade, a variety of apparel e-commerce sites have appeared. Although such sites enable users simply and easily to purchase fashion items, there frequently occur matters such that ordered items differ from what the user expected. To avoid such matters, fashion coordination sites have also been used, to which users can post photo images of their fashion coordination and appreciate the fashion photo images posted by other users. However, due to the vast amount of information posted on such sites, it is not always easy for inexperienced users to search for reference users (fashion coordinators) and photo images matching their tastes. To support such searching, we construct a prototype for a fashion chatbot that accepts a user's fashion photo image as input and outputs similar fashion photo images to the input image and account URL links of recommendable reference users (fashion coordinators). In this chatbot, we first conduct a body part segmentation to extract color features of body parts from the input fashion photo image. However, such a way does not properly consider item categories. Therefore, this chatbot also performs a fashion item segmentation to recognize item categories. Then, this chatbot searches for similar fashion photo images and recommendable reference users (fashion coordinators) based on the extracted color features of body parts and the recognized item category information. In this study, such a searching procedure for similar fashion photo images is also validated using 4,571 images posted on a fashion coordination site in Japan. Moreover, we test the chatbot with a use-case scenario.