MobileNet-based Neural Image Caption Model in Title Generation for Product's Images

In e-commerce, it is not hard to find a product title that doesn't match with the product image, or even is totally not relevant. A machine learning model can be employed to generate a suitable title product given a product image. In this paper, we describe the development of neural network model to address the aforementioned issue. We use Neural Image Caption (NIC) that is originally for general image captioning as our baseline model. We also describe the development of our own dataset to build the model, the modifications of NIC for product title generation, and the performance of the proposed model, which is measured by using BLEU score. The proposed model modifies the baseline by changing the CNN encoder with a fine-tuned MobileNetV2 and changing the inference technique to greedy search. We then provide performance evaluations for these modifications with respect to the baseline model. We also compare the model performance on the proposed preprocessed data and the raw (original) data. Training using the preprocessed dataset yields a model with a better performance than using raw data, both in higher BLEU score and faster inference time. Our modified model with fine-tuning MobileNetV2 has BLEU-1 score of 58.2, while the baseline model which uses GoogLeNet without fine-tuning has BLEU-1 score of 55.5. Furthermore, changing inference technique to greedy search results a faster inference time (0.1 seconds) compared to the baseline model that uses beam search with beam size of 20 (27.661 seconds). Our model is then implementable for the chosen problem domain, e-commerce product title generation, which needs fast inference time without sacrificing model performance.