Caption Unification for Multiple Viewpoint Lifelogging Images and Its Verification

With the development of organization and retrieval systems for visual lifelogging data, many studies have attracted more attention in adjusting the raw visual lifelogging data to facilitate the systems. Although caption generation is one of the effective methods to adjust visual lifelogging data such as images and videos, an existing image captioning model generates different captions for lifelogging images of the same scene from multiple viewpoints. This prevents efficient organization and retrieval for shared lifelogging images, and the caption unification is expected to be the solution. In this paper, we assume that the above problem is caused by the diversity of objects in each lifelogging image and propose a new caption unification method. We remove selected common objects from lifelogging images of the same scene from multiple viewpoints by an inpainting approach and also verify the performance of the caption unification in the latent representations before and after the inpainting. The experimental results show the validity of our assumption.