Inferring Emotions from Image Social Networks Using Group-Based Factor Graph Model

Inferring emotions from image social networks is a hot research topic nowadays. For image social networks (Flickr, Instagram), there is an interesting phenomenon that people would like to establish or attend virtual groups and share images with different topics and emotions in different groups. Previous researches on inferring emotions usually focus on image content and user personalization, thus leading an interesting but challenging problem: whether virtual groups can influence members(users)` emotions. In this paper, we systematically study this problem from two aspects: 1) whether group homophily in users' emotions exists in image social networks; 2) how to model this subtle and complex group homophily in image social networks. Inspired by the study results of two aspects, we introduce group information to infer emotions in image social networks, and propose a novel Group-Based Factor Graph Model (G-FGM), incorporating image content, user personalization and group information to understand the emotions behind social images better. The experimental results on a dataset containing 218, 816 emotion-labeled images from Flickr show that our model outperforms (8.6-19.4% improvement in terms of F1-Measure) several baseline methods.

1. Introduction

With the development of social networks, more and more people are willing to share images to express their feelings via social network platforms such as Flickr ¹

http://www.flickr.com

and Instagrarn ²

https://www.instagram.com

. For the world's largest image-sharing website Flickr, 38% of images are explicitly annotated as positive or negative emotion tags by their publishers [1]. Understanding the emotions behind images in social networks is an important research topic with big challenges. It can also benefit many applications such as image retrieval, market advertising and personalized recommendation.