On-device Filtering of Social Media Images for Efficient Storage

Artificially crafted images such as memes, seasonal greetings, etc are flooding the social media platforms today. These eventually start occupying a lot of internal memory of smartphones and it gets cumbersome for the user to go through hundreds of images and delete these synthetic images. To address this, we propose a novel method based on Convolutional Neural Networks (CNNs) for the on-device filtering of social media images by classifying these synthetic images and allowing the user to delete them in one go. The custom model uses depthwise separable convolution layers to achieve low inference time on smartphones. We have done an extensive evaluation of our model on various camera image datasets to cover most aspects of images captured by a camera. Various sorts of synthetic social media images have also been tested. The proposed solution achieves an accuracy of 98.25% on the Places-365 dataset and 95.81% on the Synthetic image dataset that we have prepared containing 30K instances.

I. Introduction

In recent years, there has been a huge flux of data on the internet. Social media platforms have contributed significantly in increasing the volume of images circulated. Images are being used to communicate opinions on trending news through memes. According to a survey, 3.2 billion images are shared each day over the internet. A large proportion of these images are wholly synthetic and become irrelevant to the user in a short period. Images captured from the camera have very few sharp edge transitions and are characterized by sensor pattern noise. Jan Lukas et al. [1] proposed the identification of digital cameras based on the sensors pattern noise by using a correlation detector to investigate the presence of the reference noise pattern in the given image. Corripio et al. [2] computed wavelet features for smartphone camera identification. Artificially crafted memes or seasonal greetings have sharp edge transitions. If the image is entirely artificially generated, it doesn’t have noise in its raw form. Some noise is added by the lossy compression techniques used by social media platforms. These synthetic images are generally created by adding artificial text on the camera image or stacking multiple camera images together. In this process, certain regions of the image get sharper edge transitions and uniform pixel intensities.