Contextualized Styling of Images for Web Interfaces using Reinforcement Learning

Content personalization is one of the foundations of today’s digital marketing. Often the same image needs to be adapted for different design schemes for content that is created for different occasions, geographic locations or other aspects of the target population. We present a novel reinforcement learning (RL) based method for automatically stylizing images to complement the design scheme of media, e.g., interactive websites, apps, or posters. Our approach considers attributes related to the design of the media and adapts the style of the input image to match the context. We do so using a preferential reward system in the RL framework that learns a reward function using human feedback. We conducted several user studies to evaluate our approach and demonstrate that we are able to effectively adapt image styles to different design schemes. In user studies, images stylized through our approach were the most preferred variation across a majority of our experiments. Additionally, we also release a dataset consisting of perceptual associations of web context with the associated image style.

I. Introduction

A professional-looking website with an engaging audience is a great way to promote a brand. With the availability of Stock photography services such as Shutterstock, Adobe Stock, etc., content creators can easily create impressive-looking websites. Although finding image assets is much easier, for the content to be effective for the brand (i.e., better engagement, higher click-through rates, higher conversion), the images need to be stylized or optimized. Approaches to effectively modify images to improve engagement have been extensively studied in the literature [1]. However, all these studies assume the invariability of the webpage where the image is embedded. In reality, different web pages have different design aesthetics as well as different themes. Additionally, brands also often change the aesthetics and styles of their web pages. Therefore, context is key for adopting the right image styling strategies. Context here refers to the circumstances under which the image asset will be consumed by the user. In the case of websites, context could include but is not limited to the website’s design template, the target users and their role, the task at hand or the steps in the process, the user’s location, the time and date, or the device being used. Manually styling the image assets so that it blends well with this context can be difficult and time-consuming. Also, creating and delivering image content at a scale that can resonate with the user is a very challenging task. In this paper, we, therefore, investigate how to efficiently automate this process and optimize the image style characteristics based on the specific context it is associated with.