A Reward Shaping Approach for Reserve Price Optimization using Deep Reinforcement Learning | IEEE Conference Publication | IEEE Xplore

A Reward Shaping Approach for Reserve Price Optimization using Deep Reinforcement Learning

Publisher: IEEE

Abstract:

Real Time Bidding is the process of selling and buying online advertisements in real time auctions. Real time auctions are performed in header bidding partners or ad exch...View more

Abstract:

Real Time Bidding is the process of selling and buying online advertisements in real time auctions. Real time auctions are performed in header bidding partners or ad exchanges to sell publishers' ad placements. Ad exchanges run second price auctions and a reserve price should be set for each ad placement or impression. This reserve price is normally determined by the bids of header bidding partners. However, ad exchange may outbid higher reserve prices and optimizing this value largely affects the revenue. In this paper, we propose a deep reinforcement learning approach for adjusting the reserve price of individual impressions using contextual information. Normally, ad exchanges do not return any information about the auction except the sold-unsold status. This binary feedback is not suitable for maximizing the revenue because it contains no explicit information about the revenue. In order to enrich the reward function, we develop a novel reward shaping approach to provide informative reward signal for the reinforcement learning agent. Based on this approach, different intervals of reserve price get different weights and the reward value of each interval is learned through a search procedure. Using a simulator, we test our method on a set of impressions. Results show superior performance of our proposed method in terms of revenue compared with the baselines.
Date of Conference: 18-22 July 2021
Date Added to IEEE Xplore: 20 September 2021
ISBN Information:

ISSN Information:

Publisher: IEEE
Conference Location: Shenzhen, China

I. Introduction

In recent years, Real Time Bidding (RTB) has become the main platform for trading online advertisements (ads). Considering its extremely high turnover, most website owners participate in this business by selling some blocks in their websites to the advertisers. In display advertising, these blocks are called ad slots or ad placements. Basically, selling and buying ad slots are performed through online RTB auctions. These auctions are performed in ad networks or Ad Exchange (AdX) markets in real time and the winner of an auction advertises in corresponding ad slot. The auctions are mainly second price auctions where the winner pays as much as the second highest bid. In second price auctions, the website owner adjusts a reserve price which determines the minimum price of the ad slot. The final price of sold ad slots is the maximum of the reserve price and the second highest bid which is paid to the ad publishers and the ad is shown in the end user's browser [1].

References

References is not available for this document.