Enhancing Market Making Strategies with Deep Reinforcement Learning-Based Quoting Decisions

Market making strategies often struggle to adapt to complex market dynamics in rapidly changing conditions. This paper explores the application of deep reinforcement learning (DRL) techniques to enhance market making by integrating DRL agents with the Avellaneda-Stoikov (AS) model for optimal quoting. We propose a framework where the A2C and PPO agents, responsible for deciding whether to quote or not, are employed and compared against the traditional AS agent, which continuously quotes, in a simulated market environment with realistic order matching engine. Extensive experiments are conducted using historical tick-level trade data from the BTCUSDT perpetual futures market, chosen for its high volatility and rapidly changing market dynamics, to assess the performance of the DRL agents against the AS agent. The results indicate that the DRL agents exhibit higher quote rates for filled quotes and lower quote rates for expired quotes compared to the AS agent, showcasing their ability to selectively skip unprofitable quotes. However, the agents' overall profitability is significantly impacted by substantial losses from expired quotes due to counter-reversal market dynamics.