Trend-Heuristic Reinforcement Learning Framework for News-Oriented Stock Portfolio Management

Recent studies have shown that reinforcement learning (RL) methods have brought significant performance gains for stock portfolio management (PM) because they effectively utilize historical price information and directly generate portfolio weights. We found, however, that there is still great room for improvement in how to fully consider the impact of financial news and stock trends on PM while avoiding model instability caused by feature disparity and fragile convergence due to RL itself. Addressing this allows us to develop more profitable and robust PM strategies. To this end, we propose TrendTrader, a novel RL framework for news-oriented stock portfolio management with trend heuristics. Specifically, TrendTrader utilizes a large language model (LLM) to obtain sentiment scores of stock news and generates heterogeneous contexts based on global sentiment embedding, which enhances model stability in processing multimodal features. Further, TrendTrader incorporates trend heuristics into both the network architecture and the reward function, while combining supervised learning and reinforcement learning in the form of incremental training to facilitate policy network convergence. Extensive experiments in the U.S. market and the China market verify the state-of-the-art performance of TrendTrader.