From Uniform Models To Generic Representations: Stock Return Prediction With Pre-training

The emergence of deep learning has cast new light on the century-old problem of stock return prediction. For single stock return prediction, incorporating peripheral data such as cross sectional information has become the de facto standard for target horizons denoted in hours and above. However, such approach is not directly applicable to predicting short-term stock returns due to their strong stochastic nature. Little has been reported in public domain on how to utilize the rich exogenous data in short-term scenarios effectively. We propose a representation learning solution based on a pretrain-finetune framework. To help models learn high-quality feature extractors, we further propose to use triplet loss as a novel pre-train task. We present a new sample selection criterion and three versions of triplet selection in this context: “easy sample”, “multiple samples”, and “hard sample”. Experiment results using the proposed method demonstrate significant improvement over standard approaches. We also share some insight on how to apply triplet loss effectively in the context of short-term stock return prediction. Specifically, we demonstrate that using regression labels to select triplets is more effective than using embedding similarity. The proposed training framework is model-agnostic and shows great performance improvements in various settings.