The Directionality Function Defect of Performance Evaluation Method in Regression Neural Network for Stock Price Prediction

The most important financial attribute of stock time series is the directionality of its price movement. This study found that the prediction error of neural network can only reflect the closeness between model predicted price and actual market price, but not the important financial attribute, the direction of stock price's ups and downs. Taking Chinese stock 600275 and American stock AMZN as examples, the former's prediction error is 0.0353, and its stock return rate is negative 59.49%, while the latter's prediction error is 0.0201, and its stock return rate is positive 26.49%, while the difference between the predictions of the two stocks is only 0.0152. It shows that under the same scale, absolute value constraint is the source of the problem that the prediction error has no stock up and down attributes. Therefore, in the absence of condition variable of trend direction state, it is unreliable to make the conclusion by judging the performance of regression neural network for stock price prediction based only on the value size of prediction error.

I. Introduction

The existing performance evaluation of Neural Network for Stock Price Prediction (NNSPP) is evaluated based on prediction error(PE) metric[1][2], but for securities market, this kind of numerical evaluation method lacking directionality does not have financial trading properties. Because performance evaluation metric takes the absolute value of prediction error, and the movement of stock price is directional. In short, they are either rising or falling. This is the most basic fractal feature of financial time series. Obviously many researchers have ignored the effectiveness of PE to evaluate the performance of regression neural networks. Strictly speaking, the PE does not have the basic function of evaluating financial time series, because the same level prediction error does not reflect the general movement direction of the financial time series being evaluated. Therefore, the PE as an ideal performance evaluation metric for stock price prediction can barely work well because they make it look convincing in research papers, nothing more. It can be seen that the neural network model for stock price prediction lacking functionality test and trading performance test mentioned above is difficult to meet the demand[3][4], because it is impossible for ordinary investors to directly link the model prediction error with stock price movements. One of the earliest implementation of stock prediction with neural networks was IBM’s engineer H. White, who has also mentioned that it is not enough to judge the profit and loss of securities trading using model performance by its high or low value of prediction errors[5]. Even if these models for stock price prediction that have not been evaluated for direction attributes can be used reluctantly, it is very easy to mislead investors and cause huge economic losses.