DISCUSSION, RECOMMENDATIONS, AND CONCLUSIONS
1. THE PROBLEM
My thesis explores the application of machine learning to make daily stock movements predictable. Stock market movements are highly volatile on a day-to-day basis. These movements are influenced by a variety of factors: the performance of a company or industry, investor sentiment about the overall market, political news, interest rates, and a variety of other economic factors. The interplay of these factors contributes to difficulty in predicting where stock prices will move. Some economists even hypothesize that market movements cannot be predicted at all—a theory called the Efficient Market Hypothesis, which views market movements as a “random walk.” Indeed, in general, time series forecasting can face the problem of “concept drift,” which is the idea that the underlying concept in data can change over time. My thesis explores different methods of mitigating the effects of concept drift to increase the accuracy of price movement predictions.
2. APPROACH
The objective of this project was to explore explores different methods of mitigating the effects of concept drift to increase the accuracy of price movement predictions; ultimately, accurate predictive power would translate to a maximization of profit and minimization of loss in the stock market. The final system consists of two parts. The first part is the predictive algorithm, while the second part is a cost metric that will show how profitable a stock trading strategy based on the algorithm’s price predictions would be. There were many intermediate steps to implementing a final system. This included the selection of a proper machine learning framework, tuning of that framework’s hyperparameters, collection and pre-processing of appropriate input data, feature-engineering, and extensive testing to determine what system specifications would generate the most accurate predictions.
3. DESIGN PERFORMANCE
Overall, the model that achieved the greatest amount of accuracy was an LSTM. For short-term daily predictions, the features that provided the greatest amount of accuracy were strictly historical price data (open, high, low, close, and volume). For just day-ahead predictions, the algorithm was able to predict with 70-75% accuracy the direction that the stock price was going to move. However, there was a large amount of error associated with predicting the actual magnitude of these movements. In turn, this meant that the results in the table containing the top and bottom stock were rarely correct. Still, these results exceeded the 50% guessing margin, meaning the predictions could give an investor a slight edge in determining which stock prices will increase or decrease.
For long-term predictions of daily stock movements, the features that provided the greatest amount of accuracy were a combination of historical stock prices and fundamental data (including P/E ratio, ISM Manufacturing Index, ISM Non-Manufacturing Index, Consumer Sentiment, and number of housing permits issued). The long-term predictions for which stock prices would go up and down slightly exceeded the 50% benchmark of guessing for long-term price projections. This presented a substantial improvement on long-term predictions using only historical prices or historical prices paired with technical indicators. Despite the marginal success for long-term prediction, this is an indication that the algorithm’s predictive powers quickly denigrate as the number of days predicted for stock prices is increased.
In regards to mitigating the phenomenon of concept drift, I found that some techniques were more successful than others. For instance, a lagging window was ineffective (in fact, counterproductive) to increasing predictive accuracy. Meanwhile, the use of a stateful LSTM with a built-in memory and transforming a non-stationary times series into stationary data were effective at reducing this issue.
4. RECOMMENDATIONS AND FUTURE WORK
While my machine learning algorithm achieved some success at predicting the direction that stock prices would move, there is still a lot of room for improvement in its predictive accuracy. Furthermore, there is even more work to be done at developing a system that can effectively predict the magnitudes of these price changes, since my algorithm achieved little success in that area. While the ability to predict the correct direction of stock price movement at a rate that is better than guessing is a step in the right direction, knowing the magnitudes of these moves would be even more helpful to investors. This way they could allocate funds in the most optimal way to various stock to maximize their profits and minimize their losses.
The problem of predicting stock price movements using machine learning is one where huge amounts of future work could be done. Different machine learning frameworks would be implemented to test if their predictive accuracy would be greater than an LSTM. Adding even more complexity, groups of these different machine learning frameworks could have their results combined in an ensemble to further increase predictive accuracy. Figuring out which networks would make effective predictions and how to combine the predictions of different combinations of different networks is enough work for many other technical papers. Even beyond the scope of the machine learning framework itself, there is future work that could be performed on feature engineering the network inputs to get the most accurate predictions. This could entail utilizing a number of different combinations of numerical stock data to using natural language processing to analyze sentiment about the stock market in the media as well as even weather data. Because there are so many underlying factors that affect the stock market and investor attitudes, the problem of feature engineering could also be fodder for many papers. Therefore, there is a lot of room in this realm for future work, and my recommendation is experimenting and testing to find the best algorithms—an endeavor I will likely continue on in my future.
5. LESSONS LEARNED
Throughout the process of developing this process I learned a huge number of lessons. On the technical side, I learned Python as a new programming language along with its myriad of libraries and their possibilities. In particular, I learned about the powerful machine learning resources that are readily available. The project also forced me to learn some coding practices, such as creating a repository on GitHub to save a history of changes as I develop my code. Another hugely beneficial technical skill the project taught me was how to utilize cloud computing, again with a readily available resource like AWS.
Creating my first machine learning architecture also taught me some key fundamentals about not only machine learning, but also projects that require intensive data analysis. On the machine learning side, I was unaware of the vast amount of machine learning architectures available prior to my research. Delving deeper and deeper into the topic showed me that even once a network to work with has been selected, there is still a lot of fine-tuning that must go into selecting the optimal hyperparameters and input features. Different combinations of these aspects can drastically effect the predictive power of the network. Furthermore, because there are so many different variables at play and possible combinations, training and testing a machine learning algorithm is time intensive and requires many more experiments than I had initially anticipated. Furthermore, this huge commitment of time is in addition to the large amount of time it can already take to train and test an algorithm for one of these combinations. In general, the project was able to give me a lot more insight into what machine learning entails—i.e., that it is much more than a black box that data is fed into.
The data side of things also relates to this idea. Having a lot of data is essential to types of research like this, however, more data is not always better data. Feeding a machine learning algorithm data entails a lot of time and work that must go into pre-processing and cleaning the data so that a machine learning algorithm can make proper sense of it. As I debugged my algorithm, I had to pay close attention to the data itself. At first, when training and experimenting with my algorithm, my price predictions were off by huge magnitudes of range. This forced me to make two changes. The first was that I had to normalize the data to make sure all of the inputs were on the same scale and no one input was dominating the others. The second was that I realized I cared more about the predicting the changes in price rather than the actual price of the stock itself. This is because investors make money by buying a stock at a lower price and then selling when the price rises. Thus, I changed the metric that I was actually predicting, which led to increased success. Overall, these lessons show that as you come against unexpected obstacles in the framework of a problem, sometimes you have to change the way that you are approaching the problem itself.
