CNNs VERSUS LSTMs FOR TIME SERIES FORECASTING

Author

Bidur Prasad Bhurtel, Southern Illinois University CarbondaleFollow

Date of Award

6-1-2021

Degree Name

Master of Science

Department

Electrical and Computer Engineering

First Advisor

Gupta, Lalit

Abstract

The goal of this thesis is to compare the performances of long short-term memory (LSTM) recurrent neural networks and feedforward convolution neural networks (CNNs) in time series forecasting. The forecasting problem focuses on predicting the future values of a time series using the current and a set of previous (lagged) values of the time series. LSTMs are used extensively in time series forecasting problems because they are specifically designed to process sequential and temporal data. CNNs on the other hand are not designed to process such sequential data. Although CNNs appear to be a poor choice for time series forecasting, it would be informative to compare the performances of CNNs and LSTMs by using exactly the same set of current and previous values of several combinations of multiple time-series. The specific forecasting problem considered involves predicting the outflow for a creek sub-basin using outflow, temperature, and precipitation time series consisting of 185,544 hourly datapoints. The Granger causality (GC) test is used to confirm that temperature and precipitation Granger cause outflow and should, therefore, be helpful in predicting outflow. The GC test also provides information on the lag required to influence outflow prediction. The forecasting problem is divided into developing 3 distinct types of models: one-hour forecast models, 2-hour forecast models, and a one-hour and 2-hour extreme-event forecast models. The one-hour forecast models are trained to predict the next-hour outflow using the current and a set of previous values of the time series. The 2-hour forecast models are trained to predict the outflow 2 hours ahead using the current and previous values of the time series. A variation of the two-hour model is trained with the previous, current, and the next-hour prediction of the one-hour model. The extreme-event forecast models are trained only with segments of the time series containing extreme events. The performance of the LSTM and CNN implementations of the models are compared objectively using the mean square error and subjectively by comparing the predictions visually. The results from various combinations of the creek sub-basin time series show that the CNN models outperform the LSTM models. These results are quite unexpected given that CNNs appear to be a poor choice for time series forecasting whereas LSTMs are a good choice. The primary reason for the CNN models yielding superior performance is that through the choice of appropriate filters, CNNs are capable of generating complex features which are coupled simultaneously across time series predictor variables and across time lags in the series of convolution layers. However, it cannot be concluded that CNNs will, in general, outperform LSTMs without testing the models on a large ensemble of diverse data sets.