Financial Time Series Forecasting Using Empirical Mode Decomposition and FNN : A Study on Selected Foreign Exchange Rates

The exchange rate, an economic indicator of the country is the relative price of one country’s currency in terms of another country’s currency. The stability of the exchange rate is important for a stable economic growth. Exchange rate series are non-linear and non-stationary. The fluctuations in the forecasting exchange rate are very important to the economy of the country. Researchers have proposed many hybrid machine learning models to get a more accurate forecast. This study proposes a hybrid forecasting model using Empirical Mode Decomposition (EMD) and Feedforward Neural Network (FNN) for foreign exchange rates forecasting and comparing its performance with widely used Non-linear Autoregressive (NAR) and Support Vector Regression (SVR) models. EMD is used to decompose the original non-linear and non-stationary series into several Intrinsic Mode Functions (IMFs) and one residual. The hybrid model is then used to forecast the exchange rate with IMFs and residual obtained as inputs. Empirical results obtained from forecasting daily exchange rates of Sri Lankan Rupees to Euro and Yen showed that the proposed EMD-FNN model outperforms NAR and SVR models without time series decomposition.


I. INTRODUCTION
n international trading between two countries where one country has to make payments to another country in different currency, exchange rates are used.Exchange rates influence the trade balance, inflation, decisions of investors on foreign investments, worker remittances and the reserve position of a country.Thus, knowing the future values of exchange rates, a financial time series is vital.
Financial time series highly fluctuate and forecasting financial time series is a challenging task.There are two major challenges associated with forecasting financial time series namely non-stationarity and the statistical property of the time series changing with time and non-linearity.The issue of non-linearity in forecasting financial time series can be addressed by using machine learning models NAR and SVR to some extent.It was further attempted to improve the accuracy of forecasting the financial time series by introducing EMD (for eg.[14]).
Empirical Mode Decomposition (EMD) is used to decompose the original time series into a finite set of nearly orthogonal oscillating components, called intrinsic mode functions (IMFs) thus addressing both the issues in forecasting financial time series.
This research assesses the efficiency and the accuracy of the new hybrid NAR model proposed in literature and compares its performance with the machine learning models, NAR with SCG learning algorithm and SVR with Gaussian Radial Basis Kernel function without time series decomposition.Previous studies that are relevant to this research are presented in the Section II.The methodology used was briefly discussed under the Section III.The results obtained were presented in the Section IV followed by the Section V which compares the performance of models.Finally, the Section VI presents the conclusions of the study.

II. RELATED WORK
Initially, an empirical analysis was conducted by [1] to explore the main characteristics of stochastic behaviour of LKR and concluded that the common behaviour of LKR on the currency of any country has a property of non-linearity, non-stationary with stochastic trend and non-normality.There is a vast number of methods and methodologies used for forecasting purposes in various fields.Of all the methods and methodologies used in literature, three classes of models namely, stochastic models, Neural Network based models and Support Vector Regression based models were identified by [2] as the most popular ones for modelling financial time series.According to [3], [4], stochastic models such as random walk process and GARCH models could be used to represent time series.However, according to reference [5], GARCH models failed to capture the variations in the exchange rate series and concluded inefficient in forecasting exchange rates.The failure of stochastic models to provide a better forecast when the currency market was influenced by random events, motivated researchers to turn into machine learning models.Studies [6] and [7] showed evidence for the improvement in forecasting accuracy when exchange rates were modelled using Neural Networks.Authors of [8] and [9] convinced researchers that SVM had some predictive power and could be used to forecast financial time series and showed evidence for the increase in accuracy of financial time series forecasting.Our previous study [5] modelled three selected exchange rates using NAR and SVR.The results suggested that SVR outperforms other two modelling approaches by means of values accuracy and directional accuracy.
For many years, machine learning models are used more extensively in forecasting chaotic time series.Researchers are nowadays interested in developing hybrid neural network models that overcomes the shortcomings of the benchmark models by combining several techniques to increase the accuracy and practicability of neural network models.Some of these concepts which were found to perform better were I International Journal on Advances in ICT for Emerging Regions July 2018 text mining techniques embedded in decision algorithms [10], deep learning algorithms [11], [12], hybrid modelling approaches [13] etc. Recently, [14] argued that the performance of models based on innovative concepts can be enhanced by considering the impact of dynamic environmental conditions.They proposed an intelligent system supported with a hybrid approach which combines multiple algorithms such as EMD in signal processing to form the final approach with FNN with simple back propagation learning algorithm and showed evidence for the improvement in forecasting accuracy.
In this study, we improved the forecasting accuracy of the same exchange rates that we used in [5] by building a hybrid model by combining EMD and FNN and compared the performance of the new model with the NAR and SVR models.The proposed model decomposes the original series into IMFs and a residual using EMD and then it is decided to use the decomposed series as an input for the hybrid model.

A. Dataset
Daily exchange rates of USD, EUR and JPY to LKR from 2nd July, 2012 to 31st August, 2016 (1008 trade days) obtained from official website of Central Bank of Sri Lanka, were selected for the study since they are the most commonly trading countries with Sri Lanka.However, it was found that the exchange rate of USD/LKR was fixed during 2013 to 2015, thus modelling and forecasting USD/LKR would not be appropriate and removed from the analysis.The dataset was divided as shown in Table I to avoid overfitting of machine learning models and for comparison purposes.Financial time series are chaotic in nature and hence to identify the characteristics, preliminary analysis was carried and the characteristics of the series were examined.Then, to forecast the selected foreign exchange rate series, machine learning models namely NAR and SVR models and EMD-FNN model were fitted.The forecasting accuracy of the model fitted was compared using MSE and (Directional Accuracy) DA and the most accurate model for forecasting selected exchange rates was identified.

B. Neural Network Based Models
NAR is a recurrent dynamic neural network with feedback connections enclosing several layers of the network based on linear AR models, which are commonly used for modelling time series.The network is designed in such a way that the next value of the output is based on the previous values of the output signals.Since the output is again a feedback to the input, the network can be considered as a recurrent network.If at least one nonlinear activation function is used in this network it can be considered as a non-linear AR model.The NAR neural network can be designed either as a parallel architecture or series-parallel architecture.In parallel architecture, the output is fed back as the input to FNN whereas in series parallel architecture, true value is used instead of feeding back the estimated output as in [15].One of the most impressive features of neural networks is the ability to learn the non-linear complex patterns in the input and target as described in [16].For this study, SCG learning algorithm was used to train the data as it uses conjugate direction to capture the pattern and is found to be fast and very effective.

C. Support Vector Regression Based Models
Support vector regression is an extension of Support Vector Machine (SVM).According to [17], SVM use linear model to implement non-linear boundary classifier through some non-linear mapping into a higher dimensional feature space.A linear model constructed in the feature space can represent a non-linear decision boundary in the original space.An optimal separating hyperplane is constructed in the feature space termed as "maximum margin hyperplane" which provides the maximum separation among decision classes.When optimising the hyperplane, the training samples that lie on the hyperplane called support vectors are considered for the optimisation problem while other samples are considered irrelevant for defining binary class boundaries.This concept is modified in SVR to fit a regression line.

D. Empirical Mode Decomposition
Empirical Mode Decomposition (EMD), a comparatively recent development in signal processing methods, makes use of the Hilbert transform [15] to decompose a time series into a number of components each with slowly varying amplitude and phase.It is an adaptive method that decomposes the signals into components that separate phenomena occurring on different time scales.Each component of the EMD is called an Intrinsic Mode Function (IMF).Two characteristics of IMFs noted by [18] that resemble a generalised Fourier decomposition are: An IMF only has one zero between successive extrema.An IMF must have zero local mean.The computation of IMFs that satisfies the criteria mentioned above and sum to create the signal explained by [18] are: 1.The highest frequency of IMF is determined by first fitting a cubic spline through all local maxima to create an upper envelope.2. Similarly, a lower envelope is constructed by fitting a cubic spline through all local minima.3. Envelopes together form the candidate for the timevarying amplitude of IMF 4. It makes sure that the component function has a negligible local mean.If the average of two envelopes does not fall uniformly within the threshold value of zero, the mean is subtracted from the envelope.This step is repeated until the second criterion is satisfied.5.The first IMF is obtained as a result of the inner loop and the outer loop gives the residue.6. Steps through 1 to 5 are repeated until the residue obtained is a constant or a monotone.
This procedure decomposes the series as follows: (1) where is the time series to be decomposed.EMD outputs IMFs of a very close visual match to the original signal components, with nearly indistinguishable amplitudes and frequencies thus addressing the issue of non-stationarity and multi-scaling behaviour of financial time series.

E. EMD-FNN models
The proposed hybrid ANN model is a combination of EMD and ANN models.EMD is used to decompose the original series into locally orthogonal set.These IMFs and residual are fed into the FNN with SBP learning algorithm as input and the forecasted value is obtained as the output.

F. Performance criteria
The value forecasting accuracy and the directional forecasting accuracy were measured using Mean Square Error (MSE) and Directional Accuracy (DA) respectively. ( where is the actual value, is the forecasted value and is the number of observations. (3) where, The model with high DA representing high directional accuracy and low MSE representing fitted value close to actual value was chosen as the best model for forecasting.

IV. RESULTS
The results of the descriptive analysis and the model fitting procedure are presented in this section.
A. Preliminary Analysis Time plots of the exchange rates are examined in order to identify their behaviour.Figure 2  Fig. 3 reveals that the ACF of the original series are highly significant for more lags and does not decline quickly to zero.The results obtained from the unit root test of EUR/LKR indicated evidence for the presence of stochastic trend without any drift or deterministic trend in the original series.The 1 st differenced series did not show any stochastic trend or drift and became stationary.Thus, implies where is the exchange rate of EUR/LKR on time .
The return series of EUR/LKR was obtained using the equation, (  Fig. 6 shows that the ACF of JPY/LKR series are highly significant for more lag values and does not decline quickly to zero whereas PACF of JPY/LKR are not significant The autoregressive terms included in the model were selected by fixing default values to the parameters and changing the number of lags included.The MSE values and the adequacy of the model were compared for many iterations.The one that gave minimum MSE with adequacy for most of the iterations was chosen as the optimal lag.When the number of hidden layers included in the model was increased from one, the MSE value did not decrease much.Thus, one hidden layer was included in the model.The number of nodes in the hidden layer and the learning algorithm parameters sigma and lambda were found using trial and error method.The model was repeatedly fitted with the same set of parameters for each of the 20 iterations.Each time the model trained, MSE values of training and validation sets kept changing and the adequacy of the model was fluctuating between adequate and inadequate.A model is considered to be adequate if the squared correlation is not significant for lower lag values.Different combinations of the parameters were tried and different MSE values and adequacy were obtained for each iteration.However, when the optimal set of parameters was used, the MSE value and the adequacy did not change drastically for different iterations.The models thus obtained have been discussed below.
The architecture of the designed NAR for forecasting EUR/LKR included 8 autoregressive terms with one hidden layer made up of 14 nodes and one output layer made up of one node.
function was used to pre-process the data in the input layer and , and transfer functions were used in the hidden and output layer respectively.The optimal learning algorithm parameters selected were 1e-10 for sigma and 1e-7 for lambda.The designed NAR for EUR/LKR used SCG as the learning algorithm to train the dataset and MSE as a measure to check the performance of the network.According to Fig. 8, the designed NAR described the dataset well with error ranging from -4 to +4 through zero.Also, there was no significant autocorrelation for lower lag values among the residuals and cross-correlation Architecture of the designed NAR for forecasting JPY/LKR included 12 autoregressive terms with one hidden layer of 18 nodes and one output layer of one node.
function was used to pre-process the data in the input layer and , and transfer functions were used in the hidden and output layer respectively.The optimal learning algorithm parameters included were 1e-10 for sigma and 1e-10 for lambda.The designed NAR used SCG as a learning algorithm to train the dataset with MSE used as a measure to check the performance of the network.9 shows that the designed NAR has described the dataset well with error ranging from -0.1 to +0.1 through zero.Also, there was no significant autocorrelation for lower lag values among the residuals and no significant crosscorrelation between the residual and input terms.Thus, the NAR model fitted was adequate.
International Journal on Advances in ICT for Emerging Regions July 2018

B. SVR MODELS
The SVR model was fitted in return for the exchange rate obtained by, (7) on training dataset with the function in R programming.The parameters in the model were lag values, epsilon, cost and sigma corresponding to the selected kernel function and the Gaussian radial basis kernel function.
Initially, by allowing default values for epsilon, sigma and cost, the lag values were changed from 3 to 9 and the best lag value was chosen as the one with minimum MSE in the training set.Then, the cost and sigma values were fixed and epsilon value was changed between 0 and 1.The epsilon value with the minimum MSE value in the validation set was selected.Then, the selected epsilon value and default sigma value were fixed and cost was changed between 0 and 2 10 .The cost value with the minimum MSE value in the validation set was selected.Again, the selected epsilon value and cost value were fixed and sigma was changed between 0 and 1 and the combination with minimum MSE value in the validation set was selected.The process was repeated until a minimum MSE value was obtained in the validation set.Different combinations of values were obtained with different MSE values for training and validation set.The set of values with the minimum MSE in training and validation set was chosen as the optimal set of values.SVR thus designed were discussed below.Six autoregressive terms were included in the model as the MSE value was minimum when lag value was six.Table II shows a few combinations of values, their corresponding minimum MSE values for SVR model in returns for EUR/LKR.The optimal set chosen were epsilon = 0.09, sigma = 0.9, cost = 1 as these combinations of parameter values gave a minimum MSE value in both training as well as validation set.Thus, the SVR model was designed including 6 autoregressive terms with an insensitive loss of 0.09, cost of 1 and Gaussian radial basis kernel function with sigma of 0.9.The designed SVR model was well fitted with the training set (Fig. 10).Also, the model captured the variation in the validation set but underestimated in few cases (Fig. 11).Autocorrelation of residuals in the training set as well as in the validation set were not significant for lower lag values.The autocorrelation of squared residuals in training set and validation set were not significant for lower lag values.Hence, all the patterns in the training as well as validation set were captured by the model.Thus, the designed model was adequate.Six autoregressive terms were included in the model as the MSE value was minimum when lag value was 6. Table III shows   The designed SVR model is fitted with the training set well (Fig. 12).The model also captured the variations in validation set with a few exceptions (Fig. 13).The autocorrelation of errors and autocorrelation of squared errors were not significant for lower lags in the training as well as validation set.Hence, all the patterns in the series were captured by the model.Thus, the designed SVR model was adequate.

D. EMPIRICAL MODE DECOMPOSITION OF EXCHANGE RATES
The exchange rate of EUR/LKR was decomposed into 10 IMFs using EMD (Fig. 14) The exchange rate of JPY/LKR was decomposed into 7 IMFs using EMD (Fig. 15).

E. EMD-FNN MODEL
The identified IMFs and residual series as the inputs were fed into FNN trained with simple back propagation as learning algorithm using R-programming.The parameters to be optimised are number of successive past observations of a time series to be included, number of hidden layers and number of nodes in the hidden layers.
The optimal value for the above-mentioned parameters was identified by trial and error method.
International Journal on Advances in ICT for Emerging Regions July 2018 The model with minimum MSE value for both training and validation set was selected as the one model with optimal parameter values.First, the model for forecasting EUR/LKR was trained for 20 iterations each by allowing one lag term and inserting one hidden layer with varying number of nodes and two hidden layers with different number of nodes.Then, the model was trained for 20 iterations each by allowing two lag terms and inserting one hidden layer with different number of nodes and two hidden layers with different number of nodes.The procedure was repeated allowing 3 lag terms and it was found that the increase in the number of lag terms did not show any significant increase in the model performance.Hence, one was selected as the optimal value for number of lags to be considered.
Next, to find out the number of hidden layers and number of nodes in each layer, the same approach was followed.The FNN model with one lag term as input with one hidden layer was fitted with different number of nodes each with 10 iterations.The minimum MSE values obtained for different number of nodes included were shown in Table IV.Thus, one node was selected as the optimal number of nodes in the hidden layer 1. Next, the procedure was followed until the optimal number of hidden layers and their corresponding numbers of nodes were found.Tables V   As the procedure was repeated further including hidden layers, the performance in the neural network did not vary significantly.Thus, three was selected as the optimal number of hidden layers with one node in hidden layer 1, five nodes in hidden layer 2 and four nodes in hidden layer 3.   The designed EMD-FNN model for EUR/LKR captured the variation in the series very well and forecasted the training set (Fig. 16) and the validation set (Fig. 17) well.The autocorrelation of errors and autocorrelation of squared errors were not significant for lower lags in the training as well as validation set.Hence, all the pattern in the series was captured by the model.Thus, the designed Hybrid ANN model for EUR/LKR was adequate.Now, the model for forecasting JPY/LKR was trained for 20 iterations each by allowing one successive past observation and inserting one hidden layer with different number of nodes and two hidden layers with different number of nodes.Then, the model was trained for 20 iteration each by allowing two successive past observations and inserting one hidden layer with different number of nodes and two hidden layers with different number of nodes.The procedure was repeated allowing 3 successive past observations and it was found that the increase in the number of successive past observations did not show any significant increase in the model performance.Hence, the model with one successive past observation was selected as the optimal model.
Next, to find out the number of hidden layers and number of nodes in each layer, the same approach was followed.The ANN model with one lag term as input with one hidden layer was fitted with different number of nodes each with 20 iterations.The minimum MSE values obtained for different number of nodes included were shown in Table VII.Thus, three nodes were selected as the optimal number of nodes in the hidden layer 1. Next, the procedure was followed until the optimal number of hidden layers and their corresponding numbers of nodes were found.Table VIII  As the procedure was repeated further including hidden layers, the performance in the neural network did not vary significantly.Thus, two was selected as the optimal number of hidden layers with three nodes in hidden layer 1 and one nodes in hidden layer 2. The designed EMD-FNN model for JPY/LKR captured the variation in the series very well and forecasted exchange rates corresponding to the training set (Fig. 18) and the validation set (Fig. 19) well.The autocorrelation of errors and autocorrelation of squared errors were not significant for lower lags in the training as well as validation set.Hence, all the patterns in the series were captured by the model.

F. COMPARISON OF FITTED MODELS
The forecasting accuracy of the models was compared using the test dataset.A multistep ahead forecast for 41 time points were obtained and the values and directional forecasts were compared using MSE and DA respectively.
Table X compares the forecasting accuracy of models fitted for the selected exchange rates.The conclusions obtained were based on the datasets selected for the study.All models performed well in forecasting the daily exchange rates selected.However, the proposed hybrid EMD-FNN model gave a better value forecast as well as a directional accuracy for both datasets.
The results might differ for different datasets.Also, according to [19], the country had adopted floating exchange rate system since 2001 allowing an independent adjustment of exchange rate depending on the market forces.However, in practice, they have deviated from this stated policy and intervened to stabilise the exchange rate as in [20].These findings could be useful to domestic as well as foreign users.Further, the forecasting ability can be improved by introducing evolutionary neural networks that have significant predictive and market timing ability.
Manuscript received on 15 th Dec 2017.Recommended by Dr. M. G. N. A. S. Fernando on 12 th June 2018.This paper is an extended version of the paper "A Comparison of Accuracy of Forecasting Models: A Study on Selected Foreign Exchange Rates" presented at the ICTer 2017.Ms. P. Nanthakumaran and Dr. C. D. Tilakaratne are from the Department of Statistics, University of Colombo.(prathananthan@gmail.com, cdt@stat.cmb.ac.lk).

Fig 1
explains the procedure followed in fitting the proposed EMD-FNN forecasting model for the exchange rates series.

Fig1.
Fig1.The proposed EMD-FNN forecasting model for exchange rates and 3 depicts the time plot and correlogram of EUR/LKR series respectively.According to Fig 2, EUR/LKR shows random fluctuations without any seasonality during the study period.

Fig 2
Fig 2 Time plot of EUR/LKRAlso, the partial autocorrelations of the original series are not significant for lower lags except for lag 1.Thus, EUR/LKR series might consist of a stochastic trend and is not stationary.
Fig. 3 ACF plot of EUR/LKR

Fig. 4
Fig. 4 represents the return series of EUR/LKR and shows that high periods of volatility are followed by low periods of volatility.Return series indicate the presence of volatility clusters.

Fig 4
Fig 4 Return series of EUR/LKR Figure 5 and 6 depicts the time plot and correlogram of JPY/LKR series respectively.According to Fig 5, JPY/LKR shows random fluctuations without any seasonality during the study period.Fig.6shows that the ACF of JPY/LKR series are highly significant for more lag values and does not decline quickly to zero whereas PACF of JPY/LKR are not significant

Fig. 5 Fig 7
Fig. 5 Time plot of JPY/LKR Fig 8 depicts the residual plot of the designed NAR model for EUR/LKR.

Fig 8
Fig 8 Residual plot between the residual and input terms.Thus, the NAR model fitted was adequate.Architecture of the designed NAR for forecasting JPY/LKR included 12 autoregressive terms with one hidden layer of 18 nodes and one output layer of one node.functionwas used to pre-process the data in the input layer and , and transfer functions were used in the hidden and output layer respectively.The optimal learning algorithm parameters included were 1e-10 for sigma and 1e-10 for lambda.The designed NAR used SCG as a learning algorithm to train the dataset with MSE used as a measure to check the performance of the network.Fig 9 depicts the residual plot of the designed NAR model for JPY/LKR.

Fig 9
depicts the residual plot of the designed NAR model for JPY/LKR.

Fig 9
Fig 9 Residual plot Fig.9shows that the designed NAR has described the dataset well with error ranging from -0.1 to +0.1 through zero.Also, there was no significant autocorrelation for lower lag values among the residuals and no significant crosscorrelation between the residual and input terms.Thus, the NAR model fitted was adequate.

Fig 10 and 11
illustrates the comparison of the fitted value and the actual value of the training dataset and the validation dataset of EUR/LKR respectively.

Fig 10
Fig 10 Plot of fitted vs actual value in training set a few combinations of values and their corresponding minimum MSE values for SVR model in return for JPY/LKR.The optimal set chosen was epsilon = 0.1, sigma = 0.55, cost = 20 as these combinations of parameter values gave a minimum MSE value in both training as well as validation set.Thus, SVR model for JPY/LKR series was designed by including 6 autoregressive terms with an insensitive loss of 0.1, cost of 20 and Gaussian radial basis kernel function with sigma of 0.55.Fig 12 and 13 illustrates the comparison between the fitted value and the actual value in the training dataset and the validation dataset of JPY/LKR respectively.

Fig 12
Fig 12 Plot of fitted vs actual value in training set

Fig. 15
Fig. 15 IMFs obtained for JPY/LKR series and VI show the MSE values obtained for training and validation datasets of EUR/LKR in the training process of EMD-FNN model.

Fig 16 and 17
compares the actual value and forecasted value obtained from EMD-FNN model corresponding to the training set and the validation set of EUR/LKR respectively.P. Nanthakumaran #1 , C. D. Tilakaratne #2 July 2018 International Journal on Advances in ICT for Emerging Regions

Fig 16
Fig 16 Plot of fitted vs actual values in the training set and IX show the MSE values obtained for training and validation datasets of JPY/LKR in the training process of EMD-FNN.
Fig 18 and 19 compares the actual value and forecasted value obtained from EMD-FNN model corresponding to the training set and the validation set of JPY/LKR respectively.

Fig 18
Fig 18 Plot of fitted vs actual in the training set Fig 20 and 21 visualise and compare the actual value and the forecasted values obtained from different models fitted namely NAR model, SVR model and EMD-FNN model for EUR/LKR and JPY/LKR respectively.According to Fig. 20, the forecasted values of EMD-FNN models were close to the actual values than that of other machine learning models when considering EUR/LKR.According to Fig. 21, the forecasted values of hybrid ANN models are close to the actual values of the testing period, than those of other machine learning models for JPY/LKR.July 2018 International Journal on Advances in ICT for Emerging Regions

Fig 20
Fig 20 Graphical comparison of forecasting accuracy of different models for EUR/LKR

TABLE II SELECTION
OF PARAMETERS FOR SVR OF EUR/LKR RETURNS

TABLE III SELECTION
OF PARAMETERS FOR SVR OF JPY/LKR RETURNS

TABLE IV SELECTION
OF NUMBER OF NODES IN HIDDEN LAYER 1

TABLE V SELECTION
OF NUMBER OF NODES IN HIDDEN LAYER 2

TABLE VI SELECTION
OF NUMBER OF NODES IN HIDDEN LAYER 3

TABLE VII SELECTION
OF NUMBER OF NODES IN HIDDEN LAYER 1

TABLE VIII SELECTION
OF NUMBER OF NODES IN HIDDEN LAYER 2 According to Table X, hybrid ANN model provided the best directional accuracy of about 67.5% and provided a good value forecast (minimum MSE) when considering EUR/LKR.Table X also showed that hybrid ANN model provided the best directional accuracy of about 77.5% and provided a good value forecast for JPY/LKR.