| Popis: |
In this thesis we have investigated reinforcement Learning for multiple time series data as a solution for finding optimal actions at each time point. The main aspect of the problem addressed by us is the manner in which the state of the system is defined, and the way it is inferred from the observed values of the multiple time series. Our contribution is to define the states in terms of the Vector-Auto-Regression models that fit the multiple time series for short time windows. We then present a reinforcement learning (Q-Learning) technique for predicting optimal actions based on the observed multiple time series data. Most of the existing systems use binning of the time series data to define states, primarily in terms of the bin identities. We have demonstrated the superior performance with our proposed VAR-model based description of the state of the system. This method for state description is effective in characterizing the state of the system and includes the modeling of inter-dependencies among various time series in the state description. The state descriptions are constituted using four currency exchange rate time series data and the Q-learning based system is used for predicting buy, sell or hold actions for specific currencies. We have compared our method of defining states with traditional technique using binning of the observed data. The results of our experiments show that our proposed method for state descriptions performs much better and is very robust in the context of choosing some parameters for our overall learning framework. |
