| Popis: |
In finance, a persistent issue in classification task has been the long dependency. For example, how does a model learn from the abnormal London Interbank Offer Rate movement months before the 2008 financial crisis and apply that knowledge in today’s market? For traditional time series modeling and for basic deep learning model such as long short-term memory, such knowledge is hard for a model to capture. To ameliorate this problem, we present a novel architecture, residual attention net (RAN), which merges a sequence architecture, universal transformer, and a computer vision architecture, residual net, with a high-way architecture for cross-domain sequence modeling. This approach helps the model “remember” time series events that happened long before the target event. The architecture aims at addressing the long dependency issue often faced by recurrent neural net-based structures. This chapter serves as a proof-of-concept for a new architecture, with RAN aiming at providing the model a higher level understanding of sequence patterns. To our best knowledge, we are the first to propose such an architecture. Out of the standard 85 UCR data sets, we have achieved 35 state-of-the-art results with 10 results matching current state-of-the-art results without further model fine-tuning. The results indicate that such architecture is promising in complex, long-sequence modeling and may have vast, cross-domain applications. |
