| Popis: |
Among the various algorithms present for data mining, the UCR Dynamic Time Warping (DTW) suite provided a solution to search and mine large data sets of time series data more efficiently as compared to the previously existing method of using Euclidean Distance. The UCR DTW algorithm was developed for a single CPU core. In this paper, we consider 2 methods of parallelizing the DTW algorithm. First, we consider a multi-core implementation, followed by a cluster implementation using Spark. From the multi-core implementation, we achieve nearly linear speedup. In the Spark implementation, we find that a straightforward implementation of DTW does not perform well. This is because; a major step in DTW is parallel computation of a lower bound. This paradigm is not supported well by Spark, which supports (i) broadcast variables that are broadcasts of read-only variables (ii) accumulation variables that represent distributed sums. We show how to compute distributed lower bounds efficiently in Spark and achieve nearly linear speedup with DTW in a Spark computation as well. |
