GPU Computations on Hadoop Clusters for Massive Data Processing

Autor: Wenbo Chen, Mario Donato Marino, Tien-Hsiung Weng, Shungou Xu, Hai Jiang, Yi-Siang Chen, Kuan-Ching Li
Rok vydání: 2016
Předmět:
Zdroj: Lecture Notes in Electrical Engineering ISBN: 9783319173139
Popis: Hadoop is a well-designed approach for handling massive amount of data. Comprised at the core of the Hadoop File System and MapReduce, it schedules the processing by orchestrating the distributed servers, providing redundancy and fault tolerance. In terms of performance, Hadoop is still behind high performance capacity due to CPUs’ limited parallelism, though. GPU accelerated computing involves the use of a GPU together with a CPU to accelerate applications to data processing on GPU cluster toward higher efficiency. However, GPU cluster has low level data storage capacity. In this chapter, we exploit the hybrid model of GPU and Hadoop to make best use of both capabilities, and the design and implementation of application using Hadoop and CUDA is presented through two interfaces: Hadoop Streaming and Hadoop Pipes. Experimental results on K-means algorithm are presented as well as their performance results are discussed.
Databáze: OpenAIRE