An alternative C++-based HPC system for Hadoop MapReduce

Autor: Srinivasakumar Vignesh, Vanamoorthy Muthumanikandan, Sairaj Siddarth, Ganesh Sainath
Jazyk: angličtina
Rok vydání: 2022
Předmět:
Zdroj: Open Computer Science, Vol 12, Iss 1, Pp 238-247 (2022)
Druh dokumentu: article
ISSN: 2299-1093
DOI: 10.1515/comp-2022-0246
Popis: MapReduce (MR) is a technique used to improve distributed data processing vastly and can massively speed up computation. Hadoop and MR rely on memory-intensive JVM and Java. A MR framework based on High-Performance Computing (HPC) could be used, which is both memory-efficient and faster than standard MR. This article explores a C++-based approach to MR and its feasibility on multiple factors like developer friendliness, deployment interface, efficiency, and scalability. This article also introduces Eager Reduction and Delayed Reduction techniques to speed up MR.
Databáze: Directory of Open Access Journals