Accelerated Purge Processes of Parallel File System on HPC by Using MPI Programming

Autor:	JunWeon Yoon, ChanYeol Park, Taeyoung Hong, Min-Woo Kwon
Rok vydání:	2017
Předmět:	File system Hardware_MEMORYSTRUCTURES Temporary file Computer science Large capacity ComputingMilieux_LEGALASPECTSOFCOMPUTING Directory Load balancing (computing) computer.software_genre Purge Scratch Operating system Lustre (file system) computer computer.programming_language
Zdroj:	Advances in Computer Science and Ubiquitous Computing ISBN: 9789811076046 CSA/CUTE
DOI:	10.1007/978-981-10-7605-3_181
Popis:	HPC system usually uses a shared filesystem like Lustre as temporary file storage like scratch directory. It needs automated purge process to remove unused files for maintaining optimal performance of the shared filesystem. However, the purge process in large capacity file system takes much time to search and remove target files. In this paper, accelerated purge processes using MPI are proposed. First, master/slave parallel purge (MSPP) process is the method that master node distributes purge tasks among slave nodes. Second, evenly distributed purge (EDPP) process is the method that all node are involved in purge process that improves load balancing. Experimental results show that the purging time of proposed EDPP method for 1958 GB scratch data has been reduced by 4.04 and 2.47 times, respectively, when it is compared with the results of single node purge (SNP) and MSPP methods.
Databáze:	OpenAIRE
Externí odkaz:	https://explore.openaire.eu/search/publication?articleId=doi_________::c2e72b98b06ee9b9afa000d8dd3e46ed https://doi.org/10.1007/978-981-10-7605-3_181 Zobrazit plný text záznamu