masFS: File System Based on Memory and SSD in Compute Nodes for High Performance Computers
| Autor: | Jieting Wu, Xin Liu, Chunjia Wu, Yutong Lu, Ying Lu |
|---|---|
| Rok vydání: | 2016 |
| Předmět: |
File system
020203 distributed computing Speedup Computer science business.industry 020206 networking & telecommunications 02 engineering and technology Parallel computing computer.software_genre Supercomputer Memory-mapped file Dynamic memory management POSIX Server Computer data storage 0202 electrical engineering electronic engineering information engineering Operating system Lustre (file system) business computer Data migration |
| Zdroj: | ICPADS |
| DOI: | 10.1109/icpads.2016.0081 |
| Popis: | Hard Disk Drive (HDD) based storage systems in high performance computing (HPC), such as Lustre, are being confronted with great challenges from massive parallel data-intensive applications, and becomes one of the most critical factors that influence applications performance. In this paper, we analyze the performance of Lustre, and statistically examine the distribution of file sizes, I/O patterns for typical data-intensive applications, as well as the utilization of CPU and memory resources in TH-1A supercomputer from the last five years. We present masFS, a novel file system for HPC that exploits available memory and SSD resources on compute nodes with little interference to applications running on the nodes. The masFS supports POSIX interface with an unique namespace and provides comprehensive file service for applications. We design and implement masFS with memory and SSD, and have deployed and evaluated it on TH-1A. Experimental results show that masFS works as general storage system and can meet the requirement of variety of I/O loads. It can achieve a speedup of 14.5x for read and 8.8x for write respectively in running benchmarks, and an average 7.6x faster I/O time in a real-world data-intensive application compared to Lustre. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|