MOSIQS: Persistent Memory Object Storage With Metadata Indexing and Querying for Scientific Computing
Autor: | Youngjae Kim, Hyogi Sim, Sudharshan S. Vazhkudai, Awais Khan |
---|---|
Jazyk: | angličtina |
Rok vydání: | 2021 |
Předmět: |
010302 applied physics
General Computer Science Computer science Search engine indexing General Engineering Memory pool 020206 networking & telecommunications 02 engineering and technology Data structure persistent memory storage 01 natural sciences Computational science TK1-9971 Object storage Metadata Memory management Shared memory 0103 physical sciences Scalability 0202 electrical engineering electronic engineering information engineering General Materials Science PM~index data structures Electrical engineering. Electronics. Nuclear engineering Memory-centric computing and HPC scientific metadata indexing and search |
Zdroj: | IEEE Access, Vol 9, Pp 85217-85231 (2021) |
ISSN: | 2169-3536 |
Popis: | Scientific applications often require high-bandwidth shared storage to perform joint simulations and collaborative data analytics. Shared memory pools provide a chance to satisfy such needs. Recently, a high-speed network such as Gen-Z utilizing persistent memory (PM) offers an opportunity to create a shared memory pool connected to compute nodes. However, there are several challenges to use scientific applications on the shared memory pool directly such as scalability, failure-atomicity, and lack of scientific metadata-based search and query. In this paper, we propose MOSIQS, a persistent memory object storage framework with metadata indexing and querying for scientific computing. We design MOSIQS based on the key idea that memory objects on PM pool can live beyond the application lifetime and can become the sharing currency for applications and scientists. MOSIQS provides an aggregate memory pool atop an array of persistent memory devices to store and access memory objects to accelerate scientific computing. MOSIQS uses a lightweight persistent memory key-value store to manage the metadata of memory objects, which enables memory object sharing. To facilitate metadata search and query over millions of memory objects resident on memory pool, we introduce Group Split and Merge (GSM), a novel persistent index data structure designed primarily for scientific datasets. GSM splits and merges dynamically to minimize the query search space and maintains low query processing time while overcoming the index storage overhead. MOSIQS is implemented on top of PMDK. We evaluate the proposed approach on many-core server with an array of real PM devices. Experimental results show that MOSIQS gains a 100% write performance improvement and executes multi-attribute queries efficiently with $2.7\times $ less index storage overhead offering significant potential to speed up scientific computing applications. |
Databáze: | OpenAIRE |
Externí odkaz: |