UGSD
| Autor: | Xin Liu, Chunjia Wu, Yutong Lu, Jieting Wu, Yachao Shao |
|---|---|
| Rok vydání: | 2017 |
| Předmět: |
File system
Information retrieval Data element Database Computer science Storage Resource Broker Meta Data Services 02 engineering and technology computer.software_genre Metadata repository Metadata 020204 information systems Metadata management Data_FILES 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing computer Database catalog |
| Zdroj: | ICCDA |
| DOI: | 10.1145/3093241.3093257 |
| Popis: | In existing ultra-large-scale file systems in HPC environment, file data and metadata are stored and processed separately. Efficient and scalable metadata management is one of the most critical aspects that may affect overall performance of file systems, especially in the EB era. Directory subtree partitioning and hashing function are two main techniques in metadata management. However, both of them have either low efficiency or poor performance. We present a User Group Sub-Directory (UGSD) metadata management system that separates metadata storage and services. It partitions the metadata with the granularity of User Group Sub-Directory, so as to preserve the inherent tree structure of namespace and directory locality. We use a two-level MOD mapping function and a mapping table to locate metadata, and to efficiently map the metadata storage and services between Metadata Server (MDS) and Metadata Storage Server (MDSS). We propose a synchronous updating mechanism and a peak-shaving scheme for dynamic adjustment of the number of MDS and the workloads without metadata migration. UGSD takes full advantages of subtrees partitioning and hashing function, and achieves obvious better performance. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|