System-Level Scalable Checkpoint-Restart for Petascale Computing
| Autor: | Jerome Vienne, Kapil Arya, Gene Cooperman, Shawn Matott, Jiajun Cao, Dhabaleswar K. Panda, Rohan Garg, Hari Subramoni |
|---|---|
| Rok vydání: | 2016 |
| Předmět: |
FOS: Computer and information sciences
D.1.3 020203 distributed computing Multi-core processor Operating Systems (cs.OS) C.4 Computer science InfiniBand 020206 networking & telecommunications Fault tolerance 02 engineering and technology Parallel computing Supercomputer Virtualization computer.software_genre D.2.11 Computer Science - Operating Systems Petascale computing Computer Science - Distributed Parallel and Cluster Computing 0202 electrical engineering electronic engineering information engineering Memory footprint Distributed Parallel and Cluster Computing (cs.DC) computer |
| Zdroj: | ICPADS |
| Popis: | Fault tolerance for the upcoming exascale generation has long been an area of active research. One of the components of a fault tolerance strategy is checkpointing. Petascale-level checkpointing is demonstrated through a new mechanism for virtualization of the InfiniBand UD (unreliable datagram) mode, and for updating the remote address on each UD-based send, due to lack of a fixed peer. Note that InfiniBand UD is required to support modern MPI implementations. An extrapolation from the current results to future SSD-based storage systems provides evidence that the current approach will remain practical in the exascale generation. This transparent checkpointing approach is evaluated using a framework of the DMTCP checkpointing package. Results are shown for HPCG (linear algebra), NAMD (molecular dynamics), and the NAS NPB benchmarks. In tests up to 32,752 MPI processes on 32,752 CPU cores, checkpointing of a computation with a 38 TB memory footprint in 11 minutes is demonstrated. Runtime overhead is reduced to less than 1%. The approach is also evaluated across three widely used MPI implementations. 18 pages, 5 figures, to be published in ICPADS 2016 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|