Local Rollback for Resilient Mpi Applications With Application-Level Checkpointing and Message Logging
| Autor: | María Martín, Aurelien Bouteiller, Patricia González, Nuria Losada, George Bosilca |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
Local rollback
Resilience Computer Networks and Communications Computer science Interface (Java) Message logging Distributed computing Message Passing Interface 020206 networking & telecommunications 02 engineering and technology computer.software_genre Application-level checkpointing Consistency (database systems) Hardware and Architecture 0202 electrical engineering electronic engineering information engineering Overhead (computing) 020201 artificial intelligence & image processing MPI Compiler Resilience (network) computer SPMD Software Rollback |
| Zdroj: | RUC: Repositorio da Universidade da Coruña Universidade da Coruña (UDC) RUC. Repositorio da Universidade da Coruña instname |
| Popis: | [Abstract] The resilience approach generally used in high-performance computing (HPC) relies on coordinated checkpoint/restart, a global rollback of all the processes that are running the application. However, in many instances, the failure has a more localized scope and its impact is usually restricted to a subset of the resources being used. Thus, a global rollback would result in unnecessary overhead and energy consumption, since all processes, including those unaffected by the failure, discard their state and roll back to the last checkpoint to repeat computations that were already done. The User Level Failure Mitigation (ULFM) interface – the last proposal for the inclusion of resilience features in the Message Passing Interface (MPI) standard – enables the deployment of more flexible recovery strategies, including localized recovery. This work proposes a local rollback approach that can be generally applied to Single Program, Multiple Data (SPMD) applications by combining ULFM, the ComPiler for Portable Checkpointing (CPPC) tool, and the Open MPI VProtocol system-level message logging component. Only failed processes are recovered from the last checkpoint, while consistency before further progress in the execution is achieved through a two-level message logging process. To further optimize this approach point-to-point communications are logged by the Open MPI VProtocol component, while collective communications are optimally logged at the application level—thereby decoupling the logging protocol from the particular collective implementation. This spatially coordinated protocol applied by CPPC reduces the log size, the log memory requirements and overall the resilience impact on the applications. This research was supported by the Ministry of Economy and Competitiveness of Spain and FEDER funds of the EU (Projects TIN2016-75845-P and the predoctoral grants of Nuria Losada ref. BES-2014-068066 and ref. EEBB-I-17-12005); by EU under the COST Program Action IC1305 Network for Sustainable Ultrascale Computing (NESUS) and a HiPEAC Collaboration Grant and by the Galician Government (Xunta de Galicia) under the Consolidation Program of Competitive Research (ref. ED431C 2017/04). We gratefully thank Galicia Supercomputing Center for providing access to the FinisTerrae-II supercomputer. This material is also based upon work supported by the US National Science Foundation, Office of Advanced Cyberinfrastructure , under Grants No. #1664142 and #1339763 Xunta de Galicia; ED431C 2017/04 US National Science Foundation, Office of Advanced Cyberinfrastructure; 1664142 US National Science Foundation, Office of Advanced Cyberinfrastructure; 1339763 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|