Fault-Tolerance Techniques for High-Performance Computing

325 Pages·2015·8.528 MB·English

by  Thomas Herault, Yves Robert (eds.)

Computer Communications and Networks
Thomas Herault
Yves Robert Editors
Fault-Tolerance
Techniques
for High-
Performance
Computing
Computer Communications and Networks
Series editor
A.J. Sammes
Centre for Forensic Computing
Cranfield University, Shrivenham Campus
Swindon, UK
The Computer Communications and Networks series is a range of textbooks,
monographs and handbooks. It sets out to provide students, researchers, and
nonspecialists alike with a sure grounding in current knowledge, together with
comprehensibleaccesstothelatestdevelopmentsincomputercommunicationsand
networking.
Emphasis is placed on clear and explanatory styles that support a tutorial
approach, so that even the most complex of topics is presented in a lucid and
intelligible manner.
More information about this series at http://www.springer.com/series/4198
Thomas Herault Yves Robert
(cid:129)
Editors
Fault-Tolerance Techniques
for High-Performance
Computing
123
Editors
ThomasHerault Yves Robert
University of Tennessee EcoleNormale Supérieure deLyon
Knoxville, TN Lyon
USA France
and
University of Tennessee
Knoxville, TN
USA
ISSN 1617-7975 ISSN 2197-8433 (electronic)
Computer Communications andNetworks
ISBN978-3-319-20942-5 ISBN978-3-319-20943-2 (eBook)
DOI 10.1007/978-3-319-20943-2
LibraryofCongressControlNumber:2015942754
SpringerChamHeidelbergNewYorkDordrechtLondon
©SpringerInternationalPublishingSwitzerland2015forChapters1,3,4and5
©SpringerInternationalPublishingSwitzerland(outsidetheUSA)2015forChapter2
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor
foranyerrorsoromissionsthatmayhavebeenmade.
Printedonacid-freepaper
SpringerInternationalPublishingAGSwitzerlandispartofSpringerScience+BusinessMedia
(www.springer.com)
Preface
Objective
ThemainobjectiveofthismonographistoprovideanoverviewofFault-Tolerance
Techniques for High-Performance Computing (HPC). Resilience has already
becomeaprominentissueoncurrentlarge-scaleplatforms. Theadventofexascale
computerswithmillionsofcoresandbillion-parallelismisonlygoingtoworsenthe
scenario.ThecapacitytodealwitherrorsandfaultswillbeacriticalfactorforHPC
applications to be deployed efficiently.
While there aremany research papers available on this hot and important topic,
therewasnocomprehensiveandeasy-to-accessreferenceavailableintheliterature.
The purpose of this monograph is to fill the gap, and to provide a detailed pre-
sentationandanalysisofthevariousfaulttolerancemethodsforHPCapplications.
The first part of the book is made of a single survey chapter that introduces
checkpointprotocolsandschedulingalgorithms,prediction,replication,silenterror
detection, and correction, together with some application-specific techniques such
as Algorithm-Based Fault Tolerance (ABFT). A key feature of this survey chapter
is the importance given to analytical performance models. As future extreme-scale
platforms are not yet available (by definition!), a refined (and publicly available)
performance model is the key to assess any resilience technique without bias nor
a-priori. Various scenarios can be instantiated through selecting one’s preferred
modelparameters,andfurtherexploredthroughsimulations.Theemphasisgivento
performancemodelsexplainstheunusualamountofmathematicalequationsinthe
chapter, but let the reader be comforted: (i) every method is first described infor-
mally; (ii) the mathematical derivations are detailed and complemented with
examples;and(iii)itisalwayspossibletoskipsomeproofandbebacktoitduring
a second reading.
The second part of the book is composed of four chapters, each dedicated to
further investigatingonetopic.Chapter 2surveysthevarioussourcesforerrorand
faults in real large-scale systems, details their characteristics, and focuses on
detectionandprediction.Chapter3presentsthespectrumoftechniquesthatcanbe
v
vi Preface
applied to design a fault-tolerant MPI, i.e., to enable MPI application recovery
using either fully automatic or completely user-driven techniques. Chapter 4
investigates replication (coupled with checkpointing) and compares two approa-
ches. In the first approach, entire application instances are replicated, while in the
second one, each process in a single application instance is (transparently) repli-
cated. Finally, Chap. 5 addresses the challenge of energy consumption related to
fault tolerance in extreme-scale systems, and proposes a methodology to estimate
the energy consumption offault-tolerant protocols used in HPC.
ThebestwaytoreadthebookistostartwiththeoverviewchapterinPartI,and
then to move on tothemore specializedchaptersof Part II.However, experienced
readersmaywanttoreadasinglespecificchapterinPartII.Toeasethisapproach,
we have made each chapter independent of the others, at the price of some
redundant information throughout the book. Cross-references between related
sections of different chapters and index terms have been provided to help navigate
across chapters whenever needed.
Thanks
This monograph is the follow-up of a tutorial that we gave at ICS’13. We were
approachedbySpringerVerlagandinvitedtowritethismonograph,ataskthatwe
eventually succeeded to complete after … some delay.
Wewouldliketothankallchapterauthorsfortheircontribution.Allofthemare
colleagues and Ph.D. students with whom we worked on various topics during the
pastfewyears,andallofussharemanyideasonresilienceforHPC.Hopefully,the
reader will sense a common perspective while reading the monograph!
The tutorial that George Bosilca, Aurélien Bouteiller, and the two of us gave at
SC’14 came one year after the one given at ISC’13. At that point the monograph
wasfarfromready,andintensediscussionswhenpreparingtheSC’14tutorialhave
greatly influenced the overview chapter. We thank them for this.
Finally, we would like to thank Jack Dongarra and ICL for providing a unique
place to collaborate and work on HPC-related research, from linear algebra to
resilience and more.
Knoxville Thomas Herault
Lyon Yves Robert
April 2015
Contents
Part I General Overview
1 Fault Tolerance Techniques for High-Performance Computing. . . . 3
Jack Dongarra, Thomas Herault and Yves Robert
Part II Technical Contributions
2 Errors and Faults. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Ana Gainaru and Franck Cappello
3 Fault-Tolerant MPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
Aurélien Bouteiller
4 Using Replication for Resilience on Exascale Systems. . . . . . . . . . . 229
Henri Casanova, Frédéric Vivien and Dounia Zaidouni
5 Energy-Aware Checkpointing Strategies . . . . . . . . . . . . . . . . . . . . 279
Guillaume Aupy, Anne Benoit, Mohammed El Mehdi Diouri,
Olivier Glück and Laurent Lefèvre
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
vii
Contributors
Guillaume Aupy Ecole Normale Supérieure de Lyon, Lyon, France
Anne Benoit Ecole Normale Supérieure de Lyon, Lyon, France
Aurélien Bouteiller University of Tennessee, Knoxville, TN, USA
Franck Cappello Argonne National Laboratory, Lemont, USA
Henri Casanova University of Hawai‘i, Manoa, USA
Mohammed El Mehdi Diouri IGA Casablanca, Casablanca, Morocco
Jack Dongarra University of Tennessee, Knoxville, TN, USA; Oak Ridge
NationalLaboratory,OakRidge,USA;UniversityofManchester,Manchester,UK
Ana Gainaru NCSA, University of Illinois at Urbana-Champaign, Champaign,
USA
Olivier Glück Université Claude Bernard de Lyon, Villeurbanne, France
Thomas Herault University of Tennessee, Knoxville, TN, USA
Laurent Lefèvre INRIA & Ecole Normale Supérieure de Lyon, Lyon, France
Yves Robert University of Tennessee, Knoxville, TN, USA; Ecole Normale
Supérieure de Lyon, Lyon, France
Frédéric Vivien INRIA & Ecole Normale Supérieure de Lyon, Lyon, France
Dounia Zaidouni INRIA & Ecole Normale Supérieure de Lyon, Lyon, France
ix
Part I
General Overview