Table Of ContentElements of Distributed Algorithms
Springer-Verlag Berlin Heidelberg GmbH
Wolfgang Reisig
Elements
of Distributed Algorithms
Modeling and Analysis
with Petri Nets
With 246 Figures
'Springer
Professor Dr. Wolfgang Reisig
Humboldt-Universitat zu Berlin
Institut fiir Informatik
Unter den Linden 6
D-10099 Berlin, Germany
E-mail: [email protected]
Library of Congress Cataloging-in-Publication Data
Reisig, Wolfgang, 1950-
Elements of distributed algorithms: modeling and analysis with
Petri nets/Wolfgang Reisig.
p. em.
Includes bibliographical references.
ISBN 978-l-642-08303-7 ISBN 978-3-662-03687-7 (eBook)
DOI 10.1007/978-3-662-03687-7
1. Electronic data processing-Distributed processing.
2. Computer algorithms. 3.Petri nets. I. Title.
QA76.9.D5R445 1998
004'.36-dc21 98-22854
CIP
ISBN 978-3-642-08303-7
This work is subject to copyright. All rights are reserved, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcastin~, reproduction on microfilm or in any other way;
and storage in data banks. Duplication of this publication or parts thereof is permitted
only under the provisions of the German Copyright Law of September 9, 1965, in its
current version, and permission for use must always be obtained from Springer-Verlag.
Violations are liable for prosecution under the German Copyright Law.
© Springer-Verlag Berlin Heidelberg 1998
Originally published by Springer-Verlag Berlin Heidelberg New York in 1998
Softcover reprint of the hardcover 1st edition 1998
The use of general descriptive names, trademarks, etc. in this publication does not imply,
even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
Cover Design: Kiinkel + Lopka, Werbeagentur, Heidelberg
Typesetting: Camera ready by the author
SPIN 10570552 45/3142-54 3 2 1 0-Printed on acid-free paper
Preface
The application and organization of computing systems is tending towards
distributed computing. Processor clusters, local and wide area networks, and
the forthcoming information highway are evolving new kinds of problems (the
simplest and most basic ones include, e.g., distributed organization of mutual
exclusion, or distributed detection of distributed termination). A new kind of
algorithm, called distributed algorithms, has emerged during the last decade,
aimed at efficiently solving those problems and, more generally, making dis
tributed computing systems applicable to real-world problems.
A variety of distributed algorithms are presented and proven correct in
this book. A (Petri net based) technique to model and to analyze distributed
algorithms is coincidently presented. This technique focuses on local states,
independent actions, and synchronization of distributed threads of control.
This book's scope is modest, as it sticks to a choice of small and medium
size distributed algorithms. Compositionality, stepwise refinement, interface
specification, abstraction, etc., are not covered. Nevertheless, this book's
claims are ambitious: Just as PASCAL-like programming structures and
Hoare-style proof techniques appear optimal for a wide class of sequential
algorithms, this book's formalism is suggested to be optimal for a wide class
of distributed algorithms.
Particular preliminary knowledge is not assumed in this text, besides ba
sics in formal concepts and a general intuitive understanding of computer
science.
The text provides a basis for university courses and can help the practi
tioner to design distributed algorithms. The hurried reader may just study
the pictures.
Acknowledgments
This book is the yield of a decade of research into formal methods for model
ing and analysis of Distributed Algorithms. I conducted this research together
with many colleagues, staff, and students, both at the Technical University
of Munich and, since 1993, the Humboldt-Universitat zu Berlin. It has been
supported by the Deutsche Forschungsgemeinschaft, in the framework of the
Sonderforschungsbereich 342 as well as in projects on Distributed algorithms
VI Preface
design and compositional verification. The European Union supported this
research via the ESPRIT II projects CALIBAN and DEMON.
The material matured and the presentation profited from successive
courses given at the Technical University of Munich and Humboldt-Universi
tat zu Berlin. My enjoyable summer 1997 visit to the International Computer
Science Institute at Berkeley, California, decisively boosted completion of this
book.
With pleasure I acknowledge a number of colleagues who directly con
tributed to this book: Rolf Walter suggested the concept of rounds and the
corresponding theorems of Sect. 53. He also designed the crosstalk algorithms,
crosstalk based mutex, and distributed rearrangement (Sects. 12, 13.6, and
25), and he contributed to token passing mutex, global mutex, and to the
proofs of the Echo algorithm and phase synchronization (Sects. 13.5, 34.1,
77, and 81). Ekkart Kindler designed the fairness rules (Sects. 48 and 66). He
constructed the spanning tree algorithm (Sect. 32.3) and contributed to token
passing mutex, global mutex, and rearrangement (Sects. 13.5, 34.1, and 25).
Jorg Desel contributed two conventions: how to syntactically represent net
work algorithms (Sect. 31) and how to handle equations, inequalities and log
ical expressions (Sect. 39). He also contributed to the global mutex algorithm
(Sect. 34.1). Hagen Volzer constructed the phase synchronization algorithm
together with its verification (Sects. 36 and 81) and contributed to the proof
of the Echo algorithm (Sect. 77). Jorn Freiheit corrected an earlier version of
local mutual exclusion on networks (Sect. 34.3) and contributed decisive argu
ments to its verification (Sect. 79). Tobias Vesper provided the central proof
arguments for the distributed self-stabilization algorithm (Sect. 82). Wilfried
Brauer constructed the asynchronous stack (Sect. 11). Eike Best referred me
to [Dij78], the source of this book's consensus algorithms (Sect. 35).
In addition, the above-mentioned contributed numerous comments and
corrections, as likewise did Juliane Dehnert, Adriane Foremniak, Dominik
Gomm, Keijo Heljanko, Bodo Hohberg, Irina Lomazowa, Sibylle Peuker, Tho
mas Ritz, Karsten Schmidt, Einar Smith, and Michael Weber. Many thanks
to ali of them, and to the many further casual members of the "Kaffeerunde",
for inspiring discussions and criticism.
The burden of never-ending typesetting, changes of text, and corrections
was shouldered mainly by Thomas Ritz. I am very grateful for his patience
and constructive cooperation. The pictures were drawn by Jorn Freiheit and
Abdourahaman in quite a painstaking manner. Juliane Dehnert compiled the
references. I am indebted to all of them.
I also owe much to Birgit Heene; her effective management of daily sec
retarial duties saved me a lot of time that I was able to devote to this book.
It is a pleasure to acknowledge the constructive, efficient, and friendly
support of Hans Wossner, Ingeborg Mayer, and J. Andrew Ross of Springer
Verlag. Their professional management and careful editing of the manuscript
Preface VII
have again been perfect, just as I frequently experienced during the last 15
years.
A Note to the Expert in Petri Nets
This is not a book on Petri nets. It is a book however that heavily em
ploys Petri nets, i.e., a slightly revised form of elementary net systems (en
systems), and "algebraic" high level nets. Revisions are motivated by the
realm of distributed algorithms: The role of loops is reconsidered, and as
pects of progress and fairness are added. Altogether, those revisions provide
the expressive power necessary to model elementary distributed algorithms
adequately, retaining intuitive clarity and formal simplicity.
A version of high-level nets is used to model data dependencies and algo
rithms for (classes of) networks. Technically, such nets are based on concepts
from algebraic specification, but they may be understood independently of
that field.
Multiple instances of a token in the same place will never occur, because
the vast majority of distributed algorithms by their nature do without mul
tiple instances of tokens. Local states of nets are hence propositions and
predicates, thus providing the elements for logic-based analysis techniques.
Two kinds of properties will be studied, both of them instances of [AS85]'s
safety and liveness properties. In this text they will be called state and
progress properties in order to avoid confusion with conventional safety and
liveness of Petri nets. Again, those notions are motivated by practical needs.
Place invariants and initialized traps turn out to be useful in analyzing state
properties. Progress properties have rarely been considered in the literature
on Petri nets. New concepts support their analysis, borrowed from temporal
logic.
Berlin, April 1998 Wolfgang Reisig
Contents
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Part A. Elementary System Models 3
I. Elementary Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1 A First Look at Distributed Algorithms . . . . . . . . . . . . . . . . . 5
2 Basic Definitions: Nets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3 Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4 Interleaved Runs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
5 Concurrent Runs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
6 Progress . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
7 Fairness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
8 Elementary System Nets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
II. Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
9 Sequential and Parallel Buffers . . . . . . . . . . . . . . . . . . . . . . . . . 35
10 The Dining Philosophers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
11 An Asynchronous Stack . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
12 Crosstalk Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
13 Mutual Exclusion..................................... 50
14 Distributed Testing of Message Lines. . . . . . . . . . . . . . . . . . . . 60
Part B. Advanced System Models 63
III. Advanced Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
15 Introductory Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
16 The Concept of System Nets........................... 73
17 Interleaved and Concurrent Runs . . . . . . . . . . . . . . . . . . . . . . . 75
18 Structures and Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
19 A Term Representation of System Nets.................. 80
20 Set-Valued Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
21 Transition Guards and System Schemata . . . . . . . . . . . . . . . . 88
X Contents
IV. Case Studies 91
0 0 0 0 0 0 0 0 0 0 0 0 0 0 o 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
22 High-Level Extensions of Elementary Net Models 91
0 0 0 0 0 0 0 0 0
23 Distributed Constraint Programming 96
o 0 o 0 o 0 0 0 0 0 0 0 0 0 0 0 0 0 0
24 Exclusive Writing and Concurrent Reading 100
0 0 0 0 0 0 0 0 0 0 0 0 0 0
25 Distributed Rearrangement 102
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
26 Self Stabilizing Mutual Exclusion 105
o o o 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
V. Case Studies Continued: Acknowledged Messages 107
0 0 0 0 0 0 0
27 The Alternating Bit Protocol 107
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
28 The Balanced Sliding Window Protocol 112
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
29 Acknowledged Messages to Neighbors in Networks 116
0 0 0 0 0 0 0 0
30 Distributed Master /Slave Agreement 119
0 o 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
VI. Case Studies Continued: Network Algorithms 123
0 0 0 0 0 0 0 0 0 0
31 Principles of Network Algorithms 123
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
32 Leader Election and Spanning Trees 125
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
33 The Echo Algorithm 127
o o 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
34 Mutual Exclusion in Networks 130
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
35 Consensus in Networks 134
0 0 0 0 0 0 0 o o 0 0 o o o o o o o o 0 0 0 0 0 0 0 0 0 0 0 0 0
36 Phase Synchronization on Undirected Trees 137
0 0 0 0 0 0 0 0 0 0 0 0 0 0
37 Distributed Self Stabilization 140
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Part C. Analysis of Elementary System Models 143
VII. State Properties of Elementary System Nets 145
0 0 0 0 0 0 0 0 0 0 0 0
38 Propositional State Properties 145
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
39 Net Equations and Net Inequalities 147
o o 0 0 o 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
40 Place Invariants of es-nets 150
o 0 0 0 o o 0 o 0 o o 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
41 Some Small Case Studies 153
o 0 o 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
42 Traps 156
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
43 Case Study: Mutex 159
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
VIII. Interleaved Progress of Elementary System Nets 165
0 0 0 0 0 0 0
44 Progress on Interleaved Runs 165
0 0 o 0 o o 0 o o 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
45 The Interleaved Pick-up Rule 167
0 0 0 0 0 o 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
46 Proof Graphs for Interleaved Progress 170
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
4 7 Standard Proof Graphs 172
o 0 0 o 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
48 How to Pick Up Fairness 176
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
49 Case Study: Evolution of Mutual Exclusion Algorithms 178
0 0 0 0
IX. Concurrent Progress of Elementary System Nets 187
o 0 o o 0 0 0
50 Progress on Concurrent Runs 187
o o o o o o o o o o o o o o o o o o o 0 0 0 0 0 0 0
51 The Concurrent Pick-up Rule 188
o o o o o o o o o o o o o o o o o o 0 0 0 0 0 0 0 0
52 Proof Graphs for Concurrent Progress 190
0 0 o o o o o o o o o o o o o 0 0 o 0
Contents XI
53 Ground Formulas and Rounds .......................... 191
54 Rounds of Sequential and Parallel Buffer Algorithms ...... 195
55 Rounds and Ground Formulas of Various Algorithms ...... 197
56 Ground Formulas of Mutex Algorithms .................. 200
Part D. Analysis of Advanced System Models 205
X. State Properties of System Nets ........................ 207
57 First-Order State Properties ........................... 208
58 Multisets and Linear Functions ......................... 209
59 Place Weights, System Equations, and System Inequalities. 210
60 Place Invariants of System Nets ........................ 214
61 Traps of System Nets ................................. 219
62 State Properties of Variants of the Philosopher System .... 222
XI. Interleaved Progress of System Nets .................... 227
63 Progress on Interleaved Runs . . . . . . . . . . . . . . . . . . . . . . . . . . 227
64 Interleaved Pick-up and Proof Graphs for System Nets .... 228
65 Case Study: Producer/ Consumer Systems . . . . . . . . . . . . . . . 230
66 How to Pick up Fairness ............................... 231
XII. Concurrent Progress of System Nets .................... 233
67 Progress of Concurrent Runs ........................... 233
68 The Concurrent Pick-up Rule .......................... 234
69 Pick-up Patterns and Proof Graphs ..................... 235
70 Ground Formulas and Rounds .......................... 238
XIII. Formal Analysis of Case Studies ........................ 241
71 The Asynchronous Stack .............................. 241
72 Exclusive Writing and Concurrent Reading .............. 244
73 Distributed Rearrangement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
74 Self-Stabilizing Mutual Exclusion ....................... 253
75 Master /Slave Agreement .............................. 255
76 Leader Election . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
77 The Echo Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
78 Global Mutual Exclusion on Undirected Trees ............ 266
79 Local Mutual Exclusion ............................... 269
80 Consensus in Networks ................................ 279
81 Phase Synchronization on Undirected Trees .............. 285
82 Distributed Self-Stabilization ........................... 291
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
