Efficient and Accurate Parallel Genetic Algorithms

170 Pages·2001·6.62 MB·English

by  Erick Cantú-Paz (auth.)

EFFICIENT AND
ACCURATE PARALLEL
GENETIC ALGORITHMS
GENETIC ALGORITHMS AND
EVOLUTIONARY COMPUTATION
EFFICIENT AND
ACCURATE PARALLEL
GENETIC ALGORITHMS
by
Erick Cantti-Paz
Lawrence Livermore National Lab, U.S.A.
SPRINGER SCIENCE+BUSINESS MEDIA, LLC
Library of Congress Cataloging-in-Publication Data
Cantü-Paz, Erick.
Efficient and accurate parallel genetic algorithms / by Erick Cantü-Paz.
p. cm. — (Genetic algorithms and evolutionary computation ; 1)
Includes bibliographical references and index.
ISBN 978-1-4613-6964-6 ISBN 978-1-4615-4369-5 (eBook)
DOI 10.1007/978-1-4615-4369-5
1. Genetic algorithms-Data processing. 2. Parallel processing (Electronic computers)
I. Title. II. Series.
QA402.5 .C36 2000
519.7-dc21
00-048688
Copyright ® 2001 Springer Science+Business Media New York
Originally published by Kluwer Academic Publishers in 2001
All rights reserved. No part of this publication may be reproduced, stored in a retrieval
system or transmitted in any form or by any means, mechanical, photo-copying, recording,
or otherwise, without the prior written permission of the publisher,
Springer Science+Business Media, LLC.
Printed on acid-free paper.
To J aviera and our baby
Series Foreword
Genetic Algorithms and Evolutionary Computation
David E. Goldberg, Consulting Editor
University of Illinois at Urbana-Champaign
Email: [email protected]
Researchers and practitioners alike are increasingly turning to search,
optimization, and machine-learning procedures based on natural selection
and natural genetics to solve problems across the spectrum of human
endeavor. These genetic algorithms and techniques of evolutionary
computation are solving problems and inventing new hardware and software
that rival human designs. Genetic Algorithms and Evolutionary Computation
publishes research monographs, edited collections, and graduate-level texts
in this rapidly growing field. Primary areas of coverage include the theory,
implementation, and application of genetic algorithms (GAs), evolution
strategies (ESs), evolutionary programming (EP), learning classifier systems
(LCSs) and other variants of genetic and evolutionary computation (GEC).
The series also publishes texts in related fields such as artificial life, adaptive
behavior, artificial immune systems, agent-based systems, neural computing,
fuzzy systems, and quantum computing as long as GEC techniques are part
of or inspiration for the system being described.
Erick CantU-Paz's monograph is the first book in this series, and in many
ways it is an exemplar of the kind of book we hope to attract. The book is an
elaboration of Erick's groundbreaking dissertation on the design of efficient
parallel genetic algorithms. It combines facetwise and exact design theory,
careful bounding empirical investigation, and a keen eye toward practice in a
text that has been written in an accessible, logical, and thorough manner. In
the end, it gives us critical scaling laws that accurately govern the particular
parallel architectures that have been analyzed and that approximately bound
many architectures that have not. I believe it is fair to say that prior to this
work, the design of parallel GAs was something of an empirical black art,
guided largely by ad hoc trial and error experimentation. Following this
work, the solution quality, duration, connectivity, deme size, deme count,
and migration rates succumb to the magic of Erick's careful analysis,
experimentation, and exposition. This is not to say that parallel GAs are now
completely understood. The very nature of complex systems design almost
ensures that we can never get to the bottom of the subject. On the other
hand, our understanding of parallel GAs is much enhanced by this work, and
I urgently recommend that all readers interested in parallel genetic and
evolutionary algorithms study this important book carefully and soon.
Contents
Preface xi
Acknowledgments xv
1 INTRODUCTION 1
1 An Introduction to Genetic Algorithms 2
2 A Classification of Parallel GAs 6
3 Summary 11
2 THE GAMBLER'S RUIN AND POPULATION SIZING 13
1 Background 14
2 Deciding Well Between two BBs 18
3 The Gambler's Ruin Model 20
4 Experimental Verification 23
5 Noise and Population Sizing 28
6 Effect of Selection Pressure 30
7 Summary 31
3 MASTER-SLAVE PARALLEL GAs 33
1 Background 34
2 Analysis 36
3 Experiments 42
4 Asynchronous Master-Slaves 43
5 A Distributed Panmictic Population 45
6 Summary 47
4 BOUNDING CASES OF GAs WITH MULTIPLE DEMES 49
1 Background 50
2 Parallel Speedups 54
3 Isolated Demes 55
4 Fully-Connected Demes 58
5 Summary 64
5 MARKOV CHAIN MODELS OF MULTIPLE DEMES 67
1 Fully-Connected Demes with Maximum Migration Rates 68
2 Arbitrary Migration Rates 73
3 Arbitrary Topologies 74
4 Experiments 77
5 Summary 80
6 MIGRATION RATES AND OPTIMAL TOPOLOGIES 81
1 Degree of Connectivity 82
2 Multiple Epochs and Choosing a Topology 87
x PARALLEL GENETIC ALGORITHMS
3 Parallel Demes in the Long Run 94
4 Summary 96
7 MIGRATION AND SELECTION PRESSURE 97
1 Selection Pressure 98
2 Takeover Times 100
3 Selection Intensity 102
4 Experiments 111
5 Superlinear Speedups 114
6 Variance and the Higher Cumulants 117
7 Summary 119
8 FINE-GRAINED AND HIERARCHICAL PARALLEL GAs 121
1 Fine-Grained Parallel GAs 122
2 Hierarchical Parallel GAs 126
3 Optimal Hierarchical Parallel GAs 129
4 An Example of Optimal Design 131
5 Summary 134
9 SUMMARY, EXTENSIONS, AND CONCLUSIONS 135
1 Summary 135
2 Extensions 139
3 Conclusions 141
References 145
Index 159
Preface
As genetic algorithms (GAs) become increasingly popular, they are applied to
difficult problems that may require considerable computations. In such cases,
parallel implementations of GAs become necessary to reach high-quality
solutions in reasonable times. But, even though their mechanics are simple,
parallel GAs are complex non-linear algorithms that are controlled by many
parameters, which are not well understood.
This book is about the design of parallel GAs. It presents theoretical
developments that improve our understanding of the effect of the algorithm's
parameters on its search for quality and efficiency. These developments are
used to formulate guidelines on how to choose the parameter values that
minimize the execution time while consistently reaching solutions of high
qUality.
The book can be read in several ways, depending on the readers' interests and
their previous knowledge about these algorithms. Newcomers to the field will
find the background material in each chapter useful to become acquainted with
previous work, and to understand the problems that must be faced to design
efficient and reliable algorithms. Potential users of parallel GAs that may have
doubts about their practicality or reliability may be more confident after
reading this book and understanding the algorithms better. Those who are
ready to try a parallel GA on their applications may choose to skim through
the background material, and use the results directly without following the
derivations in detail. These readers will find that using the results can help
them to choose the type of parallel GA that best suits their needs, without
having to invest the time to implement and test various options. Once that is
settled, even the most experienced users dread the long and frustrating
experience of configuring their algorithms by trial and error. The guidelines
contained herein will shorten dramatically the time spent tweaking the
algorithm, although some experimentation may still be needed for fine-tuning.
In addition to the results that practitioners will find applicable to the design of
parallel GAs, those interested in the mathematical analysis of GAs may find
the techniques used here useful for their own endeavors. Indeed, some of the
xii PARALLEL GENETIC ALGORITHMS
analysis presented here is relevant not only to parallel GAs, but also to their
serial counterparts.
This book is based on my Ph.D. dissertation (Cantu-Paz, 1999a) The
presentation of the material assumes that the reader has a good understanding
of the basic concepts of GAs and statistics. However, some of the derivations
use concepts that may not be familiar to many readers, and so I explain them
briefly the first time that they appear.
Organization
The first chapter contains a brief introduction to GAs. In addition, it presents
the four common types of parallel GAs, and it explains the design problems
that users face when using these algorithms.
Chapters 2 and 3 are about GAs with a single population. Chapter 2 deals
with finding an appropriate population size for simple serial GAs, and
introduces an abstract model of GAs that will be used in the remainder of the
book to study parallel GAs. Chapter 3 shows how to parallelize GAs using a
simple master-slave algorithm, and how to minimize its execution time. Since
this is the simplest parallel algorithm that we examine, we consider this case
to be the lower bound on the acceptable performance of parallel GAs. Other
parallel GAs should perform better than the simple master-slave.
Most of the book focuses on algorithms with multiple populations. These are
the most popular parallel GAs and potentially the most efficient, because they
require fewer communications than the master-slave. Unfortunately, they are
also the most complex, as one must face difficult and interrelated design
alternatives. Some of the main problems are to determine (1) the size and the
number of demes (subpopulations), (2) the topology that interconnects the
demes, and (3) the migration rate that controls how many individuals migrate
between demes.
It is difficult to study all these problems at once, so we consider first two
cases that represent lower and upper bounds of the migration rate and the
connectivity between demes. Practitioners do not use these extreme cases very
often, but the analysis sheds some light on the intermediate configurations, and
introduces the methodology that will be used in the following chapters.
The study of bounding cases spans two chapters. First, Chapter 4 extends the
population sizing theory of serial GAs to the parallel bounding cases, and