Table Of ContentSpringer Complexity
Springer Complexity is an interdisciplinary program publishing the best research and
academic-level teaching on both fundamental and applied aspects of complex systems –
cutting across all traditional disciplines of the natural and life sciences, engineering,
economics, medicine, neuroscience, social and computer science.
Complex Systems are systems that comprise many interacting parts with the ability to
generate a new quality of macroscopic collective behavior the manifestations of which are
the spontaneous formation of distinctive temporal, spatial or functional structures. Models
of such systems can be successfully mapped onto quite diverse “real-life” situations like
the climate, the coherent emission of light from lasers, chemical reaction-diffusion
systems, biological cellular networks, the dynamics of stock markets and of the internet,
earthquake statistics and prediction, freeway traffic, the human brain, or the formation of
opinions in social systems, to name just some of the popular applications.
Although their scope and methodologies overlap somewhat, one can distinguish the fol-
lowing main concepts and tools: self-organization, nonlinear dynamics, synergetics, turbulence,
dynamical systems, catastrophes, instabilities, stochastic processes, chaos, graphs and net-
works, cellular automata, adaptive systems, genetic algorithms and computational intelligence.
The two major book publication platforms of the Springer Complexity program are the
monograph series “Understanding Complex Systems” focusing on the various appli-
cations of complexity, and the “Springer Series in Synergetics”, which is devoted to the
quantitative theoretical and methodological foundations. In addition to the books in these
two core series, the program also incorporates individual titles ranging from textbooks to
major reference works.
EditorialandProgrammeAdvisoryBoard
PéterÉrdi
CenterforComplexSystemsStudies,KalamazooCollege,USA
andHungarianAcademyofSciences,Budapest,Hungary
KarlFriston
InstituteofCognitiveNeuroscience,UniversityCollegeLondon,London,UK
HermannHaken
CenterofSynergetics,UniversityofStuttgart,Stuttgart,Germany
JanuszKacprzyk
SystemResearch,PolishAcademyofSciences,Warsaw,Poland
Scott Kelso
Center for Complex Systems and Brain Sciences, Florida Atlantic University,Boca Raton, USA
JürgenKurths
NonlinearDynamicsGroup,UniversityofPotsdam,Potsdam,Germany
LindaReichl
CenterforComplexQuantumSystems,UniversityofTexas,Austin,USA
PeterSchuster
TheoreticalChemistryandStructuralBiology,UniversityofVienna,Vienna,Austria
FrankSchweitzer
SystemDesign,ETHZurich,Zurich,Switzerland
DidierSornette
EntrepreneurialRisk,ETHZurich,Zurich,Switzerland
Claudius Gros
Complex and Adaptive
Dynamical Systems
A Primer
With98Figuresand10Tables
ClaudiusGros
UniversitätFrankfurt
InstitutfürTheoretischePhysik
Max-von-Laue-Str.1
60438Frankfurt,Germany
ISBN:978-3-540-71873-4 e-ISBN:978-3-540-71874-1
LibraryofCongressControlNumber:2007937511
(cid:2)c 2008Springer-VerlagBerlinHeidelberg
Thisworkissubjecttocopyright. Allrightsarereserved,whetherthewholeorpartofthematerialis
concerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,broadcasting,
reproductiononmicrofilmorinanyotherway,andstorageindatabanks.Duplicationofthispublication
orpartsthereofispermitted onlyundertheprovisions oftheGermanCopyright LawofSeptember9,
1965,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Violationsare
liabletoprosecutionundertheGermanCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublicationdoesnotimply,
evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotectivelaws
andregulationsandthereforefreeforgeneraluse.
CoverDesign:WMXDesignGmbH,Heidelberg
Printedonacid-freepaper
9 8 7 6 5 4 3 2 1
springer.com
Fu¨rmeineEltern,
meinemverstorbenenVater
undmeinergrossartigenMutter
Contents
1 GraphTheoryandSmall-WorldNetworks........................ 1
1.1 RandomGraphs............................................ 1
1.1.1 TheSmall-WorldEffect............................... 1
1.1.2 BasicGraph-TheoreticalConcepts...................... 3
1.1.3 PropertiesofRandomGraphs.......................... 7
1.2 GeneralizedRandomGraphs ................................. 11
1.2.1 GraphswithArbitraryDegreeDistributions .............. 11
1.2.2 ProbabilityGeneratingFunctionFormalism .............. 15
1.2.3 DistributionofComponentSizes ....................... 17
1.3 RobustnessofRandomNetworks ............................. 20
1.4 Small-WorldModels........................................ 23
1.5 Scale-FreeGraphs.......................................... 26
Exercises ...................................................... 31
FurtherReading ................................................ 31
2 Chaos,BifurcationsandDiffusion................................ 33
2.1 BasicConceptsofDynamicalSystemsTheory .................. 33
2.2 TheLogisticMapandDeterministicChaos..................... 38
2.3 DissipationandAdaption.................................... 43
2.3.1 DissipativeSystemsandStrangeAttractors .............. 43
2.3.2 AdaptiveSystems.................................... 47
2.4 DiffusionandTransport ..................................... 51
2.4.1 RandomWalks,DiffusionandLe´vyFlights .............. 52
2.4.2 TheLangevinEquationandDiffusion ................... 55
2.5 Noise-ControlledDynamics.................................. 57
2.5.1 StochasticEscape.................................... 58
2.5.2 StochasticResonance................................. 60
Exercises ...................................................... 63
FurtherReading ................................................ 64
vii
viii Contents
3 RandomBooleanNetworks ..................................... 67
3.1 Introduction ............................................... 67
3.2 RandomVariablesandNetworks.............................. 69
3.2.1 BooleanVariablesandGraphTopologies ................ 69
3.2.2 CouplingFunctions .................................. 71
3.2.3 Dynamics .......................................... 72
3.3 TheDynamicsofBooleanNetworks .......................... 74
3.3.1 TheFlowofInformationThroughtheNetwork ........... 74
3.3.2 TheMean-FieldPhaseDiagram ........................ 76
3.3.3 TheBifurcationPhaseDiagram ........................ 78
3.3.4 Scale-FreeBooleanNetworks.......................... 82
3.4 CyclesandAttractors ....................................... 84
3.4.1 QuenchedBooleanDynamics.......................... 84
3.4.2 TheK=1KauffmanNetwork ......................... 86
3.4.3 TheK=2KauffmanNetwork ......................... 87
3.4.4 TheK=NKauffmanNetwork ......................... 88
3.5 Applications............................................... 91
3.5.1 LivingattheEdgeofChaos ........................... 91
3.5.2 TheYeastCellCycle ................................. 92
3.5.3 ApplicationtoNeuralNetworks........................ 94
Exercises ...................................................... 96
FurtherReading ................................................ 97
4 CellularAutomataandSelf-OrganizedCriticality ................. 99
4.1 TheLandauTheoryofPhaseTransitions ....................... 99
4.2 CriticalityinDynamicalSystems .............................104
4.2.1 1/fNoise ...........................................107
4.3 CellularAutomata..........................................108
4.3.1 Conway’sGameofLife...............................109
4.3.2 TheForestFireModel ................................110
4.4 TheSandpileModelandSelf-OrganizedCriticality ..............112
4.5 RandomBranchingTheory ..................................115
4.6 ApplicationtoLong-TermEvolution ..........................119
Exercises ......................................................126
FurtherReading ................................................127
5 StatisticalModelingofDarwinianEvolution ......................129
5.1 Introduction ...............................................129
5.2 MutationsandFitnessinaStaticEnvironment ..................131
5.3 DeterministicEvolution .....................................134
5.3.1 EvolutionEquations..................................135
5.3.2 BeanbagGenetics–EvolutionsWithoutEpistasis .........138
5.3.3 EpistaticInteractionsandtheErrorCatastrophe...........140
5.4 FinitePopulationsandStochasticEscape.......................144
5.4.1 StrongSelectivePressureandAdaptiveClimbing .........145
Contents ix
5.4.2 AdaptiveClimbingVersusStochasticEscape.............148
5.5 PrebioticEvolution .........................................149
5.5.1 QuasispeciesTheory .................................149
5.5.2 HypercyclesandAutocatalyticNetworks ................151
5.6 CoevolutionandGameTheory ...............................154
Exercises ......................................................158
FurtherReading ................................................159
6 SynchronizationPhenomena ....................................161
6.1 FrequencyLocking .........................................161
6.2 SynchronizationofCoupledOscillators ........................162
6.3 SynchronizationofRelaxationOscillators ......................168
6.4 SynchronizationandObjectRecognitioninNeuralNetworks......172
6.5 SynchronizationPhenomenainEpidemics......................175
Exercises ......................................................178
FurtherReading ................................................179
7 ElementsofCognitiveSystemsTheory ...........................181
7.1 Introduction ...............................................181
7.2 FoundationsofCognitiveSystemsTheory......................183
7.2.1 BasicRequirementsfortheDynamics...................183
7.2.2 CognitiveInformationProcessingVersusDiffusiveControl.187
7.2.3 BasicLayoutPrinciples...............................189
7.2.4 LearningandMemoryRepresentations ..................191
7.3 Motivation,BenchmarksandTarget-OrientedSelf-Organization ...195
7.3.1 CognitiveTasks .....................................195
7.3.2 InternalBenchmarks .................................196
7.4 CompetitiveDynamicsandWinningCoalitions .................199
7.4.1 GeneralConsiderations ...............................199
7.4.2 AssociativeThoughtProcesses.........................203
7.4.3 AutonomousOnlineLearning..........................207
7.5 EnvironmentalModelBuilding ...............................211
7.5.1 TheElmanSimpleRecurrentNetwork ..................211
7.5.2 UniversalPredictionTasks ............................215
Exercises ......................................................218
FurtherReading ................................................218
8 Solutions ......................................................221
Index .............................................................241
About this Book
FromEvolvingNetworkstoCognitiveSystemsTheory Thistextbookcoversa
wide range of concepts, notions and phenomena of a truly interdisciplinary sub-
ject of rapidly growing importance. Complex system theory deals with dynamical
systems containing a very large number of variables, showing a plethora of emer-
gentfeatures,arisinginabroadrangeofcontexts.Acentralfocusofthesenotesis
thenotionofcomplexityarisingwithinevolvinganddynamicalnetworkstructures,
suchasthegeneexpressionnetworksatthebasisofallliving,adaptiveecological
networksorneuralnetworksforcognitiveinformationprocessing.
Complex systems theory ultimately forms the basis of our long-standing quest
forexploringandunderstandingcognitivesystemsingeneralandourbraininpar-
ticular–themammalianbrainisprobablythemostcomplexofalladaptivenetworks
knowntohumanity.
Readership and Preconditions This primer is intended for graduate students or
scientistsfromnaturalsciences,engineeringorneuroscience.Technically,thereader
shouldhaveabasicknowledgeofordinaryandpartialdifferentialequationsandof
probability distributions. This textbook is suitable both for studies in conjunction
withteachingcoursesaswellasfortheindividualreader.
Course Material and the Modular Approach When used for teaching, this
primerissuitableforacourserunningover40–60lecturehours,dependingonthe
paceandonthenumberofchapterscovered.Essentiallyallmathematicaltransfor-
mationsareperformedonastep-by-stepbasisandingeneralthereadershouldhave
noproblemfollowingtherespectivederivations.
Individual chapters, apart from the first two, having an introductory character,
may be skipped whenever time considerations demand it. I have followed a basic
modular approach and the individual chapters are, as far as possible, independent
ofeachother.Notwithstanding,crossreferencesbetweenthedifferentchaptersare
includedthroughoutthetext,sinceinterrelationsbetweendistincttopicsarehelpful
forathoroughunderstanding.
xi
xii Aboutthisbook
Style This interdisciplinary primer sets a high value on conveying concepts and
notionswithintheirrespectivemathematicalsettings.Believingthataconcisestyle
helps the reader to go through the material I mostly abstained from long text
passages with general background explanations or philosophical considerations.
Widespreadusehasbeenmadeofparagraphheadings,withtheintentiontofacilitate
scientificreadinginthisway.
A Primer to Scientific Common-Sense Knowledge To a certain extent one can
regard this textbook as a primer to a wide range of scientific common-sense
knowledgeregardingcomplexsystems.Basicknowledgeaboutlife’sorganizational
principles, to give an example, such as the notion of “life at the edge of chaos”,
is important in today’s world to an educated scientist. Other areas of scientific
common-sense knowledge discussed in this primer include network theory, which
hasapplicationsrangingfromsocialnetworkstogeneexpressionnetworks,thefun-
damentalsofevolution,cognitivesystemstheoryandthebasicprinciplesofdynam-
icalsystemstheory.
Content Allofthechaptersmakingupthisbookdealwithasubjectworthdevot-
inganentirecourseto.Thisbookaddressesreadersinterestedinmultidisciplinary
aspects;Ihaveconsequentlytriedtopresentsuccinctexpositionsofthefundamental
notionsandconceptsonthebasisofthesubjectstreatedintheindividualchapters.
1. GraphTheoryandSmall-WorldNetworks
Networks,rangingfromneuralnetworks,socialnetworks,ecologicalnetworks
to gene expression networks, are at the basis of many complex systems. Net-
workstendtobeadaptiveandevolving.Networktheoryisthereforeaprereq-
uisiteforathoroughunderstandingofadaptiveand/orcomplexsystems.
2. Chaos,BifurcationsandDiffusion
Thischapterintroducesbasicnotionsofdynamicalsystemstheory,suchasat-
tractors,bifurcations,deterministicchaos,diffusionandstochasticresonances,
manyofwhichareusedthroughoutthesenotes.Complexityemergentfromdy-
namicalsystemscontainingmanyvariables,thecentralthemeofthistextbook,
is ultimately based on the concepts of classical dynamical systems theory,
treated in this chapter, which deals with differential equations involving a
handfulofvariables.
3. RandomBooleanNetworks
A prime model for complex systems with an infinite number of variables are
randomgraphswithbooleanvariables.Itallowsforthecharacterizationoftyp-
ical dynamical behaviors, e.g. “frozen” vs. “chaotic”, which are of relevance
inmanycontexts.Ofespecialimportancearerandombooleannetworksforthe
fundamentals intherealmoflife,leadingtothenotionof“lifeattheedgeof
chaos”.
4. CellularAutomataandSelf-OrganizedCriticality
Regular dynamical systems on lattices, the cellular automata, allow detailed
studiesofthedynamicsofcomplexsystems,akeyissuebeingtheorganizational
principlenecessaryforadynamicalsystemtoshowtheemergentphenomenon
of“self-organizedcriticality”.
