Table Of ContentLecture Notes in Mathematics 2141
CIME Foundation Subseries
P.R. Kumar · Martin J. Wainwright
Riccardo Zecchina
Mathematical
Foundations of
Complex Networked
Information Systems
Politecnico di Torino, Verrès, Italy 2009
Fabio Fagnani · Sophie M. Fosson
Chiara Ravazzi Editors
Lecture Notes in Mathematics 2141
Editors-in-Chief:
J.-M.Morel,Cachan
B.Teissier,Paris
AdvisoryBoard:
CamilloDeLellis,Zurich
MariodiBernardo,Bristol
AlessioFigalli,Austin
DavarKhoshnevisan,SaltLakeCity
IoannisKontoyiannis,Athens
GaborLugosi,Barcelona
MarkPodolskij,Aarhus
SylviaSerfaty,ParisandNY
CatharinaStroppel,Bonn
AnnaWienhard,Heidelberg
Moreinformationaboutthisseriesat
http://www.springer.com/series/304
FondazioneC.I.M.E.,Firenze
C.I.M.E. stands for Centro Internazionale Matematico Estivo, that is, International
MathematicalSummerCentre.Conceivedintheearlyfifties,itwasbornin1954inFlorence,
Italy,andwelcomedbytheworldmathematical community:itcontinues successfully,year
foryear,tothisday.
Many mathematicians from all over the world have been involved in a way or another in
C.I.M.E.’sactivitiesovertheyears.ThemainpurposeandmodeoffunctioningoftheCentre
maybesummarisedasfollows:everyyear,duringthesummer,sessionsondifferentthemes
from pure and applied mathematics are offered byapplication to mathematicians from all
countries. ASessionis generally basedonthree orfourmaincourses given byspecialists
ofinternationalrenown,plusacertainnumberofseminars,andisheldinanattractiverural
locationinItaly.
TheaimofaC.I.M.E.sessionistobringtotheattentionofyoungerresearcherstheorigins,
development, and perspectives of some very active branch of mathematical research. The
topicsofthecoursesaregenerallyofinternationalresonance.Thefullimmersionatmosphere
ofthecoursesandthedailyexchangeamongparticipantsarethusaninitiationtointernational
collaborationinmathematicalresearch.
C.I.M.E.Director C.I.M.E.Secretary
ElviraMASCOLO PaoloSALANI
DipartimentodiMatematica“U.Dini” DipartimentodiMatematica“U.Dini”
UniversitàdiFirenze UniversitàdiFirenze
vialeG.B.Morgagni67/A vialeG.B.Morgagni67/A
50134Florence 50134Florence
Italy Italy
e-mail:[email protected]fi.it e-mail:[email protected]fi.it
FormoreinformationseeCIME’shomepage:http://www.cime.unifi.it
CIMEactivityiscarriedoutwiththecollaborationandfinancialsupportof:
-INdAM(IstitutoNazionalediAltaMatematica)
-MIUR(Ministerodell’Universita’edellaRicerca)
P.R. Kumar • Martin J. Wainwright •
Riccardo Zecchina
Mathematical Foundations
of Complex Networked
Information Systems
Politecnico di Torino, VerreJs, Italy 2009
Fabio Fagnani, Sophie M. Fosson, Chiara Ravazzi
Editors
123
Authors
P.R.Kumar MartinJ.Wainwright
CSLandDepartmentofECE DepartmentofStatistics
UniversityofIllinois UCBerkeley
Urbana Berkeley
Illinois,USA California,USA
RiccardoZecchina
DISAT
PolitecnicodiTorino
Torino,Italy
Editors
FabioFagnani SophieM.Fosson
DISMA DET
PolitecnicodiTorino PolitecnicodiTorino
Torino,Italy Torino,Italy
ChiaraRavazzi
DET
PolitecnicodiTorino
Torino,Italy
ISSN0075-8434 ISSN1617-9692 (electronic)
LectureNotesinMathematics
ISBN978-3-319-16966-8 ISBN978-3-319-16967-5 (eBook)
DOI10.1007/978-3-319-16967-5
LibraryofCongressControlNumber:2015940535
MathematicsSubjectClassification(2010):05C80GraphTheory-Randomgraph,05C85GraphTheory
-GraphAlgorithms,05C90GraphTheory-Applications
SpringerChamHeidelbergNewYorkDordrechtLondon
©SpringerInternationalPublishingSwitzerland2015
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.
Printedonacid-freepaper
Springer International Publishing AG Switzerland is part of Springer Science+Business Media
(www.springer.com)
Preface
In the past 20 years, we have been witnessing a revolution in information and
communication technologies, consisting in an astonishing progress in different
research fields. Innovations have been developed not only in the classical fields
of wireless communication (design of high performance codes very close to the
Shannon limit), computer engineering (parallel computing, computer vision, data
mining),bioinformatics(genomeinterpretation),andVLSItechnologies,butalsoin
newareassuchaswirelesssensornetworksandsmartdust,airtrafficcontrol,large
scale surveillance, unmanned mobile multi-vehicle systems, biological networks
interpretation,andsoon.
All of these problemsdealwith complexsystems and largescale optimization,
and require a prohibitiveamountof computationalefforteven in low dimensions.
The complexity calls for robust and adaptable solutions. In spite of the inherent
difficulties,fundamentalcontributionshavealreadyappeared,basedontheprinciple
that most of complex systems are the result of the interactions of numerous, but
rather simple entities. The complexity is then a consequence of the interactions
architecture, that can be described by a network, and solutions can be developed
from mathematical models that describe how the local rules originate the global
behavior.
Themathematicaltoolsusefultoaddressthesenewscientificissuescomemainly
from combinatorics, probability theory, and statistical mechanics. The concept of
graphplaysaprominentrole:notonlyitisthenaturalmodelforanycommunication
network, but also it describes interactions among variables, as exploited, e.g., in
graphicalmodelsinmoderncodingtheory.
Inparticular,randomgraphsareasuitablemodeltodescribecomplexnetworks
(e.g., internet, wireless communication networks, sensor networks). Randomness
entersatdifferentlevels,tomimicthedevelopmentofcomplexnetworks,tomodel
faultsornoiseinthecommunicationlinks,oralsotofindtypicalcodesoralgorithms
with good properties. For a deep analysis of such random models, probabilistic
techniquesareessential,amongwhichwementionconcentrationinequalities,large
deviationtheory,and percolation.In case of aggregationof simple entities, where
v
vi Preface
complexityrisesfromthenetworkofinteractions,statisticalmechanicsisanatural
approach.
The summer school “Mathematical Foundationsof Complex Networked Infor-
mation Systems” was organized by CIME, with the contribution of Newcom++,
in Verrès (Aosta Valley, Italy) in 2009. The aim of the school was to give
an introduction to some of the fundamental scientific issues emerging in these
disciplines.Itconsistedoffourcourses,eachof6h.
The course of Prof. Béla Bollobás focused on random graphs: it provided all
thenecessaryprobabilistictoolsandproposedanumberofdifferentwaystomodel
randomness.
ThecourseofProf.P.R.Kumarfocusedoncommunicationnetworks,inpartic-
ular on the issues of the transmission of informationalonga network where there
aresimultaneouslymanypotentialreceiversandmanypotentialtransmitters.Prof.
Kumarpresentedthebasicinformationtheoreticaspectsofwirelesscommunication
networks.
The other two courses addressed the analysis of distributed algorithms over
networksand,moregenerally,overgraphicalmodels.Particularemphasiswasgiven
tothefamousmessagepassingalgorithmswhichhaveapplicationsinabroadvariety
ofcontextsincludingartificialintelligence,distributedinferentialstatistics, coding
theory, and combinatorial optimization. The course of Prof. Riccardo Zecchina
described the statistical physics interpretation of these algorithms with special
attention to applications in classical combinatorial optimization problems. The
course by Prof. Martin J. Wainwright instead focused on the probabilistic aspects
of message passingalgorithmsand their relationwith the relaxationtechniquesin
optimizationtheory.
About 70 people from all over the world attended the school. Among them,
there were senior graduate students and post-doc researchers in pure and applied
mathematics,informationengineering,andphysics.
Theoverallaimofthisbookistorecordtheresultspresentedduringthesummer
school.
Torino,Italy FabioFagnani
Torino,Italy SophieM.Fosson
Torino,Italy ChiaraRavazzi
Contents
SomeIntroductoryNotesonRandomGraphs................................ 1
FabioFagnani,SophieM.Fosson,andChiaraRavazzi
StatisticalPhysicsandNetworkOptimizationProblems.................... 27
Carlo Baldassi, AlfredoBraunstein, AbolfazlRamezanpour,
andRiccardoZecchina
GraphicalModelsandMessage-PassingAlgorithms:Some
IntroductoryLectures ........................................................... 51
MartinJ.Wainwright
BridgingtheGapBetweenInformationTheoryandWireless
Networking........................................................................ 109
P.R.Kumar
vii
Some Introductory Notes on Random Graphs
FabioFagnani,SophieM.Fosson,andChiaraRavazzi
1 Introduction
In the last decades, the words complexity and networks have constantly increased
their presence in the scientific literature and have become popular even in our
everyday life. Born in different contexts, nowadays they are often associated to
characterize the so-called complex networked systems, that are of outstanding
importance in our society and range from natural phenomena to technological
models. A common example is the internet network, today accessed by one third
of the world population and in continuous expansion. While most of the internet
usershavenodeepinsightofitsarchitecture,everyonehasanideaofitsnetworked
natureandhasmadeexperienceofitscomplexity,whichaffectsitsuseintermsof
bottlenecksandcommunicationdelays.
While giving a definition of complex networked system is tricky, it is easy to
provide examples, as they are involved in a number of different fields such as
physics, computer science, economy, finance, biology, epidemiology, sociology.
The common point of such systems is the presence of a large number of objects
(individuals, sensors, computers, cells, molecules) that interact in a way that
presentsstrongirregularities,e.g.,followingpatternsthat varyin time, sometimes
randomly,withunclearmodificationsofrolesandclusteringbehaviors.Itisworth
F.Fagnani((cid:2))
DepartmentofMathematicalSciences(DISMA),PolitecnicodiTorino,CorsoDucadegli
Abruzzi24,10129Torino,Italy
e-mail:[email protected]
S.M.Fosson•C.Ravazzi
DepartmentofElectronicsandTelecommunications(DET),PolitecnicodiTorino,CorsoDuca
degliAbruzzi24,10129Torino,Italy
e-mail:[email protected];[email protected]
©SpringerInternationalPublishingSwitzerland2015 1
F.Fagnanietal.(eds.),MathematicalFoundationsofComplexNetworked
InformationSystems,LectureNotesinMathematics2141,
DOI10.1007/978-3-319-16967-5_1
2 F.Fagnanietal.
noting that the dimension of the network (that is, the number of interacting
components)doesnotautomaticallydeterminethecomplexity:iftheypresentgood
scalabilityproperties,largenetworksmaybesimple.Complexityarisesbecauseof
thetopologyoftheinterconnectionswhichenhancerapiddiffusionofinformation
and because of the non-linear and probabilistic nature of the interaction laws
guiding the dynamics over the network. In these cases, the global behavior of
the network shows complexity features which by no means can be seen as the
additionofthemanyindividualbehaviors.Often,suchsystemscanexhibittransition
phase phenomenawith drastic change in their functional modes when even small
and localized perturbations take place. The first seminal work on complexity has
been done by physicists: from gas to chaos theory, they have studied the large
scale consequences of the interactions among a number of small components,
whichassumesanetworkedstructure;statisticalmechanicsisclearlyanappropriate
language to describe such large scale networks as well the connection between
the micro and macro observation scales. Analogously, biologists, economists,
anthropologists have analyzed existing systems of their interest considering the
complexity generated by connectivity among organisms or individuals. Complex
infrastructuralnetworkshavealsoattractedtheattentionofengineersandcomputer
scientistssince awhile.Thedigitalrevolutionhascreatedmanynewapplications:
besidestheworldwideweb,therearethewirelesscellularnetworks,andthemore
futurist sensor networks. Other remarkable examples are the electrical networks
includingtherecentsmartgridsandtransportationnetworks.Inmostoftheseuses,
artificial networkshave grown in time so much that a fully faithfulrepresentation
of them is not available. Often these infrastructure networks have been optimally
designedforthebusiness-as-usualbehaviorandtheypresenthighfragilitytosmall
perturbationswhich may lead them to unpredictable and unstable dynamics(e.g.,
cascadingfailures,blackouts,crashes).
Whatever perspective is considered, it has become fundamental to understand
and efficiently describe complex networked systems, as well to establish new
paradigmsfortheirdesignandcontrol.
This series of lectures is aimed at introducing the reader to the mathematical
theoryofcomplexnetworkedsystems.Afewfundamentalissuesareproposed,that
showhowresearchinthisfieldnecessarilyhastocopewithadoubledifficulty:the
problemofconstructingcoherentmathematicalmodelssufficientlyrich to beable
to describe the existent complex networks and, on the other side, simple enough
to be amenable for a general mathematical theory. While elegant and complete
mathematical results already exist for certain specific problems, it is true that the
mathematical theory of complex networks is far from being complete and many
outstandingproblemsstillwaitforananswer.
The required background is limited to undergraduate-level calculus, linear
algebra, probabilitytheory,informationtheory and combinatorics.As the concept
of graph plays a fundamental role in every aspect of complex networks, in this
first section, we will provide a brief introduction to graph theory with particular
focus on models of random graphs for their role of furnishing the simplest and
moreeffectivewaytoexhibitlargescalegraphshavingpropertiessimilartothereal
