Table Of ContentNetwork Analysis Made Simple
An introduction to network analysis and applied graph
theory using Python and NetworkX
Eric Ma and Mridul Seth
Thisbookisforsaleathttp://leanpub.com/nams
Thisversionwaspublishedon2021-04-08
ThisisaLeanpubbook.LeanpubempowersauthorsandpublisherswiththeLeanPublishing
process.LeanPublishingistheactofpublishinganin-progressebookusinglightweighttoolsand
manyiterationstogetreaderfeedback,pivotuntilyouhavetherightbookandbuildtractiononce
youdo.
©2020-2021EricMaandMridulSeth
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
LearningGoals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
TechnicalTakeaways . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
IntellectualGoals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
IntroductiontoGraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Aformal definitionofnetworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
ExamplesofNetworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
TypesofGraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Edgesdefinetheinterestingpartofagraph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
TheNetworkXAPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
DataModel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
LoadData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Understandingagraph’sbasicstatistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Manipulatingthegraph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
CodingPatterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
FurtherReading. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
FurtherExercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
SolutionAnswers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
GraphVisualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Hairballs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
MatrixPlot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
ArcPlot. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
CircosPlot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
HivePlot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
PrinciplesofRationalGraphViz. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Hubs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
CONTENTS
AMeasureofImportance:“NumberofNeighbors” . . . . . . . . . . . . . . . . . . . . . . . . 26
Generalizing“neighbors”toarbitrarily-sizedgraphs . . . . . . . . . . . . . . . . . . . . . . . 28
Distributionofgraphmetrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Reflections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Breadth-FirstSearch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
VisualizingPaths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
Bottlenecknodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Triangles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
TriadicClosure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Cliques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
ConnectedComponents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
GraphI/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
GraphDataasTables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
GraphModel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
PicklingGraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Othertextformats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Whytest? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Whattotest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Continuousdatatesting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
Furtherreading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
BipartiteGraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
Whatarebipartitegraphs? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
BipartiteGraphProjections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
WeightedProjection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
CONTENTS
DegreeCentrality. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
LinearAlgebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
Pathfinding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
MessagePassing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
BipartiteGraphs&Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Performance:Objectvs.Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
AccelerationonaGPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
StatisticalInference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Statisticsrefresher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Weareconcernedwithmodelsofrandomness . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
HypothesisTesting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
Stochasticgraphcreationmodels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
LoadData . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
InferringGraphGeneratingModel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
QuantitativeModelComparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
GameofThrones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
Findingthemostimportantnodei.echaracterinthesenetworks. . . . . . . . . . . . . . . . 135
Betweenesscentrality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
PageRank. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
Evolutionofimportanceofcharactersoverthebooks. . . . . . . . . . . . . . . . . . . . . . . 142
Sowhat’supwithStannisBaratheon? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
CommunitydetectioninNetworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
AirportNetwork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
Visualisetheairports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
DirectedGraphsandPageRank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
ImportantsHubsintheAirportNetwork . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
Howreachableisthisnetwork? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
Canwefindairlinespecificreachability? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
Preface
Hey,thanksforpickingupthise-Book.Wehadatonoffunmakingthematerial,andwehopeyou
haveatonoffunlearningnewthingsfromittoo.
Applied network analysis, and graph theory concepts, are getting more and more relevant in our
world. Graph problems are abound. Once you pick up how to use graphs in an applied setting,
you’ll find your view of data problems change tremendously. We hope this book can become part
ofyourlearningjourney.
The act of purchasing this book means you’ve chosen to support us, the authors. It means a ton to
us, as this book is the culmination of 5 years of learning and teaching applied network analysis at
conferencesaroundtheworld.ThereasonwewentwithLeanPubtopublishthisbookisthis:Foras
long as we issue updates to the book, you will also receive an updated copy of it. And because the
bookisdigital,it’seasyforustogetupdatesouttoyou.
Just so you know, the full text of the book is available online too, at the accompanying website,
https://ericmjl.github.io/Network-Analysis-Made-Simple. On there, you’ll find a link to Binder so
youcaninteractwiththecode,andthroughtheactofplayingaroundwiththecodeandbreakingit
yourself,learnnewthings.(Breakingcodeandfixingitissomethingyoushould bedoing-it’sone
ofthebestwaystolearn!)
Ifyouhavequestionsaboutthecontent,orfindanerratathatyou’dliketopointout,pleaseheadover
tohttps://github.com/ericmjl/Network-Analysis-Made-Simple/,andpostanissueupthere.We’llbe
suretoaddressitandacknowledgeitappropriately.
Wehopethatthisbookbecomesasteppingstoneinyourlearningjourney.Enjoy!
Eric&Mridul
Learning Goals
Ourlearninggoalsforyouwiththisbookcanbesplitintothetechnicalandtheintellectual.
Technical Takeaways
Firstly, we would like to equip you to be familiar with the NetworkX application programming
interface(API).ThereasonforchoosingNetworkXisbecauseitisextremelybeginner-friendly,and
hasanAPIthatmatchesgraphtheoryconceptsveryclosely.
Secondly, we would like to show you how you can visualize graph data in a fashion that doesn’t
involveshowingmerehairballs.Throughoutthebook,youwillseeexamplesofwhatwecallrational
graphvisualizations.Oneofourauthors,EricMa,hasdevelopedacompanionpackage,nxviz,that
provides a declarative and convenient API (in other words an attempt at a “grammar”) for graph
visualization.
Thirdly,inthisbook,youwillbeintroducedtobasicgraphalgorithms,suchasfindingspecialgraph
structures, or finding paths in a graph. Graph algorithms will show you how to “think on graphs”,
andknowinghowtodosowillbroadenyourabilitytointeractwithgraphdatastructures.
Fourthly, you will also be equipped with the connection between graph theory and other areas of
mathandcomputing,suchasstatisticalinferenceandlinearalgebra.
Intellectual Goals
Beyondthetechnicaltakeaways,wehopetobroadenhowyouthinkaboutdata.
The first idea we hope to give you the ability to think about your data in terms of “relationships”.
As you will learn, relationships are what give rise to the interestingness of graphs. That’s where
relationalinsights cancometofore.
The second idea we hope to give you is the ability to “think on graphs”. This comes with practice.
Once you master it, though, you will find yourself becoming more and more familiar with
algorithmic thinking. which is whereyou look at a problem in terms of thealgorithmthat solves
it.
Introduction to Graphs
Introduction
1 from IPython.display import YouTubeVideo
2
3 YouTubeVideo(id="k4KHoLC7TFE", width="100%")
<iframewidth=”100%”height=”300”src=”https://www.youtube.com/embed/k4KHoLC7TFE”frame-
border=”0”allowfullscreen></iframe>
In our world, networks are an immensely useful data modelling tool to model complex relational
problems. Building on top of a network-oriented data model, they have been put to great use in a
widevarietyofsettings.
A formal definition of networks
Before we explore examples of networks, we want to first give you a more formal definition of
what networks are. The reason is that knowing a formal definition helps us refine our application
ofnetworks.Sobearwithmeforamoment.
Intheslightlymoreacademicliterature,networksaremoreformallyreferredtoasgraphs.
Graphsarecomprisedoftwosets ofobjects:
• Anodeset:the“entities”inagraph.
• Anedgeset:therecordof“relationships”betweentheentitiesinthegraph.
Forexample,ifanodesetniscomprisedofelements:
n=fa;b;c;d;:::g
Then,theedgesetewouldberepresentedastuplesofpairs ofelements:
e=f(a;b);(a;c);(c;d);:::g
Ifyouextractedeverynodefromtheedgesete,itshouldformatleastasubset ofthenodesetn.(It
isatleastasubsetbecausenoteverynodeinnmightparticipateinanedge.)
If you draw out a network, the “nodes” are commonly represented as shapes, such as circles, while
the“edges”arethelinesbetweentheshapes.
IntroductiontoGraphs 4
Examples of Networks
Nowthatwehaveaproperdefinitionofagraph,let’smoveontoexploreexamplesofgraphs.
One example I (Eric Ma) am fond of, based on my background as a biologist, is a protein-protein
interactionnetwork.Here,thegraphcanbedefinedinthefollowingway:
• nodes/entitiesaretheproteins,
• edges/relationshipsaredefinedas“oneproteinisknowntobindwithanother”.
A more colloquial example of networks is an air transportation network. Here, the graph can be
definedinthefollowingway:
• nodes/entitiesareairports
• edges/relationshipsaredefinedas“atleastoneflightcarrierfliesbetweentheairports”.
Andanotherevenmorerelatableexamplewouldbeourever-prevalentsocialnetworks!WithTwitter,
thegraphcanbedefinedinthefollowingway:
• nodes/entitiesareindividualusers
• edges/relationshipsaredefinedas“oneuserhasdecidedtofollowanother”.
Now that you’ve seen the framework for defining a graph, we’d like to invite you to answer the
followingquestion:Whatexamplesofnetworkshaveyou seenbeforeinyourprofession?
Goaheadandlistitout.
Types of Graphs
Asyouprobablycansee,graphsareareallyflexibledatamodelformodellingtheworld,aslongas
the nodes and edges are strictly defined. (If the nodes and edges are sloppily defined, well, we run
intoalotofinterpretabilityproblemslateron.)
If you are a member of both LinkedIn and Twitter, you might intuitively think that there’s aslight
differenceinthestructureofthetwo“socialgraphs”.You’dbeabsolutelycorrectonthatcount!
Twitter is an example of what we would intuitively call a directed graph. Why is this so? The key
here lies in how interactions are modelled. One user can follow another, but the other need not
necessarilyfollowback.Assuch,thereisadirectionality totherelationship.
LinkedInisanexampleofwhatwewouldintuitivelycallanundirectedgraph.Whyisthisso?The
keyhereisthatwhentwousersareLinkedInconnections,weautomatically assignabi-directional
edge between them. As such, for convenience, we can collapse the bi-directional edge into an
undirected edge,thusyieldinganundirectedgraph.
If we wanted to turn LinkedIn into a directed graph, we might want to keep information on who
initiatedtheinvitation.Inthatway,therelationshipisautomaticallybi-directional.
IntroductiontoGraphs 5
Edges define the interesting part of a graph
Whileingraduateschool,I(EricMa)oncesatinaseminarorganizedbyoneoftheprofessorsonmy
thesis committee. The speaker that day was John Quackenbush, a faculty member of the Harvard
SchoolofPublicHealth.Whilethetopicofthedayremainedfuzzyinmymemory,onequotestood
out:
Theheartofagraphliesinitsedges,notinitsnodes.(JohnQuackenbush,HarvardSchool
ofPublicHealth)
Indeed,thisisakeypointtoremember!Withoutedges,thenodesaremerelycollectionsofentities.
In a data table, they would correspond to the rows. That alone can be interesting, but doesn’t yield
relationalinsights betweentheentities.
