Table Of ContentProbability and Computing
Randomization and probabilistic techniques play an important role in modern computer
science,withapplicationsrangingfromcombinatorialoptimizationandmachinelearning
tocommunicationnetworksandsecureprotocols.
This textbook provides an indispensable teaching tool to accompany a one- or two-
semester course for advanced undergraduate or beginning graduate students in computer
scienceandappliedmathematics.Itoffersacomprehensiveintroductiontotheroleofran-
domizationandprobabilistictechniquesinmoderncomputerscience,inparticulartotech-
niques and paradigms used in the development and probabilistic analysis of algorithms
andfordataanalyses.Itassumesonlyanelementarybackgroundindiscretemathematics
andgivesarigorousyetaccessibletreatmentofthematerial,withnumerousexamplesand
applications.
The first half of the book covers core material, including random sampling, expecta-
tions,Markov’sinequality,Chebyshev’sinequality,Chernoffbounds,balls-and-binsmod-
els, the probabilistic method, and Markov chains. In the second half, the authors delve
intomoreadvancedtopicssuchascontinuousprobability,applicationsoflimitedindepen-
dence,entropy,MarkovchainMonteCarlomethods,coupling,martingales,andbalanced
allocations.
This greatly expanded new edition includes several newly added chapters and sec-
tions, covering topics including normal distributions, sample complexity, VC dimension,
Rademachercomplexity,powerlawsandrelateddistributions,cuckoohashing,andappli-
cations of the Lovász Local Lemma. New material relevant to machine learning and big
dataanalysisenablesstudentstolearnup-to-datetechniquesandapplications.Amongthe
manynewexercisesandexamplesareprogramming-relatedexercisesthatprovidestudents
withpracticalexperienceandtrainingrelatedtothetheoreticalconceptscoveredinthetext.
Michael Mitzenmacher is a Professor of Computer Science in the School of Engineering
and Applied Sciences at Harvard University, where he was also the Area Dean for Com-
puter Science from 2010 to 2013. Michael has authored or co-authored over 200 confer-
enceandjournalpublicationsonavarietyoftopics,includingalgorithmsfortheInternet,
efficient hash-based data structures, erasure and error-correcting codes, power laws, and
compression.Hisworkonlow-densityparity-checkcodessharedthe2002IEEEInforma-
tionTheorySocietyBestPaperAwardandwonthe2009ACMSIGCOMMTestofTime
Award. He is an ACM Fellow, and was elected as the Chair of the ACM Special Interest
GrouponAlgorithmsandComputationTheoryin2015.
EliUpfalisaProfessorofComputerScienceatBrownUniversity,wherehewasalsothe
departmentchairfrom2002to2007.PriortojoiningBrownin1998,hewasaresearcherand
projectmanagerattheIBMAlmadenResearchCenter,andaProfessorofAppliedMath-
ematics and Computer Science at the Weizmann Institute of Science. His main research
interests are randomized algorithms, probabilistic analysis of algorithms, and computa-
tionalstatistics,withapplicationsrangingfromcombinatorialandstochasticoptimization,
computationalbiology,andcomputationalfinance.HeisaFellowofboththeIEEEandthe
ACM.
Probability and Computing
Randomization and Probabilistic
Techniques in Algorithms and
Data Analysis
Second Edition
Michael Mitzenmacher Eli Upfal
UniversityPrintingHouse,CambridgeCB28BS,UnitedKingdom
OneLibertyPlaza,20thFloor,NewYork,NY10006,USA
477WilliamstownRoad,PortMelbourne,VIC3207,Australia
4843/24,2ndFloor,AnsariRoad,Daryaganj,Delhi-110002,India
79AnsonRoad,#06-04/06,Singapore079906
CambridgeUniversityPressispartoftheUniversityofCambridge.
ItfurtherstheUniversity’smissionbydisseminatingknowledgeinthepursuitof
education,learning,andresearchatthehighestinternationallevelsofexcellence.
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781107154889
10.1017/9781316651124
©MichaelMitzenmacherandEliUpfal2017
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2017
PrintedintheUnitedStatesofAmericabySheridanBooks,Inc.
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary.
LibraryofCongressCataloginginPublicationData
Names:Mitzenmacher,Michael,1969–author.|Upfal,Eli,1954–author.
Title:Probabilityandcomputing/MichaelMitzenmacherEliUpfal.
Description:Secondedition.|Cambridge,UnitedKingdom;
NewYork,NY,USA:CambridgeUniversityPress,[2017]|
Includesbibliographicalreferencesandindex.
Identifiers:LCCN2016041654|ISBN9781107154889
Subjects:LCSH:Algorithms.|Probabilities.|Stochasticanalysis.
Classification:LCCQA274.M574 2017|DDC518/.1–dc23
LCrecordavailableathttps://lccn.loc.gov/2016041654
ISBN978-1-107-15488-9Hardback
Additionalresourcesforthispublicationatwww.cambridge.org/Mitzenmacher.
CambridgeUniversityPresshasnoresponsibilityforthepersistenceoraccuracy
ofURLsforexternalorthird-partyInternetWebsitesreferredtointhispublication
anddoesnotguaranteethatanycontentonsuchWebsitesis,orwillremain,
accurateorappropriate.
To
Stephanie,Michaela,Jacqueline,andChloe
M.M.
Liane,Tamara,andIlan
E.U.
Contents
PrefacetotheSecondEdition pagexv
PrefacetotheFirstEdition xvii
1 EventsandProbability 1
1.1 Application:VerifyingPolynomialIdentities 1
1.2 AxiomsofProbability 3
1.3 Application:VerifyingMatrixMultiplication 8
1.4 Application:NaïveBayesianClassifier 12
1.5 Application:ARandomizedMin-CutAlgorithm 15
1.6 Exercises 17
2 DiscreteRandomVariablesandExpectation 23
2.1 RandomVariablesandExpectation 23
2.1.1 LinearityofExpectations 25
2.1.2 Jensen’sInequality 26
2.2 TheBernoulliandBinomialRandomVariables 27
2.3 ConditionalExpectation 29
2.4 TheGeometricDistribution 33
2.4.1 Example:CouponCollector’sProblem 35
2.5 Application:TheExpectedRun-TimeofQuicksort 37
2.6 Exercises 40
3 MomentsandDeviations 47
3.1 Markov’sInequality 47
3.2 VarianceandMomentsofaRandomVariable 48
3.2.1 Example:VarianceofaBinomialRandomVariable 51
vii
contents
3.3 Chebyshev’sInequality 51
3.3.1 Example:CouponCollector’sProblem 53
3.4 MedianandMean 55
3.5 Application:ARandomizedAlgorithmforComputingtheMedian 57
3.5.1 TheAlgorithm 58
3.5.2 AnalysisoftheAlgorithm 59
3.6 Exercises 62
4 ChernoffandHoeffdingBounds 66
4.1 MomentGeneratingFunctions 66
4.2 DerivingandApplyingChernoffBounds 68
4.2.1 ChernoffBoundsfortheSumofPoissonTrials 68
4.2.2 Example:CoinFlips 72
4.2.3 Application:EstimatingaParameter 72
4.3 BetterBoundsforSomeSpecialCases 73
4.4 Application:SetBalancing 76
4.5 TheHoeffdingBound 77
4.6∗ Application:PacketRoutinginSparseNetworks 79
4.6.1 PermutationRoutingontheHypercube 80
4.6.2 PermutationRoutingontheButterfly 85
4.7 Exercises 90
5 Balls,Bins,andRandomGraphs 97
5.1 Example:TheBirthdayParadox 97
5.2 BallsintoBins 99
5.2.1 TheBalls-and-BinsModel 99
5.2.2 Application:BucketSort 101
5.3 ThePoissonDistribution 101
5.3.1 LimitoftheBinomialDistribution 105
5.4 ThePoissonApproximation 107
5.4.1∗ Example:CouponCollector’sProblem,Revisited 111
5.5 Application:Hashing 113
5.5.1 ChainHashing 113
5.5.2 Hashing:BitStrings 114
5.5.3 BloomFilters 116
5.5.4 BreakingSymmetry 118
5.6 RandomGraphs 119
5.6.1 RandomGraphModels 119
5.6.2 Application:HamiltonianCyclesinRandomGraphs 121
5.7 Exercises 127
5.8 AnExploratoryAssignment 133
6 TheProbabilisticMethod 135
6.1 TheBasicCountingArgument 135
viii
