Table Of ContentSvenO.Krumke
Integer Programming
PolyhedraandAlgorithms
Draft: March6,2017
5 S S
4 shrinkS
3 G GS
shrinkS¯=V\S
2
S
1
0
V\S
0 1 2 3 4 5 GS¯
ii
These course notes are based on
my lectures »Integer Program-
ming: Polyhedral Theory« and
»IntegerProgramming:Algorithms«
attheUniversityofKaiserslautern.
Iwouldbehappytoreceivefeed-
back,inparticularsuggestionsfor
improvement and notificiations
oftyposandothererrors.
SvenO.Krumke
[email protected]
File:–sourcefile– Revision:–revision– Date:2017/03/06–time–GMT
Contents
1 Introduction 1
1.1 AfirstLookatIntegerLinearPrograms . . . . . . . . . . . . . . . . . . . 1
1.2 IntegerLinearPrograms . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 NotesofCaution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5 Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.6 Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2 Basics 13
2.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 ConvexHulls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 PolyhedraandFormulations . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4 Cones . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.5 LinearProgramming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.6 Agenda . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
I PolyhedralTheory 19
3 PolyhedraandIntegerPrograms 21
3.1 ValidInequalitiesandFacesofPolyhedra . . . . . . . . . . . . . . . . . . 21
3.2 InnerandInteriorPoints . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.3 TheFundamentalTheoremofLinearInequalities . . . . . . . . . . . . . . 25
3.4 TheDecompositionTheorem . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.5 Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.6 ExtremePoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.7 Facets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.8 TheCharacteristicConeandExtremeRays . . . . . . . . . . . . . . . . . 42
3.9 MostIPsareLinearPrograms . . . . . . . . . . . . . . . . . . . . . . . . 47
iv
4 IntegralityofPolyhedra 51
4.1 EquivalentDefinitionsofIntegrality . . . . . . . . . . . . . . . . . . . . . 52
4.2 MatchingsandIntegralPolyhedraI . . . . . . . . . . . . . . . . . . . . . . 53
4.3 TotalUnimodularity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4.4 ConditionsforTotalUnimodularity . . . . . . . . . . . . . . . . . . . . . 56
4.5 ApplicationsofUnimodularity: NetworkFlows . . . . . . . . . . . . . . . 58
4.6 MatchingsandIntegralPolyhedraII . . . . . . . . . . . . . . . . . . . . . 61
4.7 TotalDualIntegrality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.8 SubmodularityandMatroids . . . . . . . . . . . . . . . . . . . . . . . . . 68
II Algorithms 73
5 BasicsaboutProblemsandComplexity 75
5.1 EncodingSchemes,ProblemsandInstances . . . . . . . . . . . . . . . . . 75
5.2 TheClassesPandNP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.3 TheComplexityofIntegerProgramming. . . . . . . . . . . . . . . . . . . 80
5.4 OptimizationandSeparation . . . . . . . . . . . . . . . . . . . . . . . . . 81
6 RelaxationsandBounds 83
6.1 OptimalityandRelaxations . . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.2 CombinatorialRelaxations . . . . . . . . . . . . . . . . . . . . . . . . . . 86
6.3 LagrangianRelaxation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
6.4 Duality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
7 DynamicProgramming 99
7.1 ShortestPathsRevisited. . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
7.2 KnapsackProblems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
8 BranchandBound 103
8.1 DivideandConquer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
8.2 PruningEnumerationTrees . . . . . . . . . . . . . . . . . . . . . . . . . . 103
8.3 LP-BasedBranchandBound: AnExample . . . . . . . . . . . . . . . . . 106
8.4 TechniquesforLP-basedBranchandBound . . . . . . . . . . . . . . . . . 109
8.4.1 Reoptimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
8.4.2 OtherIssues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
9 CuttingPlanes 115
9.1 Cutting-PlaneProofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
9.2 AGeometricApproachtoCuttingPlanes: TheChvátalRank . . . . . . . . 121
9.3 Cutting-PlaneAlgorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
9.4 Gomory’sCutting-PlaneAlgorithm . . . . . . . . . . . . . . . . . . . . . 124
9.5 MixedIntegerCuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
9.6 StructuredInequalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
9.6.1 KnapsackandCoverInequalities. . . . . . . . . . . . . . . . . . . 133
9.6.2 LiftingofCoverInequalities . . . . . . . . . . . . . . . . . . . . . 137
9.6.3 SeparationofCoverInequalities . . . . . . . . . . . . . . . . . . . 141
9.6.4 TheSet-PackingPolytope . . . . . . . . . . . . . . . . . . . . . . 142
File:–sourcefile– Revision:–revision– Date:2017/03/06–time–GMT
v
10 ColumnGeneration 149
10.1 Dantzig-WolfeDecomposition . . . . . . . . . . . . . . . . . . . . . . . . 149
10.1.1 AQuickReviewofSimplexPricing . . . . . . . . . . . . . . . . . 150
10.1.2 PricingintheDantzig-WolfeDecomposition . . . . . . . . . . . . 151
10.2 Dantzig-WolfeReformulationofIntegerPrograms . . . . . . . . . . . . . 152
10.2.1 SolvingtheMasterLinearProgram . . . . . . . . . . . . . . . . . 153
10.2.2 TheCutting-StockProblem . . . . . . . . . . . . . . . . . . . . . 155
10.2.3 TheBalancedk-CutProblem. . . . . . . . . . . . . . . . . . . . . 159
10.2.4 StrengthoftheMasterLinearProgramandRelationstoLagrangeanDuality161
10.2.5 GettinganIntegralSolution . . . . . . . . . . . . . . . . . . . . . 162
11 MoreAboutLagrangianDuality 167
11.1 ConvexityandSubgradientOptimization. . . . . . . . . . . . . . . . . . . 167
11.2 SubgradientOptimizationfortheLagrangianDual . . . . . . . . . . . . . 170
11.3 LagrangianHeuristicsandVariableFixing . . . . . . . . . . . . . . . . . . 171
A Notation 173
A.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
A.2 SetsandMultisets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
A.3 AnalysisandLinearAlgebra . . . . . . . . . . . . . . . . . . . . . . . . . 174
A.4 GrowthofFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
A.5 ParticularFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
A.6 ProbabilityTheory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
A.7 GraphTheory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
A.8 TheoryofComputation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
B Symbols 179
Bibliography 181
File:–sourcefile– Revision:–revision– Date:2017/03/06–time–GMT
1
Introduction
LinearProgramscanbeusedtomodelalargenumberofproblemsarisinginpractice. A
standardformofaLinearProgramis
(LP) max cTx (1.1a)
Ax6b (1.1b)
x>0, (1.1c)
where c Rn, b Rm are given vectors and A Rm×n is a matrix. The focus of
∈ ∈ ∈
theselecturenotesistostudyextensionsofLinearProgramswherewearegivenadditional
integralityconditionsonallorsomeofthevariables.
Problemswithintegralityconstraintsonthevariablesariseinavarietyofapplications. For
instance,ifwewanttoplacefacilities,itmakessensetorequirethenumberoffacilitesto
beaninteger(itisnotclearwhatitmeanstobuild2.28firestations). Also,frequently,one
can model decisions as 0-1-variables: thevariable iszero ifwemake anegative decision
andoneotherwise.
1.1 A first Look at Integer Linear Programs
SimpleExisanexchangestudentattheUniversityofKaiserslautern.Hehasjustcompleted
his studies and is on his way home. Therefore, he has packed his six suitcases, whose
weightssumuptoexactly20kg,whichisthetotalweightallowedbyKnapsackAirlines.
Handbaggageiscurrentlyforbiddenduetosecurityreasons.
JustbeforeSimpleExisabouttoleave,hemeetsCuttyPlain,thebestfriendofhisgirlfriend
whogiveshimanothersuitcaseofweight8kg.Ashortphonecallwithhisgirlfriendshows
himthatinnowayhemayleavethissuitcaseinKaiserslautern. Thismeansthatinsteadof
20kghehasnowonly12kgleftfortherestofhisluggage.
Timeisshortandthereisnowayofrepacking. So,hemustleavesomeluggageinKaiser-
slautern.Butwhat?SimpleExremembershislectureonLinearandNetworkOptimization.
IsthereawaytosolvethisproblembyaLinearProgram?
He first notes that the different suitcases represent different values for him. He compiles
Table1.1andsetsupaLinearProgram:
max 7x +5x +9x +4x +2x +1x (1.2a)
1 2 3 4 5 6
4x +3x +6x +3x +3x +2x 612 (1.2b)
1 2 3 4 5 6
06x 61,i=1,...,6. (1.2c)
i
2 Introduction
Suitcase1 Suitcase2 Suitcase3 Suitcase4 Suitcase5 Suitcase6
Weight 4kg 3kg 6kg 3kg 3kg 2kg
Score 7 5 9 4 2 1
Table1.1: ThesuitcasesofSimpleExwithweightsandprofits
Theoptimumsolutionisgivenby
x =1,x =1,x =5/6,x =x =x =0
1 2 3 4 5 6
withvalue19.5Thereisaproblem, however. SimpleExcannotsplitSuitcase3(orpack
it in a fractional way). As Cutty Plain points out, the “real” problem to be solved is one
wherethevariablesattainonlyvaluesin{0,1}:
max 7x +5x +9x +4x +2x +1x (1.3a)
1 2 3 4 5 6
4x +3x +6x +3x +3x +2x 612 (1.3b)
1 2 3 4 5 6
x {0,1},i=1,...,6. (1.3c)
i
∈
Simple Ex is not convinced that there is a real difficulty (except for the cursed suitcae of
Cutty). If he packs the first two suitcases, then the third one will have to stay in Kaiser-
slautern(thevariableis“rounded”to0). Thisgivesaprofitof12. Theremaining5kgcan
be filled perfectly with Suitcases 4 and 6, which yields exactly 12 kg and a profit of 17.
SimpleExishappy. Thisshouldbetheoptimumsolution,shouldn’tit?
Cutty Plain does not look convinced. In fact, a long time ago, she attended a lecture on
IntegerProgramming. Itistrivialthat19.5isanupperboundfortheprofit. Sincewemust
pack suitcases completely, the optimum integral value of (1.3) must be an integer, so we
canevenstatethat19isanupperboundfortheoptimumprofit. Cuttypointsoutthatthe
Linear Program (1.2) can be improved, since as noted before, one can not pack the first
threesuitcasesalltogether. Sowemayaddtheinequality
x +x +x 62. (1.4)
1 2 3
to the Linear Program without affecting the optimum integral solution! The new (frac-
tional)optimumsolutionis:
x =1,x =0,x =1,x =2/3,x =x =0
1 2 3 4 5 6
which gives a profit of 56/3 18.666. This shows that even 19 can not be reached as
≈
optimumprofit.
SimpleExfindsthisinteresting. Herealizesthathecanaddacoupleofmoreinequalities
inthesameway. ThisgivesthenewLinearProgram:
max 7x +5x +9x +4x +2x +1x (1.5a)
1 2 3 4 5 6
4x +3x +6x +3x +3x +2x 612 (1.5b)
1 2 3 4 5 6
x +x +x 62 (1.5c)
1 2 3
x +x +x 62 (1.5d)
1 3 4
06x 61,i=1,...,6. (1.5e)
i
Solvingthisproblemyieldsasanoptimumsolution
x =0,x =1,x =1,x =1,x =x =0
1 2 3 4 5 6
with value 18. This is better than the solution provided by Simple Ex earlier. Also, this
mustbeanoptimumsolutionto(1.3),sinceitisintegral.
SimpleExwonderswhetherthiscanbedoneallthetimeandCuttyPlaintellshimthatshe
does not remember exactly. Thus, Simple Ex cancels his flight home and decides to stay
onemoresemestertoattendacourseonIntegerProgramming.
File:–sourcefile– Revision:–revision– Date:2017/03/06–time–GMT
1.2IntegerLinearPrograms 3
1.2 Integer Linear Programs
We first state the general form of a Mixed Integer Program as it will be used throughout
thesenotes:
Definition1.1(MixedIntegerLinearProgram(MIP))
A Mixed Integer Linear Program (MIP) is given by vectors c Rn, b Rm, a matrix
∈ ∈
A Rm×nandanumberp {0,...,n}.Thegoaloftheproblemistofindavectorx Rn
∈ ∈ ∈
solvingthefollowingoptimizationproblem:
(MIP) max cTx (1.6a)
Ax6b (1.6b)
x>0 (1.6c)
x Zp Rn−p. (1.6d)
∈ ×
If p = 0, then then there are no integrality constraints at all, so we obtain the Linear
Program(1.1). Ontheotherhand,ifp=n,thenallvariablesarerequiredtobeintegral.
Inthiscase,wespeakofanIntegerLinearProgram(IP):wenoteagainforlaterreference:
(IP) max cTx (1.7a)
Ax6b (1.7b)
x>0 (1.7c)
x Zn (1.7d)
∈
If in an (IP) all variables are restricted to values from the set B={0,1}, we have a 0-1-
IntegerLinearProgramorBinaryLinearIntegerProgram:
(BIP) max cTx (1.8a)
Ax6b (1.8b)
x Bn (1.8c)
∈
Most of the time we will be concerned with Integer Programs (IP) and Binary Integer
Programs(BIP).
Example1.2
ConsiderthefollowingIntegerLinearProgram:
max x+y (1.9a)
2y−3x62 (1.9b)
x+y65 (1.9c)
16x63 (1.9d)
16y63 (1.9e)
x,y Z (1.9f)
∈
The feasible region S of the problem is depicted in Figure 1.1. It consists of the integral
pointsemphasizedinred,namely
S={(1,1),(2,1),(3,1),(1,2),(2,2),(3,2),(2,3)}.
⊳
File:–sourcefile– Revision:–revision– Date:2017/03/06–time–GMT
4 Introduction
5
4
3
b
2
b b b
1
b b b
0
0 1 2 3 4 5
Figure1.1: Exampleofthefeasibleregionofanintegerprogram.
Example1.3(KnapsackProblem)
A climber is preparing for an expedition to Mount Optimization. His equipment consists
ofnitems, whereeachitemihasaprofitp Z andaweightw Z . Theclimber
i + i +
∈ ∈
knowsthathewillbeabletocarryitemsoftotalweightatmostb Z . Hewouldliketo
+
∈
packhisknapsackinsuchawaythathegetsthelargestpossibleprofitwithoutexceeding
theweightlimit.
We can formulate the KNAPSACK problem as a (BIP). Define a decision variable xi, i=
1,...,nwiththefollowingmeaning:
1 ifitemigetspackedintotheknapsack
x =
i
(cid:14)0 otherwise
Then,KNAPSACKbecomesthe(BIP)
n
max p x (1.10a)
i i
i=1
X
n
w x 6b (1.10b)
i i
i=1
X
x Bn (1.10c)
∈
⊳
Inthepreceedingexamplewehaveessentiallyidentifiedasubsetofthepossibleitemswith
a0-1-vector. GivenagroundsetE,wecanassociatewitheachsubsetF Eanincidence
⊆
vectorχF RFbysetting
∈
1 ife F
χF= ∈
e (cid:14)0 ife /F.
∈
Then, wecanidentifythevectorχF withthesetFandviceversa. Wewillseenumerous
examplesofthisidentificationintherestoftheselecturenotes. Thisidentificationappears
frequentlyinthecontextofcombinatorialoptimizationproblems.
File:–sourcefile– Revision:–revision– Date:2017/03/06–time–GMT
