Table Of ContentCOMBINATORIAL OPTIMIZATION
LECTURE NOTES FOR CS(cid:2)(cid:3)(cid:2)(cid:4)OR(cid:2)(cid:5)(cid:6)
Andrew V(cid:2) Goldberg
Supported in part by NSF Presidential Young Investigator Grant CCR(cid:2)(cid:3)(cid:3)(cid:4)(cid:3)(cid:5)(cid:6)(cid:7) with
matching funds from AT(cid:8)T(cid:9) DEC(cid:9) and (cid:10)M(cid:9) ONR Young Investigator Award N(cid:5)(cid:5)(cid:5)(cid:11)(cid:12)(cid:2)(cid:6)(cid:11)(cid:2)J(cid:2)
(cid:11)(cid:3)(cid:4)(cid:4)
(cid:0)
Contents
Preface (cid:2)
Lecture I(cid:3) The Stable Marriage and Stable Roommates Problems (cid:4)
(cid:5)(cid:3) The Stable Marriage Problem (cid:4)
(cid:11)(cid:13)(cid:11)(cid:13) De(cid:14)nition of the Stable Marriage Problem (cid:3)
(cid:11)(cid:13)(cid:15)(cid:13) The Gale(cid:2)Shapley Algorithm (cid:6)
(cid:11)(cid:13)(cid:10)(cid:13) Correctness (cid:11)(cid:5)
(cid:11)(cid:13)(cid:12)(cid:13) Male Optimality (cid:11)(cid:5)
(cid:6)(cid:3) Introduction to Network Stability (cid:5)(cid:5)
(cid:15)(cid:13)(cid:11)(cid:13) The Stable Roommates Problem (cid:11)(cid:11)
(cid:15)(cid:13)(cid:15)(cid:13) Network De(cid:14)nitions (cid:11)(cid:11)
(cid:15)(cid:13)(cid:10)(cid:13) The Stable Roommate Problem and Network Stability (cid:11)(cid:10)
Lecture II(cid:3) The Stable Roommates Problem and Network Stability (cid:5)(cid:7)
(cid:5)(cid:3) The Stable Roommates Problem and X(cid:8)Network Stability (cid:5)(cid:7)
(cid:6)(cid:3) Adjacency(cid:8)Preserving Circuits and X(cid:8)Networks (cid:5)(cid:9)
(cid:10)(cid:3) Network Stability and Simpli(cid:11)cation (cid:5)(cid:9)
Lecture III(cid:3) The Maximum Flow Problem (cid:6)(cid:12)
(cid:5)(cid:3) Network Flows (cid:6)(cid:12)
(cid:11)(cid:13)(cid:11)(cid:13) Flows and Pseudo(cid:2)(cid:16)ows (cid:15)(cid:5)
(cid:11)(cid:13)(cid:15)(cid:13) Cuts (cid:15)(cid:5)
(cid:11)(cid:13)(cid:10)(cid:13) Residual Networks (cid:15)(cid:11)
(cid:11)(cid:13)(cid:12)(cid:13) Augmenting Paths (cid:15)(cid:15)
(cid:6)(cid:3) The Augmenting Path Algorithm (cid:6)(cid:6)
(cid:2)
CONTENTS (cid:3)
(cid:10)(cid:3) The Decomposition Theorem (cid:6)(cid:13)
(cid:13)(cid:3) The Capacity Scaling Algorithm (cid:6)(cid:9)
Lecture IV(cid:3) The Push(cid:8)Relabel Algorithm (cid:6)(cid:4)
(cid:5)(cid:3) Preliminaries (cid:6)(cid:4)
(cid:6)(cid:3) The Algorithm (cid:6)(cid:14)
(cid:15)(cid:13)(cid:11)(cid:13) A Rough Description of the Algorithm (cid:15)(cid:6)
(cid:15)(cid:13)(cid:15)(cid:13) Initializing the Algorithm (cid:15)(cid:6)
(cid:15)(cid:13)(cid:10)(cid:13) The Push Operation (cid:10)(cid:5)
(cid:15)(cid:13)(cid:12)(cid:13) The Relabel Operation (cid:10)(cid:5)
(cid:15)(cid:13)(cid:4)(cid:13) The Algorithm (cid:10)(cid:15)
(cid:10)(cid:3) Analysis of the Algorithm (cid:10)(cid:6)
Lecture V(cid:3) Implementations of Push(cid:8)Relabel Algorithms (cid:10)(cid:7)
(cid:5)(cid:3) Simple Implementation (cid:10)(cid:7)
(cid:11)(cid:13)(cid:11)(cid:13) Data Structures (cid:10)(cid:4)
(cid:11)(cid:13)(cid:15)(cid:13) Discharge Subroutine (cid:10)(cid:17)
(cid:6)(cid:3) Maximum Distance Discharge Algorithm (cid:10)(cid:2)
(cid:15)(cid:13)(cid:11)(cid:13) Implementation of Maximum Distance Discharge Algorithm (cid:10)(cid:3)
(cid:10)(cid:3) Implementation with Dynamic Trees (cid:10)(cid:14)
(cid:10)(cid:13)(cid:11)(cid:13) Dynamic Trees Data Structure (cid:10)(cid:6)
(cid:10)(cid:13)(cid:15)(cid:13) Send Operation (cid:10)(cid:6)
Lecture VI(cid:3) Minimum Cycle Mean in a Digraph (cid:13)(cid:6)
(cid:5)(cid:3) Problem Motivation (cid:13)(cid:6)
(cid:6)(cid:3) Problem De(cid:11)nition (cid:13)(cid:6)
(cid:10)(cid:3) Algorithm (cid:13)(cid:7)
(cid:13)(cid:3) Correctness (cid:13)(cid:9)
Lecture VII(cid:3) The Blocking Flow Algorithm and Bipartite Matching (cid:13)(cid:2)
(cid:5)(cid:3) Dinitz(cid:15) Blocking Flow Algorithm (cid:13)(cid:2)
(cid:6)(cid:3) Bipartite Matching (cid:13)(cid:14)
Lecture VIII(cid:3) Two Phase Push(cid:8)Relabel Algorithm (cid:7)(cid:6)
(cid:4) CONTENTS
(cid:5)(cid:3) Two Phase Push(cid:8)Relabel Algorithm (cid:7)(cid:6)
(cid:11)(cid:13)(cid:11)(cid:13) Phase I (cid:4)(cid:15)
(cid:11)(cid:13)(cid:15)(cid:13) Phase II (cid:4)(cid:15)
(cid:6)(cid:3) Correctness of the Two Phase Push(cid:8)Relabel Algorithm (cid:7)(cid:10)
(cid:10)(cid:3) Some Remarks on the Two Phase Push(cid:8)Relabel Algorithm (cid:7)(cid:13)
(cid:10)(cid:13)(cid:11)(cid:13) Exact Relabeling (cid:4)(cid:12)
(cid:10)(cid:13)(cid:15)(cid:13) E(cid:18)cient practical implementation (cid:4)(cid:12)
(cid:10)(cid:13)(cid:10)(cid:13) What is the best algorithm for the Maximum Flow Problem(cid:19) (cid:4)(cid:4)
(cid:13)(cid:3) Bipartite Matching Problem (cid:7)(cid:7)
Lecture IX(cid:3) The Minimum Cut Problem (cid:7)(cid:4)
Minimum Cut Algorithms (cid:7)(cid:4)
The Hao(cid:8)Orlin Algorithm (cid:7)(cid:14)
Selecting the next sink (cid:9)(cid:5)
Analysis of the algorithm (cid:9)(cid:6)
Running Time (cid:17)(cid:10)
Lecture X(cid:3) Minimum Cost Flow and Assignment Problems (cid:9)(cid:13)
(cid:5)(cid:3) Minimum Cost Flows (cid:9)(cid:13)
(cid:6)(cid:3) Prices and Reduced Costs (cid:9)(cid:9)
(cid:10)(cid:3) Capacity Scaling (cid:9)(cid:2)
(cid:13)(cid:3) Assignment Problem (cid:9)(cid:2)
Lecture XI(cid:3) Cost Scaling for Min(cid:8)Cost Circulation Problem (cid:9)(cid:14)
(cid:5)(cid:3) Cost Scaling (cid:9)(cid:14)
Approximate Optimality (cid:17)(cid:6)
(cid:6)(cid:3) The Algorithm (cid:2)(cid:12)
(cid:10)(cid:3) Re(cid:11)nement Based on Push(cid:8)Relabel Operations (cid:2)(cid:12)
Lecture XII(cid:3) Cost Scaling and Strongly Polynomial Algorithms (cid:2)(cid:13)
(cid:5)(cid:3) Overview (cid:2)(cid:13)
(cid:6)(cid:3) Cost(cid:8)Scaling (cid:2)(cid:13)
(cid:15)(cid:13)(cid:11)(cid:13) Maximum Node Discharge and Wave Methods (cid:7)(cid:4)
(cid:15)(cid:13)(cid:15)(cid:13) Running Times (cid:7)(cid:17)
CONTENTS (cid:5)
(cid:10)(cid:3) Strongly Polynomial Algorithms (cid:2)(cid:9)
(cid:10)(cid:13)(cid:11)(cid:13) Fitting Price Functions and Tight Error Parameters (cid:7)(cid:17)
(cid:10)(cid:13)(cid:15)(cid:13) Tight Error Parameters (cid:7)(cid:7)
(cid:10)(cid:13)(cid:15)(cid:13)(cid:11)(cid:13)
(cid:10)(cid:13)(cid:10)(cid:13) Strongly Polynomial Cost Scaling (cid:7)(cid:6)
(cid:13)(cid:3) Minimum(cid:8)Mean Cycle(cid:8)Canceling (cid:4)(cid:12)
Lecture XIII(cid:3) The Multicommodity Flow (cid:16) Concurrent Flow Problems (cid:4)(cid:10)
(cid:5)(cid:3) The Multicommodity Flow Problem (cid:4)(cid:10)
(cid:6)(cid:3) Concurrent Flow Problem (cid:4)(cid:13)
(cid:15)(cid:13)(cid:11)(cid:13) De(cid:14)nitions and Notation (cid:3)(cid:12)
(cid:15)(cid:13)(cid:15)(cid:13) Relaxed Optimality (cid:3)(cid:17)
Lecture XIV(cid:3) The Concurrent Flow Algorithm (cid:4)(cid:4)
(cid:5)(cid:3) Preliminaries (cid:4)(cid:4)
(cid:11)(cid:13)(cid:11)(cid:13) Exponential Length Function (cid:3)(cid:3)
(cid:6)(cid:3) The Algorithm (cid:4)(cid:14)
(cid:15)(cid:13)(cid:11)(cid:13) Potential Function (cid:6)(cid:5)
(cid:15)(cid:13)(cid:15)(cid:13) Initial Solution (cid:6)(cid:5)
(cid:15)(cid:13)(cid:10)(cid:13) Improving the Current Solution (cid:6)(cid:5)
(cid:15)(cid:13)(cid:12)(cid:13) Algorithm Analysis (cid:6)(cid:5)
Lecture XV(cid:3) The General Matching Problem (cid:14)(cid:10)
(cid:5)(cid:3) The Matching Problem (cid:14)(cid:10)
(cid:6)(cid:3) Finding Augmenting Paths (cid:14)(cid:13)
(cid:10)(cid:3) Blossom Shrinking Algorithm (cid:14)(cid:2)
(cid:13)(cid:3) Complexity (cid:14)(cid:4)
Lecture XVI(cid:3) The Travelling Salesman Problem (cid:5)(cid:12)(cid:12)
(cid:5)(cid:3) NP(cid:8)Complete Problems (cid:5)(cid:12)(cid:12)
(cid:6)(cid:3) Traveling Salesman Problem (cid:17) Preliminaries (cid:5)(cid:12)(cid:5)
(cid:10)(cid:3) Approximation Algorithms for the TSP (cid:5)(cid:12)(cid:6)
(cid:10)(cid:13)(cid:11)(cid:13) Preliminaries (cid:11)(cid:5)(cid:15)
(cid:10)(cid:13)(cid:15)(cid:13) (cid:15)(cid:2)Approximation Tree Algorithm (cid:11)(cid:5)(cid:12)
(cid:10)(cid:13)(cid:10)(cid:13) Christo(cid:14)des(cid:20) Algorithm (cid:11)(cid:5)(cid:4)
(cid:6) CONTENTS
(cid:13)(cid:3) Heuristic Algorithms (cid:5)(cid:12)(cid:9)
Lecture XVII(cid:3) Branch and Bound Algorithms for TSP (cid:5)(cid:12)(cid:4)
(cid:5)(cid:3) Heuristic Algorithms (cid:5)(cid:12)(cid:4)
(cid:11)(cid:13)(cid:11)(cid:13) Tour Construction Heuristics (cid:11)(cid:5)(cid:3)
(cid:11)(cid:13)(cid:15)(cid:13) Tour Improvement Heuristics (cid:11)(cid:5)(cid:3)
(cid:6)(cid:3) Branch And Bound Method (cid:5)(cid:12)(cid:14)
(cid:15)(cid:13)(cid:11)(cid:13) Integer Programming Formulation of TSP (cid:11)(cid:5)(cid:6)
(cid:15)(cid:13)(cid:15)(cid:13) Branching (cid:11)(cid:5)(cid:6)
(cid:15)(cid:13)(cid:10)(cid:13) Assignment Problem Relaxation (cid:11)(cid:11)(cid:5)
(cid:15)(cid:13)(cid:12)(cid:13) Branching Rules (cid:11)(cid:11)(cid:11)
Lecture XVIII(cid:3) Held(cid:8)Karp Lower Bound for TSP (cid:5)(cid:5)(cid:6)
(cid:5)(cid:3) Problem Motivation (cid:5)(cid:5)(cid:6)
(cid:6)(cid:3) Problem De(cid:11)nition (cid:5)(cid:5)(cid:10)
(cid:10)(cid:3) Equivalence of TSP to an Integer Linear Program (cid:5)(cid:5)(cid:10)
(cid:13)(cid:3) Branch and Bound (cid:5)(cid:5)(cid:10)
(cid:7)(cid:3) Relaxation of TSP to Lagrangian dual (cid:5)(cid:5)(cid:13)
(cid:4)(cid:13)(cid:11)(cid:13) (cid:11)(cid:2)Trees (cid:11)(cid:11)(cid:12)
(cid:4)(cid:13)(cid:15)(cid:13) Penalty Function to Introduce Bias Towards Tours (cid:11)(cid:11)(cid:12)
(cid:9)(cid:3) Revision of Relaxed Problem on (cid:5)(cid:8)Trees to Use Penalties (cid:5)(cid:5)(cid:7)
(cid:17)(cid:13)(cid:11)(cid:13) Subgradient Optimization (cid:11)(cid:11)(cid:4)
(cid:17)(cid:13)(cid:15)(cid:13) Held(cid:2)Karp lower bound is not strict (cid:11)(cid:11)(cid:17)
(cid:17)(cid:13)(cid:10)(cid:13) A Branching Rule (cid:11)(cid:11)(cid:17)
(cid:2)(cid:3) Misc (cid:5)(cid:5)(cid:2)
Lecture XIX(cid:3) Interior Point Method for Linear Programming (cid:5)(cid:5)(cid:4)
(cid:5)(cid:3) Overview (cid:5)(cid:5)(cid:4)
(cid:6)(cid:3) Gonzaga(cid:15)s Algorithm (cid:5)(cid:5)(cid:14)
Lecture XX(cid:3) Analysis of Gonzaga(cid:15)s Algorithm (cid:5)(cid:6)(cid:10)
Bibliography (cid:5)(cid:6)(cid:2)
PREFACE (cid:7)
Preface
This is a set of lecture notes for CS(cid:2)(cid:3)(cid:2)(cid:4)OR(cid:2)(cid:5)(cid:6)(cid:7) Combinatorial Optimization that I
taught in Winter (cid:11)(cid:6)(cid:6)(cid:10)(cid:13) The notes were scribed by students in the class and reviewed me(cid:13)
These notes are not ment to be in the polished form(cid:9) and probably contain some errors
and ommisions(cid:13) In particular(cid:9) some references to the literature are missing(cid:13) The reader is
directed to the relevant books and research papers for a formal treatment of the material(cid:13)
These notes are intended as an aid in teaching and for studying advanced work in the
area(cid:13) Much of the material covered in these notes is very recent and is available in recearch
papers only(cid:13) I plan toupdate the notes each time the course is tought(cid:13) I would like toknow
of any errors and ommisions so that they can be corrected in the future(cid:13)
I would like to thank the students who took the course and scribed(cid:13) I would also like to
thank my teaching assistants(cid:9) Sherry Listgarten and Robert Kennedy(cid:9) for their help with
the class in general and with these notes in particular(cid:13)
LECTURE I
The Stable Marriage and Stable Roommates Problems
Scribe(cid:2) Je(cid:3)ery D(cid:4) Oldham
The stable marriage and network stability problems introduce combinatorial optimiza(cid:2)
tion(cid:9) which involves discrete(cid:9) as opposed to continuous(cid:9) optimization(cid:13)
(cid:5)(cid:3) The Stable Marriage Problem
(cid:5)(cid:3)(cid:5)(cid:3) De(cid:11)nition of the Stable Marriage Problem(cid:3) The stable marriage problem
consists of matching members of two di(cid:21)erent sets according to the members(cid:20) preferences
for the other set(cid:20)s members(cid:13) For example(cid:9) medical school students serve as residents at
hospitals after graduation(cid:13) Under the National Intern Matching Program which matches
residents (cid:22)interns(cid:23) and hospitals(cid:9) the students submit rankings of preferred hospitals(cid:9) while
hospitalssubmitrankingsofpreferredstudents(cid:13) Usingtherankings(cid:9)eachstudentisassigned
to one hospital (cid:24)(cid:15)(cid:15)(cid:25)(cid:13)
The input to the stable marriage problem consists of(cid:26)
a set M of n males(cid:9)
(cid:0)
a set F of n females(cid:9)
(cid:0)
for each male and female(cid:9) a list of all the members of the opposite gender in order
(cid:0)
of preference(cid:13)
(cid:0)
Note that the size of the input is (cid:27)(cid:22)n (cid:23)(cid:9) since each of the (cid:15)n preference lists has length n(cid:13)
Amarriage is aone(cid:2)to(cid:2)one mapping between males and females(cid:13) We wantto(cid:14)nd a stable
marriage(cid:9) i(cid:8)e(cid:8)(cid:9) onein which thereis no pairmf such thatf prefersm toher currentpartner
and m prefers f over his current partner(cid:13) Two partners are stable if they are matched in
some stable marriage(cid:13)
Example (cid:11)(cid:2) Consider the following preference lists for the sets M (cid:28) a(cid:2)b(cid:2)c and F (cid:28)
f g
A(cid:2)B(cid:2)C (cid:8)
f g
(cid:8)
(cid:0)(cid:9) THE STABLE MARRIAGE PROBLEM (cid:10)
a(cid:26) ABC A(cid:26) bac
b(cid:26) ABC B(cid:26) acb
c(cid:26) BCA C(cid:26) abc
Male a prefers female A over females B and C(cid:9) and female A prefers male b(cid:8) The marriage
aC(cid:2)bB(cid:2)cA is not stable because a prefers B over his current partner C and B prefers a
f g
over her current partner b(cid:8) The marriage aB(cid:2)bA(cid:2)cC is stable(cid:8)
f g
(cid:5)(cid:3)(cid:6)(cid:3) The Gale(cid:8)Shapley Algorithm(cid:3) The Gale(cid:2)Shapley algorithm (cid:24)(cid:11)(cid:10)(cid:25)(cid:9) given prefer(cid:2)
ence lists for both males and females(cid:9) returns a stable marriage by having any unmatched
male propose to females in the order of his preference lists until a female accepts(cid:13) (cid:22)See
Figure I(cid:13)(cid:11)(cid:13)(cid:23) A female receiving a proposal accepts if she is not matched or she prefers the
proposer over her current partner(cid:13) In the latter case(cid:9) the partner becomes single(cid:13)
while there exists an unmarried male m do
m proposes to the (cid:14)rst female f on his preference list to which he has not proposed
if f is unmarried then match m and f
else if f prefers m to her current match m(cid:0) then
make m(cid:0) single
match m and f
else m remains single
Figure I(cid:2)(cid:3)(cid:2) Gale(cid:2)Shapley Algorithm(cid:13)
Example (cid:15)(cid:2) Suppose we desire a stable marriage for the preference lists
a(cid:26) BAC A(cid:26) acb
b(cid:26) BAC B(cid:26) bac
c(cid:26) ACB C(cid:26) cab
The algorithm yields the stable marriage aA(cid:2)bB(cid:2)cC by iterating these steps(cid:8)
f g
(cid:22)(cid:11)(cid:23) a proposes to B(cid:9) the (cid:10)rst female on his preference list(cid:9) and she accepts because she
is single(cid:8) The set of matches is aB (cid:8)
f g
(cid:22)(cid:15)(cid:23) b proposes to B(cid:9) the (cid:10)rst female on his preference list(cid:9) and she accepts(cid:9) breaking her
match with a(cid:9) because she prefers b over a(cid:8) The set of matches is bB (cid:8)
f g
(cid:22)(cid:10)(cid:23) Since a is now unmatched(cid:9) a proposes to A(cid:9) the next female on his list(cid:8) With A
accepting because she is not matched(cid:9) the set of matches is aA(cid:2)bB (cid:8)
f g
(cid:22)(cid:12)(cid:23) Sincecisthe onlyunmatched male(cid:9)cproposestoA(cid:9) the(cid:10)rstfemaleonhispreference
list(cid:8) She rejects the proposed match because she prefers her current partner a(cid:8) The
set of matches is aA(cid:2)bB (cid:8)
f g
(cid:22)(cid:4)(cid:23) c proposes to C(cid:9) who accepts because she is not matched(cid:8) Since all males are now
matched(cid:9) the algorithm terminates with matches aA(cid:2)bB(cid:2)cC (cid:8)
f g
The order of choosing unmatched males does not a(cid:11)ect the algorithm(cid:12)s result(cid:8)
