Semidefinite Programming and Approximation Algorithms for NP-hard Problems

16 Pages·2008·3.94 MB·English

by  Sanjeev Arora

Semidefinite Programming and Approximation
Algorithms for NP-hard Problems: A Survey
Sanjeev Arora
Princeton University
Arora: SDP + Approx Survey
NP-completeness
Thousands of problems are NP-complete
(TSP, Scheduling, Circuit layout, Machine Learning,..)
Pragmatic Researcher
“Why the fuss? I am perfectly content with approximately
optimal solutions.” (e.g., cost within 10% of optimum)
Good news: Possible for a few problems. (“Approximation
Algorithms”)
Bad News: NP-hard for many problems. (“PCPs”)
Arora: SDP + Approx Survey
Talk Outline
• Defn of approximation and example
• SDP and its use in approximation
• Understanding SDPs <-> high dimensional geometry
• Faster algorithms (multiplicative update rule)
• Limitations of SDPs: local vs global issues
• Connections (a) metric spaces (b) avg case complexity
(c) unique games conjecture
• Open problems
Arora: SDP + Approx Survey
Approximation Algorithms
MAX-3SAT: Given 3-CNF formula !, find assignment
maximizing the number of satisfied clauses.
An "-approximation algorithm is one that for every
formula, produces in polynomial time an assignment that
satisfies at least OPT/" clauses. (" >= 1).
Good News: [KZ’97] An 8/7-approximation algorithm exists.
Bad News: [Hastad’97] If P ! NP then for every # > 0, an
(8/7 -#)-approximation algorithm does not exist.
(Similar results for many other problems…)
Arora: SDP + Approx Survey
Good news (for me)
Status of many basic problems is still unresolved:
• Vertex Cover
• Sparsest Cut and most graph partitioning problems
• Graph coloring
• Random instances of 3SAT

My feeling: Interesting algorithms remain undiscovered;
semidefinite programming (SDP) may be helpful.
SDP = Generalization of linear programming
Graph
Vector Representation
Arora: SDP + Approx Survey
Example: 2-approximation for Min Vertex Cover
G= (V, E)
Vertex Cover = Set of vertices that
touches every edge
“LP Relaxation”
most
Claim: Value at least OPT/2
Proof: On Complete Graph K ,
n
Proof: “Rounding”
OPT = n-1
but setting all x = 1/2
i
gives feasible LP soln
Arora: SDP + Approx Survey
General Philosophy…
Interested in: NP-hard
Minimization Problem
Value = OPT Write tractable
relaxation
value=
Round to get a solution of cost
= Approximation ratio = Integrality gap
Arora: SDP + Approx Survey
Main Idea in SDP: “Simulate” nonlinear
programming
Nonlinear program for Vertex Cover
Homogenized
SDP relaxation:
New variable
intended to stand for
Arora: SDP + Approx Survey
How do you understand these
vector programs?
Ans. Interesting geometric analysis
Arora: SDP + Approx Survey
Understanding SDPs <--> Understanding
phenomena in high-dimensional geometry
Vertex Cover SDP
computes c-approximation
for c < 2 iff following is true
Vertices: n unit vectors
Edges: almost-antipodal pairs
Rn Every graph in this family has an
independent set of size
Thm [Frankl-Rodl’87] False.
Arora: SDP + Approx Survey
SDP rounding: The two generations
First generation: *Uses random hyperplane as in [GW];
* Edge-by-edge analysis
Max-2SAT and Max-CUT [GW’94] ;Graph coloring
[KMS’95]; MAX-3SAT [KZ’97]; Algorithms for
Unique Games;..
Second generation: Global rounding and analysis
Graph partitioning problems [ARV’04],
Graph deletion and directed partitioning problems [ACMM’05],
New analysis of graph coloring [ACC’06]
Disproof of UGC for expanding constraints [AKKSTV’08]

(Similarly, two generations of results showing limits
on performance of SDPs)
Arora: SDP + Approx Survey
1 s t Generation Rounding: Ratio 1.13.. for MAX-CUT
[Goemans-
Williamson’93]
G = (V,E) Find that maximizes capacity .
Quadratic Programming Formulation
Semidefinite Relaxation [DP ’91, GW ’93]
Arora: SDP + Approx Survey
Randomized Rounding
[GW ’93]
Rn
v
2 v
1
v Form a cut by partitioning v ,v ,...,v
6
1 2 n
around a random hyperplane.
v
v v
3
5 i
SDP
OPT
$
ij
v
j
Old math rides to the rescue...
Arora: SDP + Approx Survey
Fact 1: No rounding algorithm can produce a better
solution out of this SDP [Feige-Schectman]
“Edges between all pairs of vectors
making an angle 138 degrees.”
Fact 2: If P NP then impossible to get 1.09-approximation
by any efficient algorithm [Hastad’97]
Fact 3: If “unique games conjecture” is true, it is impossible
to get a better than 1.13-approximation.[KKMO’05]
(i.e., algorithm on prev. slide is optimal)
Arora: SDP + Approx Survey
2 nd Generation: for c-balanced separator
G= (V, E); constant c >0
1 -1
Goal: Find cut s.t. each side contains at
least c fraction of nodes and minimized
SDP:
“Triangle
inequality”
Angle subtended by the line joining
two of them on the third is non-obtuse;
“ “ condition.
Arora: SDP + Approx Survey
Rounding algorithm for -approximation
[ARV’04]
1. Pick random hyperplane
S
2. Remove points in “slab” of width
T
3. Remove any pair (i, j) that lie on opp.
sides of slab but
4. Call remaining sets S, T. Do BFS from
S to T according to distance
5. Output level of BFS tree with least # of edges.
T
S
Arora: SDP + Approx Survey
Geometric fact underlying the analysis
(restatement of [ARV04] “Structure Theorem” by [AL06])
Vertices: unit vectors
satisfying “triangle inequality”
If then no graph
in this family is an “expander.”
(“expander” : |%(S)|! &(|S|) )
Edges:
Proof is delicate and difficult
Arora: SDP + Approx Survey
Issue of Running Time
Solving SDPs with m constraints takes
time.
m =n3 in some of these SDPs!
Next few slides: Often, can reduce running time: O(n2) or O(n3).
[AHK’05], [AK’07]
Main idea: “Primal-dual schema.”Solve to approximate optimality;
using insights from the rounding algorithms.
“Multiplicative Weight-Update Rule for psd matrices”
Arora: SDP + Approx Survey
Classical MW update rule
(Example: predicting the market)
1$ for correct prediction
0$ for incorrect
• N “experts” on TV
• Can we perform as good as the best expert ?
Thm[Going back to Hannan, 1950s] Yes.
Arora: SDP + Approx Survey
Weighted Majority Algorithm [LW’94]
“Predict according to the weighted majority”
• Maintain a weight for each expert. Initially
• At step t, if expert i’s prediction was incorrect,
Claim: Expected Payoff of our algorithm
Similar algorithms discovered in a variety of areas:
decision theory, learning theory (“boosting”),
cryptography (“hardcore sets”), approx soln of LPs,..
(see survey [A, Hazan, Kale])
Arora: SDP + Approx Survey