New Directions in Approximation Algorithms and Hardness of Approximation

345 Pages·2014·1.52 MB·English

New Directions in Approximation Algorithms
and Hardness of Approximation
Yuan Zhou
CMU-CS-14-125
August 2014
SchoolofComputerScience
CarnegieMellonUniversity
Pittsburgh,PA15213
ThesisCommittee:
VenkatesanGuruswami,Co-chair
RyanO’Donnell,Co-chair
AnupamGupta
R.Ravi
SanjeevArora(PrincetonUniversity)
Submittedinpartialfulfillmentoftherequirements
forthedegreeofDoctorofPhilosophy.
Copyright(cid:13)c 2014YuanZhou
ThisresearchwassponsoredbytheNationalScienceFoundationundergrantnumbersCCF-0915893,CCR-
0953155, CCF-1115525; US Israel Binational Science Foundation under grant number 2008293; Simons
Foundationundergrantnumber252545;andgeneroussupportfromtheMicrosoftCorporation.
Theviewsandconclusionscontainedinthisdocumentarethoseoftheauthorandshouldnotbeinterpreted
as representing the official policies, either expressed or implied, of any sponsoring institution, the U.S.
governmentoranyotherentity.
Keywords: Approximation Algorithms, Hardness of Approximation, Combinatorial
Optimization, Convex Relaxation Hierarchies, Linear Programming, Semidefinite Pro-
gramming
DEDICATION
Tomyparents.
iv
Abstract
Combinatorialoptimizationencompassesawiderangeofimportantcom-
putational tasks such as UNIFORMSPARSESTCUT (also known as NORMAL-
IZEDCUT),MAXCUT,TRAVELINGSALESMANPROBLEM,andVERTEXCOVER.
Most combinatorial optimization problems are NP-hard to be solved opti-
mally. Ononehand,anaturalwaytocopewiththiscomputationalintractabil-
ity is via designing approximation algorithms to efficiently approximate the
optimal solutions with provable guarantees. On the other hand, given an NP-
hardoptimizationproblem,wearealsointerestedinthebestpossibleapprox-
imation guarantee that any polynomial-time algorithm could achieve, i.e. the
hardness of approximation of the problem. Both approximation algorithms
andhardnessofapproximationresultscontributetounderstandingtheapprox-
imabilityofcombinatorialoptimizationproblems.
In the last two decades, the research frontier of approximation algorithm
design has been greatly advanced thanks to the convex optimization tech-
niques such as linear and semidefinite programming. However, the exact ap-
proximability for many problems remains mysterious but some common bar-
riers for progress revolving around a problem called Unique Games has been
identified. The limitations of convex relaxation techniques answer the ques-
tion that what is the best possible approximation guarantee to be achieved by
thestate-of-the-artalgorithmicdesigntools,andshedlightontherealapprox-
imability. Therefore, the study of the power of convex relaxations becomes a
valuable new research direction to get around the current barrier on hardness
proofs.
In this thesis, using constraint satisfaction problems, assignment prob-
lems,graphpartitioningproblems(BALANCEDSEPARATOR,UNIFORMSPARS-
ESTCUT, DENSEkSUBGRAPH), and graph isomorphism as examples, we ex-
plore both the effectiveness and limitations of the most powerful convex re-
laxation techniques – convex relaxation hierarchies. We also use a proof
complexity view of the convex relaxation hierarchies to analyze their per-
formance on constraint satisfaction problems, and show that the so-called
Parrilo–Lasserre semidefinite programming relaxation hierarchy succeeds on
all hard instances constructed in literature for UNIQUEGAMES, MAXCUT,
and BALANCEDSEPARATOR.
This thesis also contains a collection of approximation algorithms for al-
mostsatisfiableconstraintsatisfactionproblemsand MAXBISECTION,detec-
tion of almost isomorphic trees, and estimation of the 2 → 4 operator norm
ofrandomlinearoperators. Therearealsoafew(conditional)hardnessofap-
proximation results for almost satisfiable linear systems over integers, almost
satisfiable MAXHORN3-SAT,anddetectionofalmostisomorphicgraphs.
vi
Acknowledgments
It would not have been possible to complete this doctoral thesis without the help and
support of many people around me, to only some of whom it is possible to give particular
mentionhere.
Above all, I owe my deepest gratitude to my Ph.D. advisors Venkat Guruswami and
Ryan O’Donnell. When still in college, I read and was intrigued by a set of beautifully
scribed online lecture notes on the PCP theorem, approximation algorithms, and hardness
of approximation. A year later, I was fortunate enough to become a student of both of
the lecturers in that course. In fact, Venkat and Ryan are the best advisors I could have
askedfor. Iwasalwaysamazedbytheirbrilliance,technicalmastery,andwideknowledge
on almost everything. Venkat could always patiently go through every ridiculous ideas of
mine; and Ryan never failed to figure a new direction after we hitting thousands of dead-
ends. Overtheyears,IenjoyedallthediscussionsthroughwhichIlearnedalotfromtheir
techniques and research styles. For all these and the freedom in choosing research topics,
thecareeradvices,andalotmoreleftout–ThankyouVenkatandRyan!
I am very grateful to my thesis committee: Venkatesan Guruswami, Ryan O’Donnell,
Sanjeev Arora, Anupam Gupta, and R. Ravi. Thank you for taking time from your busy
schedulestomakemythesisdefense.
Many thanks to each of my paper co-authors through my Ph.D. : Boaz Barak, Aditya
Bhaskara,FernandoBranda˜o,MosesCharikar,XiChen,JuliaChuzhoy,ParikshitGopalan,
Venkatesan Guruswami, Aram Harrow, Zhiyi Huang, Manuel Kauers, Jonathan Kelner,
Gabor Kun, Jian Li, Yury Makarychev, Raghu Meka, Ryan O’Donnell, Prasad Raghaven-
dra,OmerReingold,AliKemalSinop,DavidSteurer,SuguruTamaki,Li-YangTan,Mad-
hur Tulsiani, Salil Vadhan, Aravindan Vijayaraghavan, Lei Wang, John Wright, Cheng-
gang Wu, Yi Wu, Yu Wu, Yuichi Yoshida, and Jiawei Zhang. It was great joy working
withyou;andthisthesiswouldnotbepossiblewithoutyourcollaborations.
I had the rare opportunity to spend each of the four summers during graduate school
visitinganumberofplaces. Whenstillgettingfamiliarwiththeoreticalresearchinmyfirst
vii
year, I had the pleasure of interning with Julia Chuzhoy and Yury Makarychev at Toyota
Technological Institute at Chicago, as well as visiting Pinyan Lu at Microsoft Research
Asia. In the following summers, I enjoyed collaborating with Boaz Barak at Microsoft
ResearchNewEngland,ParikshitGopalanatMicrosoftResearchSiliconValley,andYury
Makarychev and Madhur Tulsiani at Toyota Technological Institute at Chicago. During
winter breaks, I also enjoyed a couple of visits to Shengyu Zhang at the Chinese Univer-
sity at Hong Kong, Yitong Yin at Nanjing University, and Xiaoming Sun at the Chinese
Academy of Sciences. I am greatly indebted to all these institutions I have visited and the
numerous people who hosted me. I would also like to thank the support from the Simons
GraduateFellowshipinTheoreticalComputerScienceduringthelasttwoyears.
I wish to thank my high school teacher Tao Jiang for introducing me to the world of
algorithmic programming contest. I am also greatly indebted to Professor Andrew Yao
and Elad Verbin at Tsinghua University and Wei Chen at Microsoft Research Asia who
furtherguidedmyinteresttotheoreticalcomputerscience.
Many thanks go to the theory group faculty members at CMU from whom I learned
a lot: Avrim Blum, Manuel Blum, Alan Frieze, Anupam Gupta, Gary Miller, and Danny
Sleator.
Also, special thanks to the friends I met in Pittsburgh and during my summer intern-
ships for making my Ph.D. life colorful: Parinya Chalermsook, Zhuo Chen, Ricky Chis,
Bin Fu, Edinah Gnang, Fan Guo, Favonia Hou, Jason Lee, Lei Li, Nan Li, Jian Peng,
Richard Peng, Julian Shun, Xiaorui Sun, Carol Wang, John Wright, Chenye Wu, Yi Wu,
Guangyu Xia, Guang Xiang, Lin Xiao, Ning Xie, Jia Xu, Kathy Zhang, Zeyuan Zhu,
MengjieZou.
Finally, it would be difficult for me to express the debt of gratitude that I owe my
parentsQihuaZhouandHangliWang,fortheirloveandconstantsupport.
viii
Contents
1 Introduction 1
1.1 Thenotionofapproximationalgorithms . . . . . . . . . . . . . . . . . . 3
1.1.1 Robustalgorithms . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1.2 Hardnessofapproximation . . . . . . . . . . . . . . . . . . . . . 4
1.2 The relaxation and rounding framework for designing approximation al-
gorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.1 Convexrelaxations . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.2 Roundingschemes . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.3 Integralitygapsandlimitationsoftherelaxation . . . . . . . . . 7
1.3 Linearandsemidefiniteprogrammingrelaxations,andmethodsofdesign-
ingthem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.3.1 Relaxationhierarchiesandtheimplicationoftheirlimitations . . 9
1.4 Abriefoverviewofcontributions . . . . . . . . . . . . . . . . . . . . . . 10
2 Preliminaries 13
2.1 Problemsstudiedinthisthesis . . . . . . . . . . . . . . . . . . . . . . . 13
2.1.1 Constraintsatisfactionproblems . . . . . . . . . . . . . . . . . . 13
2.1.1.1 The UNIQUEGAMES problem . . . . . . . . . . . . . . 14
2.1.2 Graphpartitioningproblems . . . . . . . . . . . . . . . . . . . . 14
2.1.3 Graphisomorphismandassignmentproblems . . . . . . . . . . . 15
2.1.4 Denseandlocally-denseinstances . . . . . . . . . . . . . . . . . 17
ix
2.2 Approximationandhardnessofapproximation . . . . . . . . . . . . . . 18
2.2.1 Robustalgorithms . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.3 TheUniqueGamesConjecture . . . . . . . . . . . . . . . . . . . . . . . 21
3 Summaryofcontributionsandorganizationofthisthesis 23
3.1 OverviewofPartI: studyoftheLP/SDPrelaxationhierarchies . . . . . . 23
3.1.1 Algorithmicresults . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.1.2 Integralitygaps . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2 Overview of Part II: using the Parrilo–Lasserre hierarchy to solve hard
instancesforweakerhierarchies . . . . . . . . . . . . . . . . . . . . . . 27
3.3 OverviewofPartIII: robustalgorithms . . . . . . . . . . . . . . . . . . . 29
3.4 Overview of Part IV: other approximation and hardness of approximation
results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.4.1 Approximatingthe2 → 4normofrandomlinearoperators . . . . 31
3.4.2 Hardnessof MAXΓ-2-LIN and MAXΓ-3-LIN overintegers . . . 31
3.4.3 Hardnessofapproximatingalmostsatisfiable MAXHORN3-SAT . 32
I Study of the LP/SDP relaxation hierarchies 33
4 Approximation schemes via Sherali-Adams hierarchy for dense constraint
satisfactionproblemsandassignmentproblems 35
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4.1.1 LinearProgramming(LP)relaxationandLPrelaxationhierarchies 36
4.1.2 Ourcontributions . . . . . . . . . . . . . . . . . . . . . . . . . . 37
4.1.3 Proofoverview . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.1.4 Comparisontopreviousworks . . . . . . . . . . . . . . . . . . . 39
4.1.5 Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
4.3 ConditioningoperationsforSherali-AdamsLPhierarchy . . . . . . . . . 43
4.3.1 Conditioningoperations . . . . . . . . . . . . . . . . . . . . . . 44
x