Table Of ContentCognitive Technologies
ManagingEditors: D.M.Gabbay J.Siekmann
EditorialBoard: A.Bundy J.G.Carbonell
M.Pinkal H.Uszkoreit M.Veloso W.Wahlster
M.J.Wooldridge
Forfurthervolumes:
www.springer.com/series/5216
Stanislav Živný
The Complexity
of Valued Constraint
Satisfaction
Problems
StanislavŽivný
UniversityCollege
UniversityofOxford
Oxford,UK
ManagingEditors
Prof.DovM.Gabbay Prof.Dr.JörgSiekmann
AugustusDeMorganProfessorofLogic ForschungsbereichDeduktions-und
DepartmentofComputerScience Multiagentensysteme
King’sCollegeLondon DFKI
London,UK Saarbrücken,Germany
ISSN1611-2482 CognitiveTechnologies
ISBN978-3-642-33973-8 ISBN978-3-642-33974-5(eBook)
DOI10.1007/978-3-642-33974-5
SpringerHeidelbergNewYorkDordrechtLondon
LibraryofCongressControlNumber:2012951706
ACMComputingClassification(1998): F.2.2,G.2.1,I.2
©Springer-VerlagBerlinHeidelberg2012
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.
PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations
areliabletoprosecutionundertherespectiveCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Whiletheadviceandinformationinthisbookarebelievedtobetrueandaccurateatthedateofpub-
lication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityforany
errorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,withrespect
tothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Thisbookisdedicatedtothememoryof
mymother,MartaŽivná(1954–2010)
Preface
Computerscienceisnomoreaboutcomputers
thanastronomyisabouttelescopes.
EdsgerDijkstra
Thetopicofthisbookisthefollowingoptimisationproblem:givenasetofdis-
cretevariablesandasetoffunctions,eachdependingonasubsetofthevariables,
minimise the sum of the functions over all variables. This fundamental research
problem has been studied within several different contexts of computer science
and artificial intelligence under different names: Min-Sum Problems, inference in
MarkovRandomFields(MRFs)andConditionalRandomFields(CRFs),Gibbsen-
ergyminimisation,valuedconstraintsatisfactionproblems(VCSPs),and(fortwo-
state variables) pseudo-Boolean optimisation. We present general techniques for
analysing the structure of such functions and the computational complexity of the
minimisationproblem.
ThisbookcouldnothavebeenwrittenwithoutthesupportofOxford’sUniversity
College, which funded me through a Stipendiary Junior Research Fellowship in
MathematicalandPhysicalSciencesfor3years.
ManyresultsinthisbookarejointworkwithDaveCohenandPeteJeavons,from
whomIhavelearnttheropesofacademicwork.Petehasalsoservedasmymentor
andPh.D.supervisoratOxford.Iamgratefultobothfortheiradvice,support,and
friendship.SomeresultsfromChap.2arejointworkwithPáidíCreed.Someresults
describedinChap.3arejointworkwithBrunoZanuttini.Theresultsdescribedin
Chap.7wereobtainedincollaborationwithVladimirKolmogorov,whomImetat
the Tractability Workshop in Microsoft Research Cambridge in 2010, when I was
aresearchinternthere.TheresultspresentedinChap.8arejointworkwithJohan
Thapper,whomImetattheAlgebraicCSPWorkshopattheFieldsInstituteforRe-
searchinMathematicalSciences.TheFieldsInstitutekindlyfundedmyattendance
attheworkshop.TheUKEngineeringandPhysicalSciencesResearchCouncil(EP-
SRC), the Royal Society (RS), and the French National Research Agency (ANR)
financed my trips to the University of Toulouse III. Chapter 9 briefly summarises
some of the results that have come out of these very productive research visits to
vii
viii Preface
Toulouse and have given me the opportunity to work with and learn from Martin
Cooper. I am grateful to all the above-mentioned collaborators for the time spent
together,funwehad,andeverythingIhavelearntfromthem.
Iamgratefultomysister,Radka,andmyparents-in-lawfortheirsupport.Last
butnotleast,Iwouldliketoexpressmygratitudetomywonderfulwife,Biying,for
herlove,encouragement,andinspiration.
Oxford,UK StanislavŽivný
July2012
Contents
PartI Background
1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 ValuedConstraintSatisfactionProblems . . . . . . . . . . . . . . 3
1.3 ComplexityofVCSPs . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.1 StructuralRestrictions . . . . . . . . . . . . . . . . . . . . 7
1.3.2 LanguageRestrictions . . . . . . . . . . . . . . . . . . . . 8
1.3.3 HybridRestrictions . . . . . . . . . . . . . . . . . . . . . 15
1.3.4 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . 15
1.4 Expressibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.5 AlgebraicProperties . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.6 Submodularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
PartII ExpressivePower
2 ExpressibilityofValuedConstraints . . . . . . . . . . . . . . . . . . 31
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.3 IndicatorProblem . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.4 WeightedIndicatorProblem . . . . . . . . . . . . . . . . . . . . . 35
2.5 FractionalClones . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.6 GaloisTheory . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.7.1 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . 47
2.7.2 OpenProblems. . . . . . . . . . . . . . . . . . . . . . . . 47
3 ExpressibilityofFixed-ArityLanguages . . . . . . . . . . . . . . . . 49
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.3 CrispCostFunctions . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.4 Finite-ValuedCostFunctions . . . . . . . . . . . . . . . . . . . . 58
ix
x Contents
3.5 General-ValuedCostFunctions . . . . . . . . . . . . . . . . . . . 64
3.6 CharacterisationofMul(Fmax)andfPol(Fmax) . . . . . . . . . . . 73
d d
3.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.7.1 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . 77
3.7.2 OpenProblems. . . . . . . . . . . . . . . . . . . . . . . . 78
4 ExpressibilityofSubmodularLanguages . . . . . . . . . . . . . . . . 79
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.3 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.4 Reductionto(s,t)-MIN-CUT . . . . . . . . . . . . . . . . . . . . 84
4.5 KnownClassesofExpressibleFunctions . . . . . . . . . . . . . . 86
4.6 NewClassesofExpressibleFunctions . . . . . . . . . . . . . . . 89
4.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
4.7.1 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . 93
4.7.2 OpenProblems. . . . . . . . . . . . . . . . . . . . . . . . 94
5 Non-expressibilityofSubmodularLanguages . . . . . . . . . . . . . 95
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.3 ExpressibilityofUpperFansandLowerFans . . . . . . . . . . . . 96
5.4 CharacterisationofMul(Γsub,2)andfPol(Γsub,2) . . . . . . . . . . 101
5.5 Non-expressibilityofΓ overΓ . . . . . . . . . . . . . . . . 105
sub sub,2
5.6 TheComplexityofRecognisingExpressibleFunctions. . . . . . . 108
5.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.7.1 ApplicationstoArtificialIntelligence . . . . . . . . . . . . 109
5.7.2 ApplicationstoComputerVision . . . . . . . . . . . . . . 112
5.7.3 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . 112
5.7.4 OpenProblems. . . . . . . . . . . . . . . . . . . . . . . . 113
PartIII Tractability
6 TractableLanguages . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
6.2 TractableLanguages . . . . . . . . . . . . . . . . . . . . . . . . . 117
6.3 BooleanLanguages . . . . . . . . . . . . . . . . . . . . . . . . . 122
6.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
6.4.1 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . 124
6.4.2 OpenProblems. . . . . . . . . . . . . . . . . . . . . . . . 124
7 ConservativeLanguages . . . . . . . . . . . . . . . . . . . . . . . . . 125
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
7.2 ConservativeLanguages . . . . . . . . . . . . . . . . . . . . . . . 125
7.3 GraphTechnique . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
7.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.4.1 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . 129
7.4.2 OpenProblems. . . . . . . . . . . . . . . . . . . . . . . . 129
Contents xi
8 ThePowerofLinearProgramming . . . . . . . . . . . . . . . . . . . 131
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
8.2 BasicLPRelaxation . . . . . . . . . . . . . . . . . . . . . . . . . 131
8.3 AlgorithmicConsequences . . . . . . . . . . . . . . . . . . . . . 133
8.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
8.4.1 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . 135
8.4.2 OpenProblems. . . . . . . . . . . . . . . . . . . . . . . . 135
9 HybridTractability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
9.2 TractableTriangles. . . . . . . . . . . . . . . . . . . . . . . . . . 137
9.2.1 CSPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
9.2.2 Max-CSPs . . . . . . . . . . . . . . . . . . . . . . . . . . 141
9.2.3 Finite-ValuedVCSPs . . . . . . . . . . . . . . . . . . . . 142
9.2.4 General-ValuedVCSPs . . . . . . . . . . . . . . . . . . . 146
9.3 Cross-FreeConvexity . . . . . . . . . . . . . . . . . . . . . . . . 146
9.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
9.4.1 RelatedWork . . . . . . . . . . . . . . . . . . . . . . . . 148
9.4.2 OpenProblems. . . . . . . . . . . . . . . . . . . . . . . . 149
10 SummaryandOpenProblems. . . . . . . . . . . . . . . . . . . . . . 151
10.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
10.2 OpenProblems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
Introduction
Wecanonlyseeashortdistanceahead,
butwecanseeplentytherethatneedstobedone.
AlanTuring
The main topic of this book is the following optimisation problem: given a set
ofdiscretevariablesandasetoffunctions,eachdependingonasubsetofthevari-
ables, minimise the sum of the functions over all variables. This fundamental re-
searchproblemhasbeenstudiedwithinseveraldifferentcontextsofartificialintel-
ligence, computer science, and combinatorial optimisation under different names:
Min-SumProblems[268],MAPinferenceinMarkovRandomFields(MRFs) and
Conditional Random Fields (CRFs) [203, 267], Gibbs energy minimisation [128],
valued constraint satisfaction problems [97], and (for two-state variables) pseudo-
Booleanoptimisation[34,84].
Westartoffwithseveralexamplesofinterestingandwell-studiedproblemsthat
canbemodelledinthisway.
Example1(Satisfiability) ThestandardpropositionalSATISFIABILITYproblemfor
ternaryclauses,3-SAT[125],consistsindeterminingwhetheritispossibletosatisfy
a Boolean formula given as a conjunction of ternary clauses, where each clause
is a set of three literals and each literal is either a variable or the negation of a
variable. A generalisation of 3-SAT, the 3-MAX-SAT problem [125] consists in
finding an assignment of 0s and 1s (representing FALSE and TRUE, respectively)
to all variables in the given formula such that the number of satisfied clauses is
maximised,orequivalently(withrespecttoexactsolvability),findinganassignment
of0sand1stoallvariablessuchthatthenumberofunsatisfiedclausesisminimised.
Any 3-SAT instancecanbeeasilyseenasaminimisationproblemwiththeob-
jectivefunctiongivenbyasumofternary{0,∞}-valuedfunctions,onefunctionfor
eachclause.For instance,clause (x∨¬v∨z) yieldsa ternaryfunction f defined
byf(0,1,0)=∞andf(x,y,z)=0otherwise.
Similarly,any 3-MAX-SATinstancecanbeeasilyseenasaminimisationprob-
lem with the objective function given by a sum of ternary {0,1}-valued functions,
one function for each clause. For instance, clause (x ∨¬v ∨z) yields a ternary
functionf definedbyf(0,1,0)=1andf(x,y,z)=0otherwise.
xiii
