Algorithms of informatics, vol. 3

328 Pages·2011·4.968 MB·English

by  Ivanyi A. (ed.)

ALGORITHMS
OF INFORMATICS
Volume 3
AnTonCom
Budapest, 2011
This electronic book was prepared in the framework of project Eastern Hungarian
Informatics Books Repository no. TÁMOP-4.1.2-08/1/A-2009-0046.
This electronic book appeared with the support of European Union and with the
co-financing of European Social Fund.
Editor: Antal Iványi
Authors of Volume 3: Béla Vizvári (Chapter 24), Antal Iványi and Shariefuddin
Pirzada (Chapter 25), Zoltán Kása, and Mira-Cristiana Anisiu (Chapter 26),
Ferenc Szidarovszky and László Domoszlai, (Chapter 27), László Szirmay-Kalos
and László Szécsi (Chapter 28), Antal Iványi (Chapter 29), Shariefuddin Pirzada,
Antal Iványi and Muhammad Ali Khan (Chapter 30)
Validators of Volume 3: Gergely Kovács (Chapter 24), Zoltán Kása (Chapter
25), Antal Iványi (Chapter 26), Sándor Molnár (Chapter 27), György Antal
(Chapter 28), Zoltán Kása (Chapter 29), Zoltán Kása (Chapter 30), Anna Iványi
(Bibliography)
c 2011 AnTonCom Infokommunikációs Kft.
(cid:13)
Homepage: http://www.antoncom.hu/
Contents
Introduction to Volume 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . 1207
24.The Branch and Bound Method . . . . . . . . . . . . . . . . . . . . . 1208
24.1. An example: the Knapsack Problem . . . . . . . . . . . . . . . . . 1208
24.1.1. The Knapsack Problem . . . . . . . . . . . . . . . . . . . . 1209
24.1.2. A numerical example . . . . . . . . . . . . . . . . . . . . . . 1211
24.1.3. Properties in the calculation of the numerical example . . . 1214
24.1.4. How to accelerate the method . . . . . . . . . . . . . . . . . 1216
24.2. The general frame of the B&B method . . . . . . . . . . . . . . . . 1217
24.2.1. Relaxation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1217
24.2.2. The general frame of the B&B method . . . . . . . . . . . . 1224
24.3. Mixed integer programming with bounded variables. . . . . . . . . 1229
24.3.1. The geometric analysis of a numerical example . . . . . . . 1230
24.3.2. The linear programming background of the method . . . . . 1232
24.3.3. Fast bounds on lower and upper branches . . . . . . . . . . 1240
24.3.4. Branching strategies . . . . . . . . . . . . . . . . . . . . . . 1244
24.3.5. The selection of the branching variable . . . . . . . . . . . . 1247
24.3.6. The numerical example is revisited . . . . . . . . . . . . . . 1248
24.4. On the enumeration tree . . . . . . . . . . . . . . . . . . . . . . . . 1252
24.5. The use of information obtained from other sources . . . . . . . . . 1254
24.5.1. Application of heuristic methods . . . . . . . . . . . . . . . 1254
24.5.2. Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . 1255
24.6. Branch and Cut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1255
24.7. Branch and Price . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1260
25. Comparison Based Ranking . . . . . . . . . . . . . . . . . . . . . . . 1262
25.1. Introduction to supertournaments . . . . . . . . . . . . . . . . . . . 1262
25.2. Introduction to (a,b)-tournaments . . . . . . . . . . . . . . . . . . 1264
25.3. Existence of (1,1)-tournaments with prescribed score sequence. . . 1265
25.4. Existence of an (a,a)-tournament with prescribed score sequence . 1267
25.5. Existence of an (a,b)-tournament with prescribed score sequence . 1269
25.5.1. Existence of a tournament with arbitrary degree sequence . 1269
25.5.2. Description of a naive reconstructing algorithm . . . . . . . 1269
25.5.3. Computation of e . . . . . . . . . . . . . . . . . . . . . . . . 1270
25.5.4. Description of a construction algorithm . . . . . . . . . . . 1271
1204 Contents
25.5.5. Computation of f and g . . . . . . . . . . . . . . . . . . . . 1271
25.5.6. Description of a testing algorithm . . . . . . . . . . . . . . . 1273
25.5.7. Description of an algorithm computing f and g . . . . . . . 1273
25.5.8. Computing of f and g in linear time . . . . . . . . . . . . . 1275
25.5.9. Tournament with f and g . . . . . . . . . . . . . . . . . . . 1276
25.5.10.Description of the score slicing algorithm. . . . . . . . . . . 1276
25.5.11.Analysis of the minimax reconstruction algorithm . . . . . . 1280
25.6. Imbalances in (0,b)-tournaments . . . . . . . . . . . . . . . . . . . 1280
25.6.1. Imbalances in (0,1)-tournaments. . . . . . . . . . . . . . . . 1281
25.6.2. Imbalances in (0,2)-tournaments . . . . . . . . . . . . . . . 1281
25.7. Supertournaments . . . . . . . . . . . . . . . . . . . . . . . . . . . 1286
25.7.1. Hypertournaments . . . . . . . . . . . . . . . . . . . . . . . 1286
25.7.2. Supertournaments . . . . . . . . . . . . . . . . . . . . . . . 1293
25.8. Football tournaments . . . . . . . . . . . . . . . . . . . . . . . . . . 1294
25.8.1. Testing algorithms . . . . . . . . . . . . . . . . . . . . . . . 1294
25.8.2. Polynomial testing algorithms of the draw sequences . . . . 1303
25.9. Reconstruction of the tested sequences . . . . . . . . . . . . . . . . 1309
26.Complexity of Words . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1312
26.1. Simple complexity measures . . . . . . . . . . . . . . . . . . . . . . 1312
26.1.1. Finite words . . . . . . . . . . . . . . . . . . . . . . . . . . . 1312
26.1.2. Infinite words . . . . . . . . . . . . . . . . . . . . . . . . . . 1314
26.1.3. Word graphs . . . . . . . . . . . . . . . . . . . . . . . . . . 1316
26.1.4. Complexity of words . . . . . . . . . . . . . . . . . . . . . . 1321
26.2. Generalized complexity measures . . . . . . . . . . . . . . . . . . . 1332
26.2.1. Rainbow words . . . . . . . . . . . . . . . . . . . . . . . . . 1332
26.2.2. General words . . . . . . . . . . . . . . . . . . . . . . . . . . 1343
26.3. Palindrome complexity . . . . . . . . . . . . . . . . . . . . . . . . . 1343
26.3.1. Palindromes in finite words . . . . . . . . . . . . . . . . . . 1343
26.3.2. Palindromes in infinite words . . . . . . . . . . . . . . . . . 1346
27. Conflict Situations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1351
27.1. The basics of multi-objective programming . . . . . . . . . . . . . . 1351
27.1.1. Applications of utility functions . . . . . . . . . . . . . . . . 1355
27.1.2. Weighting method . . . . . . . . . . . . . . . . . . . . . . . 1357
27.1.3. Distance-dependent methods . . . . . . . . . . . . . . . . . 1358
27.1.4. Direction-dependent methods . . . . . . . . . . . . . . . . . 1361
27.2. Method of equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . 1364
27.3. Methods of cooperative games . . . . . . . . . . . . . . . . . . . . . 1369
27.4. Collective decision-making . . . . . . . . . . . . . . . . . . . . . . . 1372
27.5. Applications of Pareto games . . . . . . . . . . . . . . . . . . . . . 1379
27.6. Axiomatic methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 1382
28. General Purpose Computing on Graphics Processing Units . . . 1388
28.1. The graphics pipeline model . . . . . . . . . . . . . . . . . . . . . . 1390
28.1.1. GPU as the implementation of incremental image synthesis 1392
28.2. GPGPU with the graphics pipeline model . . . . . . . . . . . . . . 1395
28.2.1. Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1395
Contents 1205
28.2.2. Input. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1396
28.2.3. Functions and parameters . . . . . . . . . . . . . . . . . . . 1397
28.3. GPU as a vector processor . . . . . . . . . . . . . . . . . . . . . . . 1398
28.3.1. Implementing the SAXPY BLAS function . . . . . . . . . . 1400
28.3.2. Image filtering . . . . . . . . . . . . . . . . . . . . . . . . . 1401
28.4. Beyond vector processing. . . . . . . . . . . . . . . . . . . . . . . . 1402
28.4.1. SIMD or MIMD . . . . . . . . . . . . . . . . . . . . . . . . 1402
28.4.2. Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1404
28.4.3. Implementing scatter . . . . . . . . . . . . . . . . . . . . . . 1405
28.4.4. Parallelism versus reuse . . . . . . . . . . . . . . . . . . . . 1407
28.5. GPGPU programming model: CUDA and OpenCL . . . . . . . . . 1409
28.6. Matrix-vector multiplication . . . . . . . . . . . . . . . . . . . . . . 1409
28.6.1. Making matrix-vector multiplication more parallel . . . . . 1411
28.7. Case study: computational fluid dynamics . . . . . . . . . . . . . . 1414
28.7.1. Eulerian solver for fluid dynamics . . . . . . . . . . . . . . . 1416
28.7.2. Lagrangian solver for differential equations. . . . . . . . . . 1421
29. Perfect Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1428
29.1. Basic concepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1428
29.2. Necessary condition and earlier results . . . . . . . . . . . . . . . . 1431
29.3. One-dimensional perfect arrays . . . . . . . . . . . . . . . . . . . . 1431
29.3.1. Pseudocode of the algorithm Quick-Martin . . . . . . . . 1431
29.3.2. Pseudocode of the algorithm Optimal-Martin. . . . . . . 1432
29.3.3. Pseudocode of the algorithm Shift . . . . . . . . . . . . . . 1432
29.3.4. Pseudocode of the algorithm Even . . . . . . . . . . . . . . 1433
29.4. Two-dimensional perfect arrays . . . . . . . . . . . . . . . . . . . . 1433
29.4.1. Pseudocode of the algorithm Mesh . . . . . . . . . . . . . . 1434
29.4.2. Pseudocode of the algorithm Cellular . . . . . . . . . . . 1434
29.5. Three-dimensional perfect arrays . . . . . . . . . . . . . . . . . . . 1435
29.5.1. Pseudocode of the algorithm Colour . . . . . . . . . . . . 1435
29.5.2. Pseudocode of the algorithm Growing . . . . . . . . . . . 1436
29.6. Construction of growing arrays using colouring . . . . . . . . . . . 1437
29.6.1. Construction of growing sequences . . . . . . . . . . . . . . 1437
29.6.2. Construction of growing squares. . . . . . . . . . . . . . . . 1438
29.6.3. Construction of growing cubes. . . . . . . . . . . . . . . . . 1439
29.6.4. Construction of a four-dimensional double hypercube . . . . 1439
29.7. The existence theorem of perfect arrays . . . . . . . . . . . . . . . 1441
29.8. Superperfect arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . 1443
29.9. d-complexity of one-dimensional arrays . . . . . . . . . . . . . . . . 1443
29.9.1. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1444
29.9.2. Bounds of complexity measures . . . . . . . . . . . . . . . . 1446
29.9.3. Recurrence relations . . . . . . . . . . . . . . . . . . . . . . 1448
29.9.4. Pseudocode of the algorithm Quick-Martin . . . . . . . . 1450
29.9.5. Pseudocode of algorithm d-Complexity . . . . . . . . . . . 1450
29.9.6. Pseudocode of algorithm Super . . . . . . . . . . . . . . . 1451
29.9.7. Pseudocode of algorithm MaxSub . . . . . . . . . . . . . . 1451
29.9.8. Construction and complexity of extremal words . . . . . . . 1452
1206 Contents
29.10.Finite two-dimensional arrays with maximal complexity . . . . . . 1456
29.10.1.Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1456
29.10.2.Bounds of complexity functions . . . . . . . . . . . . . . . . 1458
29.10.3.Properties of the maximal complexity function . . . . . . . 1459
29.10.4.On the existence of maximal arrays . . . . . . . . . . . . . . 1460
30. Score Sets and Kings . . . . . . . . . . . . . . . . . . . . . . . . . . . 1463
30.1. Score sets in 1-tournaments . . . . . . . . . . . . . . . . . . . . . . 1464
30.1.1. Determining the score set . . . . . . . . . . . . . . . . . . . 1464
30.1.2. Tournaments with prescribed score set . . . . . . . . . . . . 1466
30.2. Score sets in oriented graphs . . . . . . . . . . . . . . . . . . . . . . 1473
30.2.1. Oriented graphs with prescribed scoresets . . . . . . . . . . 1475
30.3. Unicity of score sets . . . . . . . . . . . . . . . . . . . . . . . . . . 1480
30.3.1. 1-unique score sets . . . . . . . . . . . . . . . . . . . . . . . 1481
30.3.2. 2-unique score sets . . . . . . . . . . . . . . . . . . . . . . . 1482
30.4. Kings and serfs in tournaments . . . . . . . . . . . . . . . . . . . . 1484
30.5. Weak kings in oriented graphs . . . . . . . . . . . . . . . . . . . . . 1492
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1504
Subject Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1518
Name Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1524
Introduction
The third volume contains seven new chapters.
Chapter 24 (The Branch and Bound Method) was written by Béla Vizvári
(Eastern Mediterrean University), Chapter 25 (Comparison Based Ranking) by
Antal Iványi (Eötvös Loránd University) and Shariefuddin Pirzada (University of
Kashmir), Chapter 26 (Complexity of Words) by Zoltán Kása, (Sapientia Hungar-
ian University of Transylvania) and Mira-Cristiana Anisiu (Tiberiu Popovici Insti-
tute of Numerical Mathematics), Chapter 27 (Conflict Situations) by Ferenc Szi-
darovszky(UniversityofArizona)andbyLászlóDomoszlai(EötvösLorándUniver-
sity), Chapter 28 (General Purpose Computing on Graphics Processing Units) by
László Szirmay-Kalos and László Szécsi (both Budapest University of Technology
and Economics), Chapter 29 (Perfect Arrays) by Antal Iványi (Eötvös Loránd Uni-
versity), and Chapter 30 by Shariefuddin Pirzada (University of Kashmir), Antal
Iványi (Eötvös Loránd University) and Muhammad Ali Khan (King Fahd Univer-
sity).
The LATEX style file was written by Viktor Belényesi, Zoltán Csörnyei, László
Domoszlai and Antal Iványi. The figures was drawn or corrected by Kornél Locher.
and László Domoszlai. Anna Iványi transformed the bibliography into hypertext.
The DOCBOOK version was made by Marton 2001. Kft.
Usingthedataofthecolofonpageyoucancontactwithanyofthecreatorsofthe
book. We welcome ideas for new exercises and problems, and also critical remarks
or bug reports.
The publication of the printed book (Volumes 1 and 2) was supported by De-
partment of Mathematics of Hungarian Academy of Science. This electronic book
(Volumes 1, 2, and 3) was prepared in the framework of project Eastern Hungarian
Informatics Books Repository no. TÁMOP-4.1.2-08/1/A-2009-0046.
This electronic book appeared with the support of European Union and with
the co-financing of European Social Fund.
Budapest, September 2011
Antal Iványi ([email protected])
24. The Branch and Bound Method
Ithasseriouspracticalconsequencesifitisknownthatacombinatorial problemis
NP-complete. Then one can conclude according to the present state of science that
no simple combinatorial algorithm can be applied and only an enumerative-type
method can solve the problem in question. Enumerative methods are investigating
many cases only in a non-explicit, i.e. implicit, way. It means that huge majority
of the cases are dropped based on consequences obtained from the analysis of the
particular numerical problem. The three most important enumerative methods are
(i) implicit enumeration, (ii) dynamic programming, and (iii) branch and bound
method.Thischapterisdevotedtothelatterone.Implicitenumerationanddynamic
programmingcanbeappliedwithinthefamilyofoptimizationproblemsmainlyifall
variableshavediscretenature.Branchandboundmethodcaneasilyhandleproblems
having both discrete and continuous variables. Further on the techniques of implicit
enumerationcanbeincorporatedeasilyinthebranchandboundframe.Branchand
bound method can be applied even in some cases of nonlinear programming.
TheBranch and Bound(abbreviatedfurtheronasB&B)methodisjustaframeofa
large family of methods. Its substeps can be carried out in different ways depending
on the particular problem, the available software tools and the skill of the designer
of the algorithm.
Boldface letters denote vectors and matrices; calligraphic letters are used for
sets. Components of vectors are denoted by the same but non-boldface letter. Cap-
ital letters are used for matrices and the same but lower case letters denote their
elements. The columns of a matrix are denoted by the same boldface but lower case
letters.
Someformulaewiththeirnumbersarerepeatedseveraltimesinthischapter.The
reason is that always a complete description of optimization problems is provided.
Thus the fact that the number of a formula is repeated means that the formula is
identical to the previous one.
24.1. An example: the Knapsack Problem
In this section the branch and bound method is shown on a numerical example.
The problem is a sample of the binary knapsack problem which is one of the easiest
24.1. An example: the Knapsack Problem 1209
problems of integer programming but it is still NP-complete. The calculations are
carried out in a brute force way to illustrate all features of B&B. More intelligent
calculations, i.e. using implicit enumeration techniques will be discussed only at the
end of the section.
24.1.1. The Knapsack Problem
Therearemanydifferentknapsackproblems.Thefirstandclassicaloneisthebinary
knapsack problem. It has the following story. A tourist is planning a tour in the
mountains.Hehasalotofobjectswhichmaybeusefulduringthetour.Forexample
ice pick and can opener can be among the objects. We suppose that the following
conditions are satisfied.
Eachobjecthasapositivevalueandapositiveweight.(E.g.aballoonfilledwith
•
helium has a negative weight. See Exercises 24.1-1 and 24.1-2) The value is the
degree of contribution of the object to the success of the tour.
The objects are independent from each other. (E.g. can and can opener are not
•
independent as any of them without the other one has limited value.)
The knapsack of the tourist is strong and large enough to contain all possible
•
objects.
The strength of the tourist makes possible to bring only a limited total weight.
•
Butwithinthisweightlimitthetouristwanttoachievethemaximaltotalvalue.
•
The following notations are used to the mathematical formulation of the prob-
lem:
n the number of objects;
j the index of the objects;
w the weight of objectj;
j
v the value of objectj;
j
b the maximal weight what the tourist can bring.
For each object j a so-called binary or zero-one decision variable, say x , is
j
introduced:
1 if object j is present on the tour
x =
j 0 if object j isn’t present on the tour.
(cid:26)
Notice that
w if object j is present on the tour,
w x = j
j j 0 if object j isn’t present on the tour
(cid:26)
is the weight of the object in the knapsack.
Similarly v x is the value of the object on the tour. The total weight in the
j j
knapsack is
n
w x
j j
j=1
X
1210 24. The Branch and Bound Method
whichmaynotexceedtheweightlimit.Hencethemathematicalformoftheproblem
is
n
max v x (24.1)
j j
j=1
X
n
w x b (24.2)
j j
≤
j=1
X
x =0or1, j =1,...,n. (24.3)
j
The difficulty of the problem is caused by the integrality requirement. If con-
straint (24.3) is substituted by the relaxed constraint, i.e. by
0 x 1, j =1,...,n, (24.4)
j
≤ ≤
then the Problem (29.2), (24.2), and (24.4) is a linear programming problem. (24.4)
means that not only a complete object can be in the knapsack but any part of it.
MoreoveritisnotnecessarytoapplythesimplexmethodoranyotherLPalgorithm
to solve it as its optimal solution is described by
Theorem 24.1 Suppose thatthe numbers v ,w (j =1,...,n) are allpositive and
j j
moreover the index order satisfies the inequality
v v v
1 2 n
. (24.5)
w ≥ w ···≥ w
1 2 n
Then there is an index p (1 p n) and an optimal solution x such that
∗
≤ ≤
x =x = =x =1, x =x = =x =0.
∗1 ∗2 ··· ∗p−1 ∗p+1 ∗p+2 ··· ∗p+1
Noticethatthereisonlyatmostonenon-integercomponentinx .Thisproperty
∗
will be used at the numerical calculations.
From the point of view of B&B the relation of the Problems (29.2), (24.2), and
(24.3) and (29.2), (24.2), and (24.4) is very important. Any feasible solution of the
first one is also feasible in the second one. But the opposite statement is not true.
In other words the set of feasible solutions of the first problem is a proper subset of
the feasible solutions of the second one. This fact has two important consequences:
The optimal value of the Problem (29.2), (24.2), and (24.4) is an upper bound
•
of the optimal value of the Problem (29.2), (24.2), and (24.3).
IftheoptimalsolutionoftheProblem(29.2),(24.2),and(24.4)isfeasibleinthe
•
Problem (29.2), (24.2), and (24.3) then it is the optimal solution of the latter
problem as well.
Thesepropertiesareusedinthecourseofthebranchandboundmethodintensively.