Table Of ContentState-Space Search
Springer Science+B usiness Media, LLC
Weix iong Zhang
State-Space Search
Algorithms, Complexity,
Extensions, and Applications
With 51 Figures
Springer
Weixiong Zhang
Information Sciences Institute and
Department of Computer Science
University of Southem California
Marina del Rey, CA 90292
USA
[email protected]
Library of Congress Cataloging-in-Publication Data
Zhang, Weixiong.
State space search : a1gorithms, complexity, and applications /
Weixiong Zhang.
p. cm.
Includes bibliographical references and index.
ISBN 978-1-4612-7183-3 ISBN 978-1-4612-1538-7 (eBook)
DOI 10.1007/978-1-4612-1538-7
1. State-space methods. 2. Combinatorial optimization.
3. Algorithms. 1. Title.
QA402.Z475 1999
519.7'6-dc21 99-24756
Printed on acid-free paper.
© 1999 Springer Science+Business Media New York
Originally published by Springer-Verlag New York, IDc. in 1999
Softcover reprint ofthe hardcover 1 st edition 1999
AlI rights reserved. This work may not be translated or copied in whole or in part without the
written permission ofthe publisher (Springer Science+Business Media New York), except for
brief excerpts in connection with reviews or scholarly analysis. Use in connection with any
form of information storage and retrieval, electronic adaptation, computer software, or by
similar or dissimilar methodology now known or hereafter developed is forbidden.
The use of general descriptive names, trade names, trademarks, etc., in this publication, even ifthe
former are not especia1ly identified, is not to be taken as a sign that such names, as understood by
the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.
Production managed by Robert Bruni; manufacturing supervised by Jacqui Ashri.
Typeset from the author's LaTeX files.
9 8 7 6 5 4 3 2 1
ISBN 978-1-4612-7183-3
To Hui, Kevin, Jay, and Elaine
Preface
This book is about problem solving. Specifically, it is about heuristic
state-space search under branch-and-bound framework for solving com
binatorial optimization problems. The two central themes of this book
are the average-case complexity of heuristic state-space search algorithms
based on branch-and-bound, and their applications to developing new
problem-solving methods and algorithms.
Heuristic state-space search is one of the fundamental problem-solving
techniques in Computer Science and Operations Research, and usually
constitutes an important component of most intelligent problem-solving
systems. The search algorithms considered in this book can be classified
into the category of branch-and-bound. Branch-and-bound is a general
problem-solving paradigm, and is one of the best techniques for optimally
solving computation-intensive problems, such as scheduling and planning.
The main search algorithms considered include best-first search, depth
first branch-and-bound, iterative deepening, recursive best-first search,
and space-bounded best-first search. Best-first search and depth-first
branch-and-bound are very well known and have been used extensively
in Computer Science and Operations Research. One important feature of
depth-first branch-and-bound is that it only requires space this is linear in
the maximal search depth, making it very often a favorable search algo
rithm over best-first search in practice. Iterative deepening and recursive
best-first search are the other two linear-space search algorithms. Iterative
deepening is an important algorithm in Artificial Intelligence, and plays an
irreplaceable role in building a real-time game-playing program. Recursive
best-first search expands new nodes in best-first order, but with a space
viii Preface
linear in search depth. It is also more efficient than iterative deepening
when the cost function used is monotonic. Space-bounded best-first search
is a generalized recursive best-first search, and is able to utilize whatever
memory available to avoid reexpanding a node if possible.
The purpose of algorithm analysis is twofold. The first is to better un
derstand the problem under consideration. To this end, this book presents
average-case complexity of heuristic state-space search algorithms, espe
cially those run in space that is linear in the maximal search depth. In
addition to the analysis of average-case complexity, it reveals the condi
tions under which problems are easy or difficult to solve on average. The
results shows that the difficult problem instances are clustered into one
region and the easy ones into the other. Furthermore, the transition from
the difficult problem region to the easy one is abrupt. This phenomenon is
similar to a phase transition of a physical system, such as water changing
from the solid phase, ice, to the liquid phase, water, when the temperature
rises.
The second purpose of algorithm analysis is to develop new, more efficient
problem-solving methods. Along this direction, this book demonstrates
three successful applications of the analytical results. They are state-space
transformation methods in principle, and can be used to find approximate
as well as optimal solutions quickly and to develop new anytime algorithms
for applications where computation resources are limited. The first is a
state-space transformation method developed from an observation on the
phase transitions of heuristic state-space search algorithms on an analytic
model. It simply transforms a difficult state space into an easy one and
finds the optimal goal in the simplified space as an approximation. The
second application is a forward pruning method in principle, and is thus
presented under the framework of forward pruning, which is an old tech
nique in heuristic search. The third application is a forward pruning method
for multiagent game-tree search. The key idea is to construct a more infor
mative heuristic evaluation function based on the static evaluation function
in game playing.
In addition, this book also specifically studies computational complex
ity of branch-and-bound in the asymmetric Traveling Salesman Problem,
an important NP-hard combinatorial optimization problem. Specifically,
it proves that branch-and-bound, using the assignment-problem cost func
tion, cannot find an optimal solution to the asymmetric Traveling Salesman
Problem in polynomial time of the problem size in an average case. This
settles a debate if the problem can be solved in polynomial or exponential
time. Nevertheless, it shows that depth-first branch-and-bound significantly
outperforms a local search algorithm on the asymmetric Traveling Salesman
Problem, and demonstrates that depth-first branch-and-bound is efficient,
anytime and approximation algorithms.
The book is structured as follows. Chapter 1 is an introduction to
problem solving and state-space search. It discusses a few combinatorial
Preface ix
problems, their state-space descriptions, how complex problems can be de
composed under the principle of inclusion and exclusion, and how they can
be solved by heuristic state-space search. Chapter 2 considers state space in
detail, and describes many state-space search algorithms, including best
first search, depth-first branch-and-bound, iterative deepening, recursive
best-first search, and space-bounded best-first search, and discusses their
properties. This chapter also discusses how a heuristic state-space search
algorithm under branch-and-bound paradigm can be tailored to an ap
proximation algorithm, and describes local search, which includes' many
general and efficient approximation algorithms for combinatorial optimiza
tion problems. The average-case analyses of state-space search algorithms
appear in Chapter 3. Chapter 4 studies complexity transitions in search
problems and examines the intrinsic problem features which cause complex
ity transitions. Chapter 5 then considers how to select a search algorithm
for a given problem, by taking into account the factors of running time
and memory requirement. Chapter 6 studies the complexity of branch
and-bound on the asymmetric Traveling Salesman Problem, proving that
branch-and-bound using the assignment-problem cost function cannot find
an optimal solution to the asymmetric Traveling Salesman Problem in
polynomial expected time, and showing that depth-first branch-and-bound
significantly outperforms local search. Chapter 7, Chapter 8 and Chapter 9
contain the applications of the analytical results of average-case analyses.
Chapter 7 presents a state-space transformation method that transforms
a complex state space into one that is easy to search. Chapter 8 considers
a forward pruning method for single-agent problem solving. Chapter 9 de
velops a new method to construct a more informative evaluation function
from a static evaluation function for game playing. These three chapters
also present experimental results from the applications of the methods in
solving realistic combinatorial optimization problems. To put the overall
discussion into perspective, each chapter makes a note on bibliography and
makes remarks on historical development of related topics.
The reader with a background of heuristic search or combinatorial search
techniques can skip Chapter 1 and Chapter 2. The reader who is interested
in average-case analysis can read Chapter 3 through Chapter 5. The reader
who is only interested in the main results of Chapter 3 can go directly to
Section 3.8. The reader who is interested in solving practical problems can
cover Chapter 4 through Chapter 9.
By their nature, the analyses of average-case complexity are presented
mathematically, using probability theory and recurrence relations. It is
assumed that the reader is familiar with the basic concepts of random vari
ables, distribution functions, and expectations, which are taught in a first
course in probability theory, and those of recursive equations, which are
covered in an undergraduate course in discrete mathematics. The average
case analyses make use of some results from the theory of branching
processes, which are described and summarized in Appendix A at the end
x Preface
of the book. In addition, in order to make algorithm descriptions easily
readable to someone without a lot of programming experience, I do not
usc a particular programming language for algorithm description. Instead,
I use an informal and proselike language.
This book is written with a vision of a broader audience, and should
be accessible to someone who is interested in Artificial Intelligence and/or
combinatorial optimization. I use terminology and examples from these
two areas. As it now stands, this book can serve as a textbook, a mono
graph, and a reference in Computer Science and Operations Research. It
can be used as a reference book for graduate courses on problem solving,
combinatorial optimization, and algorithm analysis.
My research interest in problem solving and search algorithms started in
the middle 1980s when I was a graduate student, studying and doing re
search on heuristic search and robot planning under Bo Zhang, at Tsinghua
University, Beijing, China. In 1985, I read Judea Pearl's book Heuristics:
Intelligent Search Stmtegies for Computer Problem Solving. I was amazed
by the beauty of the mathematical results presented in the book. At that
time, I even ambitiously tried to translate the book into Chinese, although I
never had a chance to complete that task. My continued interest in heuristic
search and combinatorial optimization finally led me to pursue my Ph.D.
at the University of California at Los Angeles (UCLA), where two of the
most active and productive researchers in the area of heuristic search, Judea
Pearl and Richard Korf, teach. This book contains the research results that
I obtained at UCLA and at the Information Sciences Institute, University
of Southern California (USC/lSI), some of which were previously published
in journals and conference proceedings. Specifically, Chapter 3 is based on
[153, 154, 156J, Chapter 4 uses materials from [155, 156, 157], Chapter 5 is
based on [156], Chapter 6 combines [148J and [150J, Chapter 7 draws upon
[158, 117], Chapter 8 is based on [151, 152], and Chapter 9 is from [149J.
I take great pleasure in closing this preface by acknowledging those who
helped me, and contributed to and/or influenced my research. First and
foremost, I would like to thank my Ph.D. thesis advisor, Richard Korf,
for his invaluable guidance and collaboration, especially during those four
years I spent at UCLA, and his friendship. Rich has been demanding, ask
ing difficult questions that have significantly improved my research. He has
been encouraging as well, cheering me up when I was stuck and pushing me
forward when I made progress. I am very grateful to Sheila Greibach and
Judea Pearl. Sheila's comments and the opportunities that she provided for
me to present some of my results at her theoretical computer science semi
nar at UCLA clarified and sharpened many of my ideas and results. Judea
explained to me his previous work, upon which one of the major results of
this book is based. My research greatly benefited from many discussions
with him. My special thanks go to my former officemate Joseph Pember
ton for his collaboration and major contributions to the approximation
method presented in Chapter 7. Peter Cheeseman, Tad Hogg, Colin Mc-
Preface xi
Diarmid, and Greg Provan provided many discussions on heuristic search
and phase transitions. My research has also benefited from discussions with
and comments from many other researchers. At the risk of missing some
of them, they are, in alphabetical order, Liz Borowsky, Eli Gafni, Lars
Hagen, David Johnson, Richard Karp, Andrew Kahng, Colin McDiarmid,
Donald Miller, Dick Muntz, Dana Nau, Bruno Repetto and Colin Williams.
I thank Kai-Fu Lee for the static evaluation function of his Othello pro
gram Bill. I am grateful to Sheila Coyazo for proofreading many of my
papers. I also thank my editors at Springer-Verlag for providing the oppor
tunity to publish my research results. Thanks to Dr. Martin Gilchrist, who
started this project, to Dr. Bill Sanders, who oversaw it to the end, and to
my production editor Mr. Robert Bruni and his staff, who provided valu
able assistance in preparing the manuscript. My research was supported in
part by three National Science Foundation grants (No. IRI-8552925, No.
IRI-9119825, and No. IRI-9619554), a grant from Rockwell International,
a fellowship from the Graduate Division of UCLA (1990-93), a fellowship
from GTE Corporation (1992-93), and the UCLA Chancellor's Disserta
tion Fellowship (1993-94). Finally and most importantly, I am indebted
to my family and particularly, my wife, Hui Zhou. Without her love and
support, it would not have been possible for me to finish this book. She has
sheltered me from many duties that I should have carried, especially taking
care of Kevin, Jay and Elaine. I wish that I could have spent with them
those countless evening hours and weekends that I worked on my research.
I dedicate this book to them. I thank my parents and parents-in-law for
their continued support.
Marina del Rey, California, U.S.A. Weixiong Zhang
