Combinatorial Data Analysis: Optimization by Dynamic Programming (Monographs on Discrete Mathematics and Applications)

176 Pages·1987·16.37 MB·English

by  Lawrence Hubert

COMBINATORIAL
DATA ANALYSIS
SIAM Monographs on
Discrete Mathematics and Applications
The series includes advanced monographs reporting on the most recent theoretical, computational, or
applied developments in the field; introductory volumes aimed at mathematicians and other mathematically
motivated readers interested in understanding certain areas of pure or applied combinatorics; and graduate
textbooks. The volumes are devoted to various areas of discrete mathematics and its applications.
Mathematicians, computer scientists, operations researchers, computationally oriented natural and social
scientists, engineers, medical researchers, and other practitioners will find the volumes of interest.
Editor-in-Chief
Peter L. Hammer, RUTCOR, Rutgers, The State University of New Jersey
Editorial Board
M. Aigner, Freie Universitat Berlin, Germany A. V. Kostochka, Siberian Branch of the Russian Academy of
N. Alon, Tel Aviv University, Israel Sciences, Russia
E. Balas, Carnegie Mellon University, USA F. T. Leighton, Massachusetts Institute of Technology, USA
C. Berge, E. R. Combinatoire, France T. Lengauer, Gesellschaft fur Mathematik und
J.- C. Bermond, Universite de Nice-Sophia Antipolis, France Datenverarbeitung mbH, Germany
J. Berstel, Universite Marne-la-Vallee, France S. Martello, DEIS University of Bologna, Italy
N. L. Biggs, The London School of Economics, United Kingdom M. Minoux, Universite Pierre et Marie Curie, France
B. Bollobas, University of Memphis, USA R. Mohring, Technische Universitat Berlin, Germany
R. E. Burkard, Technische Universitat Graz, Austria C. L. Monma, Bellcore, USA
D. G. Cornell, University of Toronto, Canada J. Nesetril, Charles University, Czech Republic
I. Gessel, Brandeis University, USA W. R. Pulleyblank, IBM T. J. Watson Research Center, USA
F. Glover, University of Colorado, USA A. Recski, Technical University of Budapest, Hungary
M. C. Golumbic, Bar-llan University, Israel C. C. Ribeiro, Catholic University of Rio de Janeiro, Brazil
R. L. Graham, AT&T Research, USA H. Sachs, Technische Universitat llmenau, Germany
A. J. Hoffman, IBM T. J. Watson Research Center, USA A. Schrijver, CWI, The Netherlands
I Ibaraki, Kyoto University, Japan R. Shamir, Tel Aviv University, Israel
H. Imai, University of Tokyo, Japan N. J. A. Sloane, AT&T Research, USA
M. Karoriski, Adam Mickiewicz University, Poland, and Emory W. T. Trotter, Arizona State University, USA
University, USA D. J. A. Welsh, University of Oxford, United Kingdom
R. M. Karp, University of Washington, USA D. de Werra, Ecole Polytechnique Federate de Lausanne,
V. Klee, University of Washington, USA Switzerland
K. M. Koh, National University of Singapore, Republic of P. M. Winkler, Bell Labs, Lucent Technologies, USA
Singapore Yue Minyi, Academia Sinica, People's Republic of China
B. Korte, Universitat Bonn, Germany
Series Volumes
Hubert, L., Arabie, P., and Meulman, J., Combinatorial Data Analysis: Optimization by Dynamic
Programming
Peleg, D., Distributed Computing: A Locality-Sensitive Approach
Wegener, I., Branching Programs and Binary Decision Diagrams: Theory and Applications
Brandstadt, A., Le, V. B., and Spinrad, J. P., Graph Classes: A Survey
McKee, T. A. and McMorris, F. R., Topics in Intersection Graph Theory
Grilli di Cortona, P., Manzi, C., Pennisi, A., Ricca, R, and Simeone, B., Evaluation and Optimization
of Electoral Systems
COMBINATORIAL
DATA ANALYSIS
Optimization by
Dynamic Programming
Lawrence Hubert
University of Illinois Urbana-Champaign
Champaign, Illinois
Phipps Arable
Rutgers University
Newark, New Jersey
Jacqueline Meulman
Leiden University
Leiden, The Netherlands
51HJTL
Society for Industrial and Applied Mathematics
Philadelphia
Copyright ©2001 by the Society for Industrial and Applied Mathematics.
10 9 8 7 6 5 4 3 21
All rights reserved. Printed in the United States of America. No part of this book may
be reproduced, stored, or transmitted in any manner without the written permission of the
publisher. For information, write to the Society for Industrial and Applied Mathematics,
3600 University City Science Center, Philadelphia, PA 19104-2688.
Library of Congress Cataloging-in-Publication Data
Hubert, Lawrence J.
Combinatorial data analysis : optimization by dynamic programming / Lawrence
Hubert, Phipps Arabic, Jacqueline Meulman.
p. cm. - (SIAM monographs on discrete mathematics and applications)
Includes bibliographical references and index.
ISBN 0-89871-478-8
1. Dynamic programming. 2. Combinatorial optimization. I. Arabic, Phipps.
II. Meulman, Jacqueline. III. Title. IV. Series.
T57.83 .H83 2001
519.7'03~dc21
00-049656
Siam
is a registered trademark.
To J. Douglas Carroll
fabri meliori rationum viarumque
This page intentionally left blank
Contents
Preface ix
1 Introduction 1
2 General Dynamic Programming Paradigm 7
2.1 An Introductory Example: Linear Assignment 7
2.2 The GDPP 13
3 Cluster Analysis 17
3.1 Partitioning 17
3.1.1 Admissibility Restrictions on Partitions 21
3.1.2 Partitioning Based on Two-Mode Proximity
Matrices 25
3.2 Hierarchical Clustering 28
3.2.1 Hierarchical Clustering and the Optimal Fitting
of Ultrametrics 36
3.2.2 Constrained Hierarchical Clustering 45
4 Object Sequencing and Seriation 51
4.1 Optimal Sequencing of a Single Object Set 53
4.1.1 Symmetric One-Mode Proximity Matrices 54
4.1.2 Skew-Symmetric One-Mode Proximity Matrices 62
4.1.3 Two-Mode Proximity Matrices 69
4.1.4 Object Sequencing for Symmetric One-Mode
Proximity Matrices Based on the Construction
of Optimal Paths 73
4.2 Sequencing an Object Set Subject to Precedence Constraints . . 78
4.3 Construction of Optimal Ordered Partitions 83
5 Heuristic Applications of the GDPP 91
5.1 Cluster Analysis 92
5.2 Object Sequencing and Seriation 97
6 Extensions and Generalizations 103
6.1 Introduction 103
vii
viii CONTENTS
6.1.1 Multiple Data Sources 104
6.1.2 Multiple Structures 107
6.1.3 Uses for the Information in the Sets fii,..., fik 108 108
6.1.4 A Priori Weights for Objects and/or Proximities 11
6.2 Prospects 113
Appendix: Available Programs 115
Bibliography 149
Author Index 157
Subject Index 159
Preface
The first part of this monograph's title, Combinatorial Data Analysis (CDA),
refers to a wide class of methods for the study of relevant data sets in which
the arrangement of a collection of objects is absolutely central. Characteristi-
cally, CDA is involved either with the identification of arrangements that are
optimal for a specific representation of a given data set (usually operationalized
with some specific loss or merit function that guides a combinatorial search de-
fined over a domain constructed from the constraints imposed by the particular
representation selected), or with the determination in a confirmatory manner
of whether a specific object arrangement given a priori reflects the observed
data. As the second part of the title, Optimization by Dynamic Programming,
suggests, the sole focus of this monograph is on the identification of arrange-
ments; it is then restricted further, to where the combinatorial search is carried
out by a recursive optimization process based on the general principles of dy-
namic programming. For an introduction to confirmatory CDA without any
type of optimization component, the reader is referred to the monograph by
Hubert (1987). For the use of combinatorial optimization strategies other than
dynamic programming for some (clustering) problems in CDA, the recent com-
prehensive review by Hansen and Jaumard (1997) provides a particularly good
introduction.
Our purpose in writing this monograph is to provide an applied documen-
tation source, as well as an introduction to a collection of associated computer
programs that would be of interest to applied statisticians and data analysts
but also accessible to a notationally sophisticated but otherwise substantively
focused user. Such a person would typically be most interested in analyzing a
specific data set by implementing the flexible dynamic programming method for
any of a number of seemingly diverse problems encountered in CDA. The back-
ground we have tried to assume is at the same level as that required to follow
the documentation for good, commercially available optimization subroutines,
such as the Numerical Algorithms Group (NAG) Fortran subroutine library, or
at the level of one of the standard texts in applied multivariate analysis usually
used for a graduate second-year methodology course in the behavioral or social
sciences. An excellent example of the latter would be the widely used text now
in its fourth edition by Johnson and Wichern (1998). Draft versions of the cur-
ix