Table Of ContentCLASSIFICATION
AND CLUSTERING
Edited by J. Van Ryzin
Proceedings of an Advanced Seminar
Conducted by the Mathematics Research Center
The University of Wisconsin at Madison
May 3-5, 1976
Academic Press, Inc.
New York · San Francisco · London 1977
A Subsidiary of Harcourt Brace Jovanovich, Publishers
COPYRIGHT © 1977, BY ACADEMIC PRESS, INC.
ALL RIGHTS RESERVED.
NO PART OF THIS PUBLICATION MAY BE REPRODUCED OR
TRANSMITTED IN ANY FORM OR BY ANY MEANS, ELECTRONIC
OR MECHANICAL, INCLUDING PHOTOCOPY, RECORDING, OR ANY
INFORMATION STORAGE AND RETRIEVAL SYSTEM, WITHOUT
PERMISSION IN WRITING FROM THE PUBLISHER.
ACADEMIC PRESS, INC.
Ill Fifth Avenue, New York, New York 10003
United Kingdom Edition published by
ACADEMIC PRESS, INC. (LONDON) LTD.
24/28 Oval Road, London NW1
Library of Congress Cataloging in Publication Data
Main entry under title :
Classification and clustering.
(Publication of the Mathematics Research
Center, University of Wisconsin-Madison ; no. 37)
Includes index.
1. Discriminant analysis-Congresses.
2. Cluster analysis-Congresses. I. Van Ryzin,
John. II. Wisconsin. University-Madison.
Mathematics Research Center. III. Series:
Wisconsin. University-Madison. Mathematics
Research Center. Publication ; no. 37.
QA3.U45 no. 37 [QA278.65] 510'.8s [519.5'3]
77-7139
ISBN 0-12-714250-9
PRINTED IN THE UNITED STATES OF AMERICA
List of Contributors
Susan W. Ahmed, Department of Biostatistics, The University of North Carolina at
Chapel Hill, Chapel Hill, North Carolina 27514
Frank B. Baker, Department of Statistics, The University of Wisconsin at Madi
son, Madison, Wisconsin 53706
Pi Yeong Chi, National Institute of Environmental Health Sciences, Research
Triangle Park, North Carolina 27709
Jerome Cornfield, Department of Statistics, George Washington University,
Washington, D.C. 20006
Rosalie A. Dunn, Biostatistics Section, The Veterans Administration Research
Center, Washington, D.C. 20422
K. S. Fu, Department of Electrical Engineering, Purdue University, West
Lafayette, Indiana 47907
Seymour Geisser, Director, School of Statistics, University of Minnesota, Min
neapolis, Minnesota 55455
I. J. Good, Department of Statistics, Virginia Polytechnic Institute and State Uni
versity, Blacksburg, Virginia 24061
J. A. Hartigan, Department of Statistics, Yale University, New Haven, Connec
ticut 06520
Lawrence J. Hubert, Department of Statistics, The University of Wisconsin at
Madison, Madison, Wisconsin 53706
Raul Hudlet, IIMAS, Universidad Nacional Autonoma de Mexico, Apartado
Postal 20-726, Mexico, 20 DF
Richard Johnson, Department of Statistics, The University of Wisconsin at Madi
son, Madison, Wisconsin 53706
Joseph Kruskal, Mathematics and Statistics Research Center, 600 Mountain Av
enue, Bell Telephone Laboratories, Murray Hill, New Jersey 07974
Peter A. Lachenbruch, School of Public Health, Department of Biostatistics, The
University of North Carolina at Chapel Hill, Chapel Hill, North Carolina 27514
David W. Matula, Department of Computer Science, Southern Methodist Univer
sity, Institute of Technology, Dallas, Texas 75275
Hubert V. Pipberger, Department of Medicine, The Veterans Administration Re
search Center, Washington, D.C. 20422
vii
vili LIST OF CONTRIBUTORS
C. Radhakrishna Rao, Indian Statistical Institute, 7, S.J.S. Sansanwal Marg,
New Delhi 110029, India
Robert R. Sokal, Department of Ecology and Evolution, State University of New
York, Stony Brook, New York 11790
Herbert Solomon, Department of Statistics, Stanford University, Stanford, Califor
nia 94305
J. Van Ryzin, Department of Statistics, The University of Wisconsin at Madison,
Madison, Wisconsin 53706
Grace Wahba, Department of Statistics, The University of Wisconsin at Madison,
Madison, Wisconsin 53706
L. A. Zadeh, Department of Electrical Engineering, University of California at
Berkeley, Berkeley, California 94720
Preface
This volume contains the Proceedings of the Advanced Seminar on Classification
and Clustering held in Madison, Wisconsin, May 3-5, 1976, sponsored by the
Mathematics Research Center, University of Wisconsin at Madison, with finan
cial support from the National Science Foundation under grant No. MCS 75 - 21351
and the United States Army under Contract No. DAAG29-75-C-0024.
At the Advanced Seminar thirteen invited speakers presented papers, twelve of
which appear in these proceedings. We very much regret that it has not been
possible to include the manuscript of Professor Louis Guttman's excellent talk on
4'Categorical and Monotone Data Analysis." Nevertheless, the contribution of his
talk and his lively discussion is gratefully acknowledged. In addition, this volume
contains four papers by authors here at the University of Wisconsin — Madison
who gave talks in an evening session consisting of twelve short presentations on
research work in classification and clustering.
The six sessions at the Advanced Seminar were chaired by:
John Van Ness, University of Texas at Dallas, Dallas, Texas;
Robert V. Hogg, University of Iowa, Iowa City, Iowa;
P. R. Krishnaiah, Wright-Patterson Air Force Base, Dayton, Ohio;
Ingram Olkin, Stanford University, Stanford, California;
Douglas Tang, Walter Reed Army Institute of Research, Washington, D.C.;
Herman Chernoff, Massachusetts Institute of Technology, Cambridge, Mas
sachusetts.
These people not only did an excellent job of chairing the sessions but all contri
buted much to the discussion.
The members of the program committee were Bernard Harris, Lawrence
Hubert, Richard Johnson, and the editor as Program Chairman, all of the Univer
sity of Wisconsin at Madison. Mrs. Gladys Moran, Program Secretary, handled
all of the organizational details in a most efficient and pleasant way. Mrs. Dorothy
Bowar accomplished with cheer and care the difficult task of typing the manuscripts
and corresponding with the authors relative to corrections and like matters. Finally,
thanks are extended to Ben Noble, Director of the Mathematics Research Center for
his continual support of this effort, to J. Michael Yohe, Associate Director, for help
IX
x PREFACE
in the financial planning, and to Dean E. David Cronon, College of Letters and
Science, for his welcoming address on those sunny pleasant May days.
The advanced seminar's attendance numbered more than two-hundred and sev
enty-five — far exceeding expectations. This in itself was justification enough for
the effort involved by all concerned. It also is a great testimony to the importance of
the research problems being attacked by the scientists in the field of classificaiton
and clustering, be they mathematicians, biological scientists, social scientists,
computer scientists, statisticians, or engineers. All these disciplines to one degree or
another were represented in the attendees and speakers. It is only hoped that in some
small way this volume can contribute to the diversity and advancement of the twin
fields of classification and clustering. We thank all those who have participated.
J. Van Ryzin
Clustering and Classification:
Background and Current Directions
Robert R. Sokal
In this paper I will provide a general background into the nature
and purpose of clustering and classification. Coming as it does in the
middle of the symposium, some of what I have to say has already been
alluded to by other speakers. Much of what I propose to say will be
familiar to at least some of you who have been intimately concerned with
classification problems for a considerable time. Yet I hope there is some
value in my attempt to describe the field as I see it and to delineate the
directions that I believe it should take.
It is frequently stated that classification is one of the funda
mental processes in science. Facts and phenomena must be ordered be
fore we can understand them and to develop unifying principles explaining
their occurrence and apparent order. From this point of view, classifi
cation is a higher level intellectual activity necessary to our understand
ing of nature. But since classification is the ordering'of objects by their
similarities (Sneath and Sokal, 197 3) and objects can be conceived of in
the widest sense including processes and activities -- anything to
which a vector of descriptors can be attached, we recognize that classi
fication transcends human intellectual endeavor and is indeed a
fundamental property of living organisms. Unless they are able to group
stimuli into like kinds so as to establish classes to which favorable or
avoidance reactions can be made, organisms would be ill-adapted for
survival. Such reactions can be learned (among organisms capable of
learning), or instinctive, programmed into the genome of the population
through natural selection. So what we taxonomists do is a very natural
1
2 ROBERT R. SOKAL
activity and the procedures of cluster analysts for setting up classes of
similar objects merely systematize and quantify one of the most funda
mental processes of humans and other living organisms.
• Attempts to develop techniques for automatic classification,
necessitated the quantification of similarity. The ability to perceive
any two objects as more similar to each other than either is to a third,
must surely have been present in the ancestors of the human species.
Many animal choice experiments involve the very same principle. When
a bird, for example, is given a choice between two different models,
either as mates or threat objects, the bird's nervous system must be
making a judgement of relative similarity of either model to a learned
pattern residing in its nervous system. One can quantify estimates of
resemblances by humans in distinct ways. One can ask a single person
to make an overall similarity judgement between objects and to express
this as a number from 0 to 1 or in terms of inequalities for pairs or
triplets of objects. Social scientists frequently follow this approach,
but unless the perception of only one person is the object of study, a
single table of resemblances as quantified by any one individual is suf
ficient. Customarily, a sample of subjects is asked to make the esti
mates of resemblance, and averages of these resemblances are used to
obtain a classification of the objects.
In much classifica tory work, it would be impractical to obtain
estimates of taxonomic similarity in an assemblage of objects from a
sample of subjects. Furthermore, scientific practice generally eschews
judgement based on majority vote or popularity. So a second approach
to the quantification of estimates of resemblance is through attempts to
find the basis for similarity judgements. This is usually done by the
detailed description of characteristics on the basis of which it is be
lieved that similarities are expressed. This approach has led to a partic-
ularization and atomization of the descriptors of the objects to be clas
sified. Long lists of descriptors, vectors of character states, are at
tached to each object and the classification is carried out on a data
BACKGROUND AND CURRENT DIRECTIONS 3
matrix, composed of a collection of such vectors. Important theoretical
issues turn on the nature of the fundamental unit characters but since
these issues differ with the field of application, I shall not go into them
here.
The consequence of such procedures is that the objects to be
classified, -- operational taxonomic units (OTU 1 s for short), are repre
sented in a space whose dimensions are the characters. This attribute
space (A-space) is formally of n_ dimensions (for n characters), but
because of correlations of characters across the OTU 1 s can usually be
reduced to fewer dimensions with little loss of information. Such re
duction of dimensionality is carried out by ordination methods discussed
below.
The comparative study of objects in A-space across a variety of
disciplines will rapidly convince the observer that only in unusual cir
cumstances are objects aggregated in compact regions of the full attri
bute hyperspace. Although aggregated with respect to many of the dimen
sions of the space, OTU's from a given cluster may well be distributed
along the entire range of some character dimensions. Different OTU's
will deviate from the cluster along different character axes. What this
means is that classes of objects can be defined without resorting to
uniformity of position of the objects on all character axes or even on any
given character axis. While any one OTU must resemble all others of its
cluster in most of its characteristics, it need not resemble them in all
characters. Class membership is thus defined by "majority vote" (the
greatest number of shared character states) and no one characteristic
becomes defining for membership in a given class or taxon. Such rela
tionships have been found empirically by researchers investigating sup
posedly natural classifications in a variety of disciplines and it has been
formalized as a principle of classification by Beckner (19 59). Classes
defined in this manner are called polythetic (Sneath, 1962) and most
methods of cluster analysis and classification aim at obtaining polythetic
classes. In the converse system, monothetic classes or taxa,
4 ROBERT R. SOKAL
membership is defined by common values for all or at least some charac
ters. This makes monothetic classifications useful for constructing
taxonomic keys, but the resulting arrangement of natural objects is fre
quently unsatisfactory.
Acceptance of polythetic classification has important conse
quences. Membership in a taxonomic class is a function of the distribu
tion of the OTU's in A-space. It is not a two-valued function such as
belonging or not belonging, as is the case in monothetic classifications,
but belongingness is determined along a continuous scale. In polythetic
classifications taxonomic structure becomes a statistical rather than a
purely geometric problem.
It follows from what has been said so far that quantitative poly
thetic classifications require many characters for a correct grouping of
the objects. Such classifications are frequently called natural; their
members will be in some sense more similar to each other than they are
to members of other taxa. Classifications based on a great variety of
characters will likely be of general utility while those based on only a
few characters should be less useful generally, except for the special
purposes relevant to the chosen characters. For purposes of general
classification and for determining the natural order among objects or
phenomena, a natural classification based on all available information
(characters) is clearly preferable. For special purposes, where only
some properties of the objects are of interest to the investigator or where
the classification is to serve the needs of some special practical appli
cation, a special classification is indicated. Using few rather than many
characters in effect weights the characters employed by unity and those
omitted by zero and brings up the general subject of weighting which has
been a very controversial one in taxonomy. I shall not discuss this at
length since it has been treated adequately elsewhere (Sneath and Sokal,
197 3). Suffice it to state that while weighting of characters for identifi
cation purposes (allocation of known OTU's to predetermined classes) is
an accepted part of taxonomic procedure, no consistent scheme for
