Table Of ContentLecture Notes ni
Computer Science
detidE yb .G Goos dna .J Hartmanis
24
II II
Parallel Processing
sgnideecorP of eht eromagaS
August Computer Conference, 20-23, 1974
detidE yb Tse-yun gneF
galreV-regnirpS
Berlin. Heidelberg. New York 91 75
Editorial Board" .P Brinch Hansen • D. Gries
C. Moler • G. Seegm~ller • N. Wirth
Dr. Tse-yun Feng
Syracuse University
Department of Electrical
and Computer Engineering
Link Hall
Syracuse, NY 13210/USA
Library of Congress Cataloging in Publication Data
Sagamore Computer Conference 3d~ s Raquette Lake~ N. Y.~
1974,
Parallel processing.
(Lecture notes in computer science 24) ;
Bibliography: p,
Includes index.
I. Parallel processing (Electron~c computers)--Con-
gresses. I. Feng~ Tse-yun~ 1928- ed. II. T~tle,
!II, Series.
QA76,6,$23 1974 001,6'4 74-34429
AMS Subject Classifications (1970): 00A10
CR Subject Classifications (1974): 2.2
ISBN 3-540-07135-0 Springer-Verlag Berlin • Heidelberg • New York
ISBN 0-387-07135-0 Springer-Verlag New York. Heidelberg • Berlin
This work is subject to copyright. All rights are reserved, whether the whole
or part of the material is concerned, specifically those of translation,
reprinting, re-use of illustrations, broadcasting, reproduction by photo-
copying machine or similar means, and storage in data banks.
Under § 54 of the German Copyright Law where copies are made for other
than private use, a fee is payable to the publisher, the amount of the fee to
be determined by agreement with the publisher.
© by Springer-Verlag Berlin • Heidelberg 1975. Printed in Germany.
Offsetdruck: Julius Beltz, Hemsbach/Bergstr.
ECAFERP
The Sagamore Computer Conference has been held annually for the past three
years at the former Vanderbilt summer estate, a 1300-acre preserve surrounding the
private Sagamore Lake, in the Central Adirondack Mountains. The first Sagamore
Computer Conference was held on August 23 - 25, 1972. The subject of that confer-
ence was on "RADCAP (Rome Air Development Center Associative Processor) and Its
Applications". About 90 invited participants attended the 2-day conference to hear
the 17 technical papers presented. The Conference Proceedings were published and
distributed by RADC and Syracuse University, co-sponsors of the Conference. In
1973, the Conference broadened its scope to "Parallel Processing" and issued the
Call-for-Papers announcement. Among the submitted papers, 34 were accepted and
presented on August 22 - 24, 1973. The 1973 Conference was sponsored by Syracuse
University in cooperation with IEEE and ACM. Copies of its Proceedings (IEEE
Catalog Number 73 CH0812-8 )C may be available from any one of these institutions.
This year's conference was extended to three days (August 20 - ,32 1974) to
provide the participants with more time for individual activities. One new feature
added to the conference was the Keynote Address by Col. William Whitaker of the
Office of Director of Defense Research and Engineering. We thank Col. Whitaker for
his informative remarks. A panel session was organized this year to review the
state-of-the-art in parallel processing. We would also llke to thank our dis-
tinguished panelists: Drs. .J .B Dennis, Robert Downs, David Kuck, Richard Merwin,
.C V. Ramamoorthy, and .S .S Yau, for sharing their experience and knowledge with
us.
We appreciate the efforts of the authors who submitted papers for consideration.
A special acknowledgement is due to the reviewers who evaluated the papers with
exceptional attention and punctuality. The success of the Conference is also
attributable to the generous assistance we received from the session chairpersons
and session organizers. Finally, we would also like to express our appreciation to
Diane Sims, Anne Woods, Hazel Laidlaw, Louise Naylor, and Gladys Stlth for their
administrative assistance and typing help.
The next Sagamore Computer Conference on Parallel Processing is to be held on
August 19 - 22, 1975.
Tse-yun Peng
Department of Electrical & Computer Engineering
Syracuse University
ELBAT FO STNETNOC
KEYNOTE ADDRESS
Trends in Defense Computation
Lt. Col. WilliamWhltaker
Office of Director of Defense Research and Engineering
SESSION i: SCHEDULING i
Chairperson: Professor S. Schindler, Technisehe Universitat Berlin
On a Class of Scheduling Algorithms for Multiprocessors
Computing Systems
N. F. Chen, C. L. Liu . . . . . . . . . . . . . . . . . . . . .
Scheduling Unft~Time Tasks with Limited Resources
A. C.-C. Yao . . . . . . . . . . . . . . . . . . . . . . . . . . 71
SESSION 2: ASSOCIATIVE PROCESSORS
Chairperson: Captain R. W. Johnson, Rome Air Development Center
Programmable Radar Signal Processing Using the Rap
G. R. Couranz, M. Gerhardt, C. Young .............. 73
Analysis and Design of a Cost-Effective Associative
Processor for Weather Computations
W.-T. Cheng, T. Feng . . . . . . . . . . . . . . . . . . . . . . 35
The Implementation of APL on an Associative Processor
M. J. Harrison, W. H. Harrison ................. 57
A Unified Associative and von-Neumann Processor
EGPP and EGPP Array
W. Handler . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Discriminating Content Addressahle Memories
C. Smith, L. Wittie . . . . . . . . . . . . . . . . . . . . . . i00
SESSION 3: MODELLING AND PARALLELIS M DETECTION
Chairperson: Professor G. H. Foster, Syracuse University
A Fundamental Theorem of Asynchronous Parallel Computation
R. M. Keller . . . . . . . . . . . . . . . . . . . . . . . . . . 102
The Hyperplane Method for an Array Computer
L. Lamport . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Syntactic Recognition of Parallel Processes in Formally
Defined Complexes of Interacting Digital Systems
P. Z. Smith, D. R. Fitzwater , ................. 132
SESSION 4: SYSTEM CONTROL
Chairperson: Dr. K. E. Batcher, Goodyear Aerospace Corporation
A Control Structure for Parallel Processing
S. K. Shastry . . . . . . . . . . . . . . . . . . . . . . . . . 134
Toward the Design of a Network Manager for a Distributed
Computer Network
J. H. Rowan, D. A. Smith, M. D. Swensen ............. 148
V
The Typeset-lO Message Exchange Facility -- A Case
Study in Systemic Design
M. J. Spier, R. L. Hill, T. J. Stein, D. Bricklin ........ 167
SESSION 5: SCHEDULING II
Chairperson~ Professor H. J. Gonzalez, Northwestern University
Optimal Resource Allocation and Scheduling Among
Parallel Processes
A. Lew ............ . ............. , . . 187
A Recognizer and Post-Recognizer for Optimizing
Execution Times of Programs
P. F. Reynolds, K. M. Chandy . . . . . . . . . . . . . . . . . . 203
Analytic and Implementation Considerations of Two-
Facility Sequencing in Computer Systems
.S S. Reddi . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
Computer Models with Constrained Parallel Processors
T. W. Keller, K. M. Chandy . . . . . . . . . . . . . . . . . . . 207
SESSION 6: RADCAP - THE RADC ASSOCIATIVE PROCESSOR
Chairperson: Mr. J. L. Previte, Rome Air Development Center
Implementation of Data Manipulating Functions on
the STARAN Associative Processor
L. H. Bauer . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
Mixed Mode Arithmetic for STARAN
E. P. Stabler . . . . . . . . . . . . . . . . . . . . . . . . . . 228
AAPL: An Array Processing Language
J. G. Marzolf . . . . . . . . . . . . . . . . . . . . . . . . . . 230
The Evolution of a Parallel Active Tracking Program
M. W. Summers, D. F. Trad . . . . . . . . . . . . . . . . . . . . 238
Implementation of the AWACS Passive Tracking Algorithms
on a Goodyear STARAN
B. W. Prentice . . . . . . . . . . . . . . . . . . . . . . . . . 250
Experiences with an OperationalAssociative Processor
D. L. Baldauf . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
SESSION :7 PANEL: PARALLEL PROCESSING -- PAST~ PRESENT,
AND FUTURE
Chairperson: Professor T. Feng, Syracuse University
Panelists: J. B. Dennis, MIT
Robert Downs, Systems Control
David Kuck, University of llllnois
Richard E. Merwin, Safeguard System Office
C. V. Ramamoorthy, University of California,
Berkeley
Stephen S. Yau, Northwestern University
SESSION 8: COMPUTATION ALGORITHMS
Chairperson: Professor A. Sameh, University of Illinois
Matrix Computations on an Associative Processor
P. A. Gilmore . . . . . . . . . . . . . . . . . . . . . . . . . . 272
lV
Optimal Searching Algorithms for Parallel Pipelined Computers
D. L. Weller, E. .S Davidson .................. 291
A New Parallel Algorithm for Network Flow Problems
I.-N. Chen ........................... 306
SESSION :9 SYSTEM ARCHITECTURE AND COMPONENT DESIGN
Chairperson: Professor O. N. Garcla, Unlversity of South Florida
Parallel Processing by Virtual Instruction
.C H. Kaplinsky ......................... 308
An Approach to Restructurable Computer Systems
.S .S Reddi, E. A. Feustel ................... 319
On Programmable Parallel Data Routing Networks via
Cross-Bar Switches for Multiple Element Computer
Architectures
.C .J Chen, A. A. Frank ..................... 338
A Reconfigurable Parallel Arithmetic Unit
.C P. Hsu, T. Feng ....................... 370
SESSION i0: PARALLEL PROCESSOR ARCHITECTURES FOR AIR TRAFFIC CONTROL
Chairperson: Mrs. V. J. Hobbs, Department of Transportation
Architectural Considerations in Interfacing a Parallel
Processor to the Air Traffic Control System
W. L. Heimerdinger, G. .F Muething, .S J. Nuspl
.L .B Wing ........................... 372
An Efficient Implementation of Conflict Prediction
in a Parallel Processor
H. .G Sehmitz, C.-C. Huang ................... 383
An Associative Processor Architecture for Air
Traffic Control
H. N. Boyd ............. .............. 400
Application of an Associative Processor to
Aircraft Tracking
.E .E Eddey, W. .C Meilander .................. 417
Analysis of Parallel Processing for Air Defense
and Air Traffic Control for Thailand
.C Messajjee, W. R. Adrion ................... 429
AUTHOR INDEX ............................. 431
REVIEWERS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432
ON A CLASS OF SCHEDULING ALGORITHMS
FOR MULTIPROCESSORS COMPUTING SYSTEMS )a(
N. F. CHEN
Department of Mathematics
C. L. LIU
Department of Computer Science
University of Illinois
Urbana, Illinois 61801
Abstract: -- In this paper we study a class of scheduling algorithms for multi-
processors computing systems which we call level algorithms. These algorithms are
simple and easy to implement. They produce optimal schedules in some cases, and
produce close-to-optimal schedules in other cases. Sufficient conditions under
which schedules produced by the level algorithms will be optimal are obtained. For
the sub-optimal schedules, we are able to compare quantitatively their performance
against that of an optimal schedule.
INTRODUCTION
Recent progress in hardware technology and computer architecture has led to the
design and construction of computer systems that contain a large number of proces-
sors. Because of their capability of executing several tasks simultaneously, the
problem of job scheduling in multiprocessor computing systems is of both theoretical
and practical interest. Since, for a finite set of tasks, there is only a finite
number of possible schedules, the problem of determining a best schedule could in
principle be solved by an exhaustive examination of all the possibilities. However,
such an exhaustive search will require considerable computation time and will offset
the advantage gained by using an optimal schedule. As was proposed by J. Edmonds
[2], a scheduling algorithm is considered to be efficient if the number of steps in
the algorithm is a polynomial function of the number of tasks to be scheduled.
Several authors have designed efficient algorithms to produce schedules which mini-
mize the total execution time and thus achieve optimal utilization of the processors
[i], [3], [5]. Unfortunately, these algorithms are only applicable to some special
cases. As a matter of fact, efficient algorithms that produce optimal schedules are
known only for the following cases:
(i) All jobs have equal execution time and the partial ordering over the jobs
is such that either every job has at most one successor or every job has at most one
predecessor[5].
(ii) All jobs have equal execution time and there are only 2 processors in the
computing systems [i], [3].
(a)Research supported by NSF GJ-41538.
In view of the difficulty, simple algorithms that produce only sub-optimal
schedules are often used in practice. Such a choice becomes even more appealing
when the performance, in terms of the total execution time, of these algorithms can
be compared quantitatively with that of algorithms producing optimal schedules. In-
deed, to study the performance of a class of simple algorithms is the theme of this
paper.
DESCRIPTION OF THE SYSTEM
We consider the problem of scheduling a set of jobs on a multiprocessor com-
puting system that has a set of identical processors capable of independent opera-
tion on independent jobs. Let PI,P2,...,Pn denote the n identical processors in a
multi-processor computing system. Let P = {Ji,J2,...,Jr} denote a set of jobs to be
executed on the computing system. We assume that the execution of a job occupies
one and only one processor. Moreover, since the processors are identical, a job can
be executed on any one of the processors. Let ~(Ji ) denote the execution time of
task Ji' that is, the amount of time it takes to execute Ji on a processor. There
is also a partial ordering < specified over P. It is required that if J. < J. then
! 3
the execution of job J. cannot begin until the execution of J. has been completed.
3 l
(Ji is said to be a predecesso r of Jj, and Jj is said to be a successor of ~..J j.J
is an immediate successor of l'J if there is no Jk such that Ji < Jk < Jj') Formally,
a set of tasks is specified by an ordered triple (P,~,<) where ~ is a function from
P to the reals. A set of jobs can be described by a directed graph such as that in
Fig. 2(a). There is a directed arc from the vertex (job) J. to the vertex (job) J.
l j
if and only if J. is an immediate successor of J..
j i
By scheduling a set of jobs on a multiprocessor computing system we mean to
specify for each job J. the time interval within which it is to be executed and the
I
processor Pk on which execution will take place. (b) (Without loss of generality, we
assume that execution of the set begins at t = 0). An explicit way to describe a
schedule is a timing diagram, which is also known as the Gantt chart. As an example,
the timing diagram of a schedule for the execution of the set of jobs in Fig. 2(a) on
a two-processor computing system is shown in Fig. 2(b). In a given schedule, an
idle period of a processor is defined to be a time interval within which the proces-
sor is not executing a job (while at least one other processor is executing some
job). In a schedule, a processor might be left idle for a period of time either be-
cause there is no executable job within that time period or because it is an inten-
tional choice. (A job is said to be ready at a certain time instant if the execution
of its predecessors have all been completed at that time.) Clearly, it is never
necessary nor beneficial in a schedule to leave all processors idle at the same time.
(bJThroughout~ our discussion in this section, we assume that once the execution
of a job begins on a processor, it will continue until completion without any
interruption.
The total execution time of a schedule is the total time it takes to execute all the
jobs according to the schedule. We shall use completion time as the criterion for
computing different schedules.
Throughout this paper, the following assumptions are made:
)i( All jobs have equal execution time, say, unit time.
(ii) All schedules arenon-preemptive, meaning that once the execution of a
task begins on a processor it will continue until completion.
With these two assumptions it can be easily seen that an optimal schedule can
always be found among schedules with no intentional idle periods. That is, a proc-
essor is left idle for a certain period of time if and only if no job is ready with-
in that period. Thus we need only be concerned with such schedules. In this case, a
scheduling algorithm can be specified by merely giving the rules on how jobs are to
be chosen for execution at any instant when one or more processor is free. (Of
course, the choice is only among jobs that are executable at that instant.) A sim-
ple way to spell out the rule is to assign priorities to the jobs so that jobs with
higher priorities will be executed instead of jobs with lower priorities when they
are competing for processors. (If two or more processors are available, pick the
one with smallest index.) Consequently, we call such algorithms priority-driven
schedu!inga~_gorithms. The usual practice is to list the jobs in descending order
of their priorities from left to right. Such a list is called a priority list.
Whenever a processor is free, the priority list is searched from left to right and
the first ready job encountered will be executed.
LEVEL AND LEVEL ALGORITHMS
For a given set of jobs and a partial ordering over them, we define the level
of a job as follows.
(i) The level of a job that has no successor is equal to i.
(ii) The level of a job that has one or more successors is equal to one plus
the maximal value of the levels of the successors of the job.
We study in this paper a class of algorithms which we shall call level
algorithms. They are priority-driven scheduling algorithms in which a job of a
higher level will be assigned higher priority than a job of a lower level. A level
schedule is one produced by a level algorithm. Level algorithms are interesting be-
cause
)i( they are simple,
and
(ii) Hu's algorithm ]5[ and Coffman and Graham's algorithm ]i[ are both level
algorithms.
THE SIMPLE LEVEL ALGORITHMS
We define a simple level algorithm (SLA) as a level algorithm in which the
assignment of priorities of jobs within the same level is completely arbitrary. For
a given set of jobs, we shall use h to denote the value of the highest level and use
Ni, i = l,...,h to denote the number of jobs at level i. Two jobs are independent
if there is no precedence relation between them.
Theorem :i For an n-processors system, if the partial ordering over the jobs satis-
fies the condition that the number of independent jobs in the union of the sets of
immediate successors of any n jobs is at most n. Then the SLA produces an optimal
schedule.
Remark: Theorem 1 is a generalization of Hu's theorem [5]. In his theorem the
partial ordering is assumed to be a tree or forest and those partial orderings
clearly satisfy the hypothesis of Theorem .i
Figure l(a) represents a partial ordering which is not a tree but satisfies the
hypothesis of Theorem 1 in the case of 2-processors.
~igure l(a)
We shall establish 3 lemmas which are needed in the proof of Theorem i.
Lemma l(a): Let (P,<) be a set of jobs with unit execution time. A necessary con-
dition for completing the execution of the jobs in time t, where t is an integer, is
h + N Nh_ 1 + ... + l.N --< n(t-i+l) (i-i)
for i = h, h - i, ..., i~ where n is the number of processors in the computing
system.
Proof: If we are to complete the execution of all jobs at time ,t we must complete
the execution of all the jobs at levels h, h-l, ..., i by t-i+l. Because if this is
not the case, there will be a chain of i jobs to be executed in the remaining i-i
time units, which is an impossibility. Since the computing system can execute at
most n(t-i+l) jobs in t-i+l units of time, the inequalities in (i-I) follow
immediately.
For an arbitrary set of jobs P, there certainly exists a t which satisfies (I-i).
Let t(P) he the smallest of all such t's which satisfy (i-i).
