PDF version also avaliable

202 Pages·1999·0.71 MB·English

Enabling Technologies for Peta(FL)OPS
Computing
Thomas Sterling
Universities Space Research Association
Paul Messina
California Institute of Technology
Paul H. Smith
National Aeronautics and Space Administration HQ
i
Abstract
The Workshop on Enabling Technologies for Peta(FL)OPS Computing
was held on February 22 through 24, 1994 at the DoubleTree Hotel
in Pasadena, California. More than 60 experts in all aspects of high-
performance computing technology met to establish the basis for con-
sidering future research initiatives that will lead to the development,
production, and application of PetaFLOPS scaled computing systems.
Theobjectives oftheworkshopwere to: 1)Identifyapplicationsthatre-
quire PetaFLOPS performance and determine their resource demands,
2) Determine the scope of the technical challenge to achieving e(cid:11)ective
PetaFLOPS computing, 3) Identify critical enabling technologies that
lead to PetaFLOPS computing capability, 4) Establish key research is-
sues, and 5) Recommendelements of a near-term research agenda.
The workshop focused on four major and inter-related topic areas:
ApplicationsandAlgorithms,Device Technology,Architecture andSys-
tems, and Software Technology. The workshop participants engaged in
focused sessions of small groups and plenary sessions for cross-cutting
discussions. The (cid:12)ndings produced re(cid:13)ect the potential opportunities
and the daunting challenges that confront designers and users of future
PetaFLOPS computing systems. A PetaFLOPS computing system will
be feasible in two decades and will be important, perhaps even criti-
cal, to key applications at that time. This prediction is based, in part,
on the key assumptionthat there willbe acontinuationthroughout the
twenty-yearperiodofthecurrentsemiconductorindustryadvancesboth
in speed enhancement and in cost reduction through improved fabrica-
tion processes. While no paradigmshift is required in systems architec-
ture, active latency managementwill be essential requiring a very high
degree of (cid:12)ne-grain parallelism and the mechanisms to exploit it. A
mix of technologies will be required including semi-conductor for main
memory, optics for inter-processor (and perhaps inter-chip) communi-
cations and secondary storage, and possibly cryogenics (e.g., Josephson
Junction) for very high clock rate and very low power processor logic.
E(cid:11)ectiveness and applicabilitywill rest on dramatic per device cost re-
ductionandinnovativeapproachestosystemsoftwareandprogramming
methodologies.
Near-term studies are required to re(cid:12)ne these (cid:12)ndings through more
detailed examinationof system requirements and technology extrapola-
ii
tion. Thisreportdocumentstheissuesand(cid:12)ndingsofthe1994Pasadena
PetaFLOPS workshop and makes speci(cid:12)c recommendations for near-
term research initiatives.
iii
Acknowledgments
The editors of this publicationwish to thank allthose who participated
intheworkshopformakingitanhistoricaleventintheevolutionofhigh-
performance computing. In addition, the editors wish to acknowledge
theimportantcontributionsmadebyseveralassociateswhowererespon-
sible for the excellent workshop arrangements and the high professional
qualityofthis publication. MichaelMacDonaldprovidedtechnical edit-
ing,reviewingallaspectsofthisreportandcontributingsubstantivelyto
anumberofitssections. TerriCanzianprovidedexhaustiveanddetailed
editing of the entire text and is responsible for the document’s profes-
sionalformatandtypesetting. TinaPauna’spainstakingeditingweeded
out countless awkward phrases and glitches. Tim Brice is credited for
the success of the local arrangements and excellent logistical support
throughout the workshop. Michele O’Connell provided important as-
sistance to the workshop organizers prior to, during, and following the
workshop and was responsible for coordination between the organizing
committee, program committee, and local arrangements. Mary Goro(cid:11),
ErlaSolomon,andChipChapmanassisted withregistration,computers
and copying equipment, and in handling the many details that arise in
the course of a dynamicworkshop.
Executive Summary
A PetaFLOPS is a measure of computer performance equal to a mil-
lion billion operations (or (cid:13)oating point operations) per second. It is
comparable to more than ten times all the networked computing ca-
pability in America and is ten thousand times faster than the world’s
most powerful massively parallel computer. A PetaFLOPS computer
is so far beyond anything within contemporary experience that its ar-
chitecture, technology, and programmingmethods may require entirely
new paradigmsinorder toachievee(cid:11)ective use ofcomputingsystemsat
this scale. For the U.S. to retain leadership in high-performance com-
puting development and application in the future, planning and even
early research into PetaFLOPS system design and methodologies may
be essential now. To start these processes a number of Federal agencies
combined to sponsor the (cid:12)rst majorconference in this emergingarea.
The Workshop on Enabling Technologies for Peta(FL)OPS Comput-
ingwashostedbytheJetPropulsionLaboratoryinPasadena,California
iv
fromFebruary22through24,1994andincludedover60invitedcontrib-
utors fromindustry, academia,and government. They met to establish
the basisforconsideringfuture research initiativesthatwillleadtoU.S.
preeminence in developing, producing, and applying PetaFLOPS-scale
computingsystems.
The broad goal of the Workshop on Enabling Technologies for
Peta(FL)OPS Computingwastoconduct andproduce the (cid:12)rst compre-
hensive assessment of the (cid:12)eld of PetaFLOPS computing systems and
to establish a baseline of understanding of its opportunities, challenges,
and critical elements with the intent of setting near-term research di-
rections to reduce uncertainty and enhance our knowledge of this (cid:12)eld.
The major objectives of the workshop were to
Identify Applications ofeconomic,scienti(cid:12)c,andsocietalimportance
requiring PetaFLOPS scale computing.
Determine Challenge in terms of technical barriers to achieving ef-
fective PetaFLOPS computing systems.
Reveal Enabling Technologies that may be critical to the imple-
mentationofPetaFLOPScomputersanddeterminetheirrespectiveroles
in contributing to this objective.
Derive Research Issues that de(cid:12)ne the boundary between today’s
state-of-the-art understanding and the critical advanced concepts toto-
morrow’sPetaFLOPS computingsystems.
Set Research Agenda for initialnear-term work focused on immedi-
ate questions contributing to the uncertainty of our understanding and
imposingthe greatest risk to launching a major long-term research ini-
tiative.
Theworkshopwas sponsored jointlybytheNationalAeronautics and
Space Agency, the Department of Energy, the National Science Foun-
dation, the Advanced Research Projects Agency, the National Security
Agency, and the Ballistic Missile Defense Organization. Invited partic-
ipants were selected to ensure the highest quality and coverage of the
driving technical areas as well as representation from all elements of
the high-performance computing community. The direction and nature
of the workshop were set by opening talks presented by Seymour Cray
and Konstantin Likharev. The workshop was organized into four work-
ing groups re(cid:13)ecting the pace-setting disciplines that both enable and
v
limitprogress toward practical PetaFLOPS computing systems. These
working groups were
Applications and Algorithms
(cid:15)
Device Technology
(cid:15)
Parallel Architectures and System Structures
(cid:15)
System Software and Tools.
(cid:15)
The Applications Working Group considered the classes of applica-
tions and algorithms that were both important to national needs and
capableofexploitingthisscaleofprocessing. Throughthese discussions,
some understanding of the resource requirements for such applications
wasderived. TheDevice TechnologyWorkingGroupexplored the three
most likely technologies to contribute to achieving PetaFLOPS perfor-
mance: semiconductor, optics, and cryogenic superconducting. This
groupestablishedprojectionsofthecapabilitiesforeachtechnologyfam-
ilyanddistinguishedthemintermsoftheirstrengths andweaknesses in
supporting PetaFLOPS computing. The Architecture Working Group
examined three alternative structures comprising processor, communi-
cation, and memorysubunits enabled by future technologies and scaled
to PetaFLOPS performance. They investigated the most likely organi-
zationsandmixesoffunctionalelementsatdi(cid:11)erent levelsoftechnology
capabilityto reveal aspectrum of possible systems. The SoftwareTech-
nology Working Group took on the challenging task of delineating the
principal obstacles imposed by current software environments to e(cid:11)ec-
tive application of future PetaFLOPS computing systems. They also
examinedtheimplicationsofalternativeenvironmentsandfunctionality
that mightsubstantively contribute to enhanced usefulness.
This (cid:12)rst comprehensive review of the emerging (cid:12)eld of PetaFLOPS
computing systems produced a number of important (cid:12)ndings that
broadly de(cid:12)ne the challenge, opportunities, and approach to realizing
this ambitions goal. These were as much derived from interactions
amongthe workinggroups as comingfromdeliberations withinanysin-
gle group. The following re(cid:13)ect the major (cid:12)ndings of the workshop
combiningkey contributions fromall four of the workinggroups:
1. Construction of an e(cid:11)ective PetaFLOPS computing system will be fea-
sible in approximately20 years, based on current technology trend pro-
vi
jections.
2. Thereareandwillbeawiderangeofapplicationsinscience,engineering,
economics,andsocietalinformationinfrastructureandmanagementthat
willdemand PetaFLOPS capability in the near future.
3. Cost, more than any other single aspect of a PetaFLOPS initiative,
will dominate the ultimate viability and the time frame in which such
systems will come into practical use.
4. Reliability of PetaFLOPS computer systems will be manageable but
only because cost considerations will preclude systems having a much
greaternumberofcomponentsthancurrentmassivelyparallelprocessing
systems.
5. No fundamental paradigm shift in system architecture is required to
achieve PetaFLOPS capable systems. Advanced variations on the
NUMA MIMD (and possibly SIMD) architecture model should su(cid:14)ce,
although speci(cid:12)c details mayvary signi(cid:12)cantly from today’s implemen-
tations.
6. It is likely that a PetaFLOPS computer will exhibit a wide diameter,
i.e., the propagation delay across a system measured in system clock
cycles. Latency management techniques and very high concurrency on
the order of a million-foldwill be key facets of systems of this scale.
7. The PetaFLOPS computer will be dominated by its memory. But, at
leastforscienceandengineeringapplications,memorycapacitywillscale
less than linearly with performance. A system capable of PetaFLOPS
performance willrequire on the order of 30 terabytes of mainmemory.
8. To achieve PetaFLOPS performance, such computers will comprise a
mixoftechnologyprovidingbetter performancetocost thanpossibleby
anysingletechnology. Semiconductortechnologywilldominatememory
withsomelogic,andprogresstowardthisgoalwillbetiedtoadvancesin
the semiconductor industry. Optics will provide high bandwidth, inter-
module communication at all levels and mass storage but little or no
logic. Superconducting Josephson Junction technology may yield very
high-performance logic and exceptionally low power consumption.
9. Majoradvancesinsoftwaremethodologiesforprogrammingandresource
managementwillbenecessaryifsuchsystemsaretobepracticalforend-
user applications.
vii
During the course of deliberations amongthe workshop participants,
manyissues were brought to light, clarifyingthe space of opportunities
and obstacles but leaving many questions unanswered. For example,
assumptions about semiconductor technology in 20 years were derived
fromSIAprojectionstotheyear2007andrequiredextrapolationbeyond
thatpoint. Theeconomicsofspecialtyhardwarewasquestioned,leaving
unresolved the degree towhichanyfuture PetaFLOPS computerdesign
must rely on commodity parts developed for more general commercial
application. The nature oftheuser base forPetaFLOPS computerswas
highlycontested. Thepossibilitiesincludedclassicalscience/engineering
problems, total immersion virtual reality human interfacing, and mas-
sive information management and retrieval. The di(cid:14)culty of program-
ming even today’s massively parallel processing systems left open the
possibility that signi(cid:12)cant resources would be committed to achieving
ease-of-use at the cost of sustained performance. But how such systems
would ultimately be programmed is uncertain. The narrow scope of
architectures examined was still very broad with respect to the tech-
nology issues they posed. Although for each of the three architectures
latency is seen as an issue driving system architecture decisions, the
space of alternatives was too wide to permit a speci(cid:12)c approach to be
recommended over all others. And, beyond the approaches explicitly
examined, there remains the possibility of completely untried architec-
tures that mightaccelerate greatly the pace to PetaFLOPS computing.
These and other issues, while revealed as important at this workshop,
remained unresolved at its close.
Finally,the workshop concluded with key recommendationsfor near-
term initiatives to reduce uncertainty and advance U.S. capability to-
ward the achievement of PetaFLOPS computing. In the area of de-
vice technology,itwas considered imperativethatbetter projections for
semiconductor evolution be developed, and that the true potential of
superconducting technology be better understood. With regards to ap-
plications,speci(cid:12)c examplesidenti(cid:12)edascandidatesforPetaFLOPS ex-
ecutionshouldbestudiedindepthtodeterminethebalanceofresources
required atthatscale inorder tovalidatethe appropriateness ofthe pri-
mary candidate architectures. Such a study should include at least one
exampleofanapplicationforwhich there islittlecurrent use but which
is potentially important to the future. The architecture working group
covered manyfacets of PetaFLOPS architecture and produced a mean-
viii
ingfuloverview of a tenable PetaFLOPS computer structure, but many
details had to be left unspeci(cid:12)ed. It is recommended that a near-term
study be initiated to (cid:12)ll in the gaps, determining the requirements of
the constituent elements of such a future machine. These speci(cid:12)cations
are essential for validatingthe approach and determining requirements
for allof the technologies used in its implementation.
In conclusion, the Workshop on Enabling Technologies for
Peta(FL)OPSComputingwasanhistoricmeetingthatbroughttogether
a remarkable set of experts in the (cid:12)eld of high-performance computing
and focused their talents on a question of great future importance to
ourNation’sstrength inscience andengineering,as wellas itseconomic
leadership in the world of the next century. Ideas, both conservative
and controversial,were explored and the workshop resulted inan initial
set of (cid:12)ndings that will set the course toward the ultimateachievement
of a PetaFLOPS computer. But, an important immediate consequence
ofthisworkshopbeyond the greater understanding ofPetaFLOPS com-
puting systems achieved was the extraordinary synergism and cross fer-
tilizationofideasthatoccurred amongsomeofthisNation’smajorcon-
tributors to computer science.
Contents
Abstract i
Acknowledgments iii
Executive Summary iii
PART I x
1 Introduction 1
1.1 Overview 1
1.2 Objectives 3
1.2.1Identify the Applications 3
1.2.2Determine the Challenge 4
1.2.3Reveal the Enabling Technologies 4
1.2.4Derive the Research Issues 4
1.2.5Set the Research Agenda 4
1.3 Approach 4
1.4 Background 7
1.5 Issues 10
1.6 Report Organization 12
2 PetaFLOPS from Two Perspectives 15
PART II 27
3 Summary of Working Group Reports 29
3.1 Applications 29
3.2 Device Technology 32
3.2.1Semiconductor Technologies 32
3.2.2Optical Technologies 34
3.2.3Superconducting Technologies 36
3.3 Architecture 37