NAS Grid Benchmarks: A Tool for Measuring Performance of Computational Grids

3 Pages·2001·0.12 MB·English

by  VanderWijngaart, Rob F.

.... _' "I /
NASGrid Benchmarks: PhilipDormer Stanhope, Ear/of Chester_eld (1746):
Ajob worth doing _ worth doing well
A Tool for Measuring Performance of
Computational Grids
Nerd's addendum:
AdiSlJTbutedjob worth doing is worth doing well
Michael Frumkin (NAS Division)
Rob F. Vander Wijngaart (Computer SciencesCorp.)
Nerd's addendum 'saddendum:
NASA Ames Research Center
Adistributed job worth doing is worth measuring
Moffett Field, CA
lie,el,o_ 7.1o,2{I_i
Contents Brief history of benchmarkinq at NAS
NAS Parallel Benchmarks (NPB)
1. Brief history of benchmarking atNAS Goal: MMM Measure efficiency ofmodern (parallel)
machines running scientific application
2. Measuring Grid performance
Audtence:M User: application engineer, researcher
3. Preliminary results
Buyer + vendor
Contents: 5kernels, 3 pseudo apps (implicit CFD)
Approach: NPBI: Paper-and-pencil specs- "91
NPB2: Source codeimplementations
(F/7/C/MPI)- '97
PBN: Sourcecodeimplementations
(HPF/OpenMP/.lava)- _J9
$ $
NAS Parallel Benchmark codes Measurinq Grid performance
NAS Grid Benchmarks (NGB)
Kernels: • EP Random-number generator Goals: MM Measure efficiency + functionality of modern
MMMM Grids running (distributed) scientific application
•IS Integer sort (includes public network performance)
•CG Conjugate gradient
Audience: Grid user: application engineer, researcher
•MG Multigrid method for Poisson eqn Audience: Grid software/infrastructure developer
•FT Spectral method (FFT) for Laplace eqn
Contents: 4 compound tasks: 3 pseudo apps, 2kernels,
all from NPB
Pseudo •BT ADI; Block-Tddiagonal systems
apps: Approach: NGBI: Paper-and-pencil specs - "01
•SP ADI; Scalar Pentadiagonal systems NGB2: Source code implementations
•LU Lower-Upper symmetric Gauss-Seidel (NPB2/PBN + Globus/Legion/CORB/V
Grid Engine/Condor/Java ...)
e_ll, oa _-Io,zoel
Grid benchmark requirements Grid benchmark construction
• Tests computational aspects ofenvironment • Provide synthetic Grid app. forscientific computing:
• Is representative of scientific computing tasks -. Data Flow Graph (DFG) coupling scientific codes
• Uses basicGridservices •Specify:
• Is notintrusive (nothroughput stress testing) .- abstract services: authenticate, create task,
communicate
• Contains communicating processes
• Does significant communication ,,-problem sizes (classes): S, A, B,C....
• Is verifiable (deterministic, notinteractively steered) • Do no_.s_ptecify:
• Needs noinitialization data files r Mapping, scheduling, fault tolerance, data security
r Is fair (no favorite before dust settles) •Measure and report tu_rour_ time
_,"_"_1*_ *,,,_,II ,_
Nodes of NGB Data Flow Graphs NGB Data Flow Graphs, Sample size
Modified NPB mesh-based codes SP, BT, LU, MG, FT: Embarrassingly Distributed (ED) Helical Chain (HC)
,- Well-accepted, well-studied, widely portable
,- Solid verification procedures exist
.- Parallel versions available: MPI/OpenMP/HPF/Java
r No data files required, but ...
-.... output ofone NPB may be input foranother
.- Symbolize components ofscientific apps:
• BT, SP, LU: scientific comps (flow solvers)
• MG: post-procossing (data smoother)
• FT:visualization (spectral analysis) Parameter study Cyclic process (restart)
$ e41w3,od 7._,o.2ool
NGB Data Flow Graphs, Sample size Approximate rationale for NGB DFG classes
Visualization Pipe (VP) Mixed Bag(MB)
• S:for testing only; completes inseconds
* A:reasonable sizeeng. problem; fitsonmodern SS[
machine without time sharing processes
o B:challenging problem sizefor advanced SSI machine
• C:won't fiton mostadvanced SS!machines
Considerations:
• Keep # verification values limited
• Provide significant range of problem sizes
• Complete inreasonable time (s-hrs oncritical path)
. Benchmarks non-converging, numerically stable
Visualization cycle Unbalanced chain • Ensure T(compute) _ T(communicate)
2
fl
?
•_,], od T-I(N2aot wl, od z-l_,_ $
Larqer NGB DFG classes Preliminary results
(ED.X IXE{A,B,C}) = 7_,{16,64,256} SP.X •Pilot implementations in Java and Gtobus under
(HC.X IXE{A,B,C}) = El {3,6,12} (BT.X,SP.X, LU.X) development.
(VP.XlXE{A,B,C}) = rl{g,18,36}(B'F.XM,G.X,Fr.X) •Preliminary results
for ED.S and HC.A
MB onproto-cluster and .....
VP
Origin cluster, resp. ' I_
•x]
,7
r.GF_Oct7-10._01 $ "Gr,p_,oct7-i_ 2_Ol $
I
Preliminary results: Java / HC.S I Preliminary results: Globus / ED.A
0. I
J
.Ii!III!!IILI.I!.III!.!I. 1.!].!.
o 2 6 o • Io 1t 14
RUnNuttier
NGB into the future Errata
Possible useof NGBs; interpretation ofresults • No NGBclassW
Primitive Grid services • Only verification in Report node
Grid maturity
• Functionality increases • LP -* HC, CP _ VP
• Consistency among runs I1
Complete resources reservation
• Vanation ofsingle resource
Definition ofuniform Grid currency (G$)
• Cost ofNGB pedormance per G$