Table Of ContentReport Documentation Page Form Approved
OMB No. 0704-0188
Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and
maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington
VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it
does not display a currently valid OMB control number.
1. REPORT DATE 3. DATES COVERED
2003 2. REPORT TYPE 00-00-2003 to 00-00-2003
4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER
NAVO MSRC Navigator. Spring 2003
5b. GRANT NUMBER
5c. PROGRAM ELEMENT NUMBER
6. AUTHOR(S) 5d. PROJECT NUMBER
5e. TASK NUMBER
5f. WORK UNIT NUMBER
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION
Naval Oceanographic Office (NAVO),Major Shared Resource Center REPORT NUMBER
(MSRC),1002 Balch Boulevard,Stennis Space Center,MS,39522
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)
11. SPONSOR/MONITOR’S REPORT
NUMBER(S)
12. DISTRIBUTION/AVAILABILITY STATEMENT
Approved for public release; distribution unlimited
13. SUPPLEMENTARY NOTES
14. ABSTRACT
15. SUBJECT TERMS
16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF
ABSTRACT OF PAGES RESPONSIBLE PERSON
a. REPORT b. ABSTRACT c. THIS PAGE Same as 32
unclassified unclassified unclassified Report (SAR)
Standard Form 298 (Rev. 8-98)
Prescribed by ANSI Std Z39-18
The Director’s Corner
Steve Adamec, NAVO MSRC Director
Changing to Better
Serve You
In the past several months we have seen substantial change and progress here at the NAVO MSRC. We've
brought on board our new MSRC technical services provider, Lockheed Martin (more in the article that
follows on page 4), who brings substantial expertise and enthusiasm to their support of this MSRC and the
DoD HPC Modernization Program (HPCMP). We're also completing several major Center enhancements,
designated as Technology Insertion for Fiscal Year 2003 (TI-03), across several major technology areas
within the MSRC. These include substantial upgrades to the IBM POWER4 HPC (MARCELLUS) system,
the Remote Storage Facility (RSF), and the internal MSRC networking capability. When complete, these
enhancements will provide almost 10 teraflops of aggregate peak computing capability with
commensurately balanced storage and networking capabilities. This enormous computational capability
will continue to enable unparalleled advances in the DoD science and technology areas served by the
HPCMP.
As I've mentioned in past issues of the Navigator, we recognize that it is critically important for us to
redouble our efforts in assessing and implementing common user environments, practices, and tools within
and across the Centers. Your individual and collective user feedback through the User Advocacy Group
makes it clear that you consider this to be one of your highest priorities for us. In response, we've formed
new internal teams whose primary goal is to strengthen our linkage and participation with the HPCMP
Programming Environment and Training (PET) program elements that emphasize user environment, tools,
and productivity.
My staff and I look forward to seeing you in June at the 2003 HPCMP Users' Conference in Bellevue,
Washington. As always, please take every opportunity to let us know how we can better serve you. Your
feedback is critically important to us and to the HPCMP.
ABOUT THE COVER:
This image shows the temperature variable in a dataset created by a computational model run on the IBM
POWER4 (MARCELLUS) in support of the Airborne Laser Challenge Project II. The data were visualized using
Alias|Wavefront Maya 4.5 on a Windows 2000 workstation. A volumetric surface rendering technique was used
for data elements where temperature was in the top 20 percent of the data range, 0.8 to 1.0. Temperature data
values from 0.65 to 0.8 were rendered using a volume cloud technique. See “Data Visualization with
Alias|Wavefront Maya 4.5,” page 22, for further information about how this image was created.
2 SPRING 2003 NAVO MSRC NAVIGATOR
Contents
The Naval Oceanographic Office (NAVO)
Major Shared Resource Center (MSRC):
Delivering Science to the Warfighter
The NAVO MSRC provides Department of
Defense (DoD) scientists and engineers with high
performance computing (HPC) resources, The Director’s Corner
including leading edge computational systems,
large-scale data storage and archiving, scientific
2 Changing to Better Serve You
visualization resources and training, and expertise
in specific computational technology areas (CTAs). 4 A New Teammate Joins the NAVO MSRC
These CTAs include Computational Fluid
Dynamics (CFD), Climate/Weather/Ocean
Modeling and Simulation (CWO), Environmental Feature Articles
Quality Modeling and Simulation (EQM),
Computational Electromagnetics and Acoustics 5 Lattice-Boltzmann Large-Eddy Simulation of Turbulent
(CEA), and Signal/Image Processing (SIP).
Jet Flows
NAVO MSRC 9 High Performance Computing and Simulation for
Code N7 Advanced Armament Propulsion
1002 Balch Boulevard
16 Clear Air and Optical Turbulence in a Jet Stream in
Stennis Space Center, MS 39522
the Airborne Laser Context
1-800-993-7677 or
[email protected]
High Performance Computing
11 Largest NAVO MSRC System Becomes Even Bigger
and Better
11 Using the smp Queue on MARCELLUS
Programming Environment and Training
NAVO MSRC Navigator
www.navo.hpc.mil/Navigator 13 Environmental Quality Modeling Activities Under PET
24 NAVO MSRC PETUpdate
NAVO MSRC Navigator is a biannual technical
publication designed to inform users of the news, 24 PET Distance Learning: Ready to Serve 24/7
events, people, accomplishments, and activities of
the Center. For a free subscription or to make 25 A Consistent, Well-Documented Computational
address changes, contact NAVO MSRC at the
Environment for the DoD HPC Centers
above address.
27 PET Climate/Weather/Ocean (CWO) Modeling and
EDITOR:
Simulation—A Brief Review
Gioia Furness Petro, [email protected]
Scientific Visualization
DESIGNERS:
Cynthia Millaudon, [email protected]
22 Data Visualization with Alias|Wavefront Maya 4.5
Kerry Townson, [email protected]
Lynn Yott, [email protected] The Porthole
Any opinions, conclusions, or recommendations in
23 Visitors to the Naval Oceanographic Office
this publication are those of the author(s) and do
not necessarily reflect those of the Navy or NAVO Major Shared Resource Center
MSRC. All brand names and product names are
trademarks or registered trademarks of their
Navigator Tools and Tips
respective holders. These names are for
information purposes only and do not imply
endorsement by the Navy or NAVO MSRC. 29 Programming TotalView and Vampir
Upcoming Events
Approved for Public Release 31 Conference Listings
Distribution Unlimited
NAVO MSRC NAVIGATOR SPRING2003 3
A New Teammate Joins the NAVO MSRC
Linda Wise Pyfrom, Program Manager, Lockheed Martin at NAVOMSRC
On 15 January 2003, Lockheed Martin (LM) became the provided hardware and software management and technical
newest member of the Naval Oceanographic Office Major direction to global Fortune 500 companies.
Shared Resource Center (NAVO MSRC) team. LM is excited Ms. Pyfrom authored and instructed the Trusted Software
about providing Technical Operations and User Support Methodology for the National Security Agency Cryptologic
Services to the NAVO MSRC and supporting the Center and assisted in the transition of this methodology to
NAVOCEANO High Performance Computing (HPC) team. the Carnegie Mellon University Software Engineering
LM brings a wealth of HPC expertise to the NAVO MSRC Institute. She was Principal Systems Engineer in support of
through leading research into, and participation in the the Strategic Defense Initiative for Martin Marietta and
development of, next-generation HPC systems. LM provides General Electric. Ms. Pyfrom began her career as a Test
HPC hardware and software services for large-scale Engineer for Ford Aerospace serving the U.S. Air Force at its
computational users and utilizes HPC resources in the design Cheyenne Mountain Space Defense Operations Center.
of advanced technology products. Whether producing HPC
hardware with a five order-of-magnitude performance CHARLIEROBERTSON- MANAGER, TECHNICALOPERATIONS
improvement at Sandia National Laboratories or linking the Charlie Robertson has more than 40 years of management,
LM team with customer laboratories in real-time simulations technical, and supervisory experience. He has served as site
for the Joint Strike Fighter Program, LM is an innovative manager for HPC facility management services and site
member of the HPC community. manager for military command and control software
LM brings a highly experienced group to the NAVO MSRC development projects. Most recently, he was Program
team that is honored to work with the Operational and Director for Technical Operations for the NAVO MSRC. Prior
Research users of NAVO MSRC services. The LM team is to that position, he served as the Program Director of the
committed to continuing the evolution of the NAVO MSRC U.S. Navy Primary Oceanographic Prediction System
HPC capabilities in the 21st century. (POPS).
Key management team members include:
JEFFGOSCINIAK- MANAGER, USERSERVICES
LINDAWISEPYFROM - LM PROGRAMMANAGER Jeff Gosciniak has 20 years of experience in the leadership
Linda Wise Pyfrom brings to the NAVO MSRC team of software development efforts, including the development
extensive experience in large-scale Information Technology of Highly Available Enterprise System Architectures. Most
(IT), strategic direction, program management, engineering, recently, he served as Manager, Information Systems
and process implementation for commercial, defense, and Engineering and Security for LM on the Consolidated Space
civil government customers. Operations Contract (CSOC) for NASA. In this position he
She has served as Director of Information Technology and served as the Chief Systems Architect for the CSOConline
Director of Information Systems for LM in support of NASA, computing infrastructure. Mr. Gosciniak also served more
the Navy, and other government customers. As Program than 10 years for LM Aeronautics, where he played an
Manager, Ms. Pyfrom led a 186-person team in the instrumental role in the IT efforts that supported the LM win
development of the Naval Standard Integrated Personnel of the Joint Strike Fighter contract. Prior to joining LM, Mr.
System (NSIPS), the newly operational system supporting Gosciniak worked in the Technology Laboratories for the
Navy active service personnel and retirees. Ms. Pyfrom has Eveready Battery Company.
The new LM team
leaders (L-R):
Jeff Gosciniak,
Linda Wise Pyfrom, and
Charlie Robertson.
4 SPRING 2003 NAVO MSRC NAVIGATOR
Lattice-Boltzmann Large-Eddy Simulation of
Turbulent Jet Flows
S. Menon and H. Feiz, School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA
Sponsored by Army Research Office
Active control using fuel modulation
(by employing embedded micro-
synthetic jets inside fuel injectors) has
been experimentally shown to be an
effective approach to control
combustion instability in gas turbine
engines. Numerical simulations can
help in the design cycle if the
dynamics of the interaction between
the actuators and the combustion
process can be properly modeled.
However, this involves resolving
motion over a wide range of scales.
For example, a typical fuel injector
orifice can be as small as 1-5
millimeters (mm), and the embedded
microscale actuators are even smaller.
On the other hand, the typical scale of Figure 1. Computational domain for the synthetic and forced jet
simulations. In both cases, the flow at the entrance of the inlet pipe is
a combustor is around 30 centimeters
forced at the same frequency. But, in the jet case only, a mean flow is
(cm). The resolution requirement to
also added to the inflow condition. The flow at the exit plane of the
resolve the microjets and the flow orifice evolves naturally in both cases.
outside in the combustor is too severe
for any single numerical method. finite-difference schemes because it collision process allows the recovery of
recovers the Navier-Stokes equations the nonlinear macroscopic advection
The Lattice-Boltzmann (LB) method,
and is computationally very efficient, through multi-scale expansions. Second,
when combined with the conventional
more stable, and easily parallelizable. because the macroscopic properties of
Finite-Volume Large-Eddy Simulation
Additionally, the LB method solves a the flow field are not solved directly,
(FV-LES), has the potential to provide
single continuous particle distribution the LB method avoids solving the
a collaborative resolution to this
multiscale problem. (which is analogous to the particle Poisson equation, which is numerically
distribution function in kinetic theory) difficult in most finite difference
In this approach, the LB-LES
in a lattice (or grid). methods. Third, the macroscopic
approach is employed to resolve
regions inside the microjets and fuel The introduction of the Bhatnager- properties are obtained from the
injector while FV-LES is employed Gross-Krook (BGK) single relaxation microscopic particle distributions
elsewhere in the combustor. This time model for the collision operator through simple arithmetic integration.
article reports on the ability of the further simplifies the algorithm and In this model, a new, second-order,
LB-LES approach to capture eliminates the lack of Galilean accurate Three Dimensional (3D) LB
complex dynamics in jet flows in a invariance and the dependence of method has been developed using a 3D
computationally efficient manner. The pressure on velocity. Solving the LB cubic lattice model with the 19-bit
coupled LB-LES and FV-LES multi- Equation (LBE) instead of the Navier- velocity discretization (used here to
scale approach is currently being Stokes equation has three distinct recover the Navier-Stokes equation).
validated and will be described in advantages: First, due to the kinetic This model has been extended to
the near future. nature of the LB method, the deal with complex geometries and
The LB method is considered an convection operator is linear. Simple to include a variable grid without loss
attractive alternative to conventional convection in conjunction with a of accuracy.
NAVO MSRC NAVIGATOR SPRING2003 5
breakdown as the flow expands
downstream of the orifice. Dissipation
is maximum in the high strain regions
that typically reside in the braid
regions and in the regions
surrounding the vortices.
The square jet also shows an axis-
switching behavior seen in the
experiments as well. Axis switching is
indicated by the crossover of the
Additionally, spreading rate of the jet in the two
to enhance its a planes. In the near-field region of the
applicability to high jet exit, the vortex structures at the
Reynolds number flow, an LES corners are formed farther
version of this model has been downstream with respect to the
developed whereby a localized sides. This triggers the axis switching
dynamic subgrid model is employed since it results in the formation of
to compute an additional subgrid THE SQUARE SYNTHETIC AND nonplanar vortex structure.
relaxation time in the BGK model of FORCED JETS Comparisons with data from Feiz et
the LBE. The dynamic evaluation The dimensions of the square jet al.1 show reasonable agreement with
eliminates the need to specify any ad computational domain are shown in past experiments.
hoc parameters since all model Figure 1. The grid is stretched from
coefficients evolve naturally as a part the high resolution in the orifice SQUARE JET IN CROSS-FLOW
of the simulation. region, but the stretching is The computational domain for the test
To expedite the turnaround time, the maintained below 10 percent to case shown in Figure 3 is resolved
LBE-LES solver is implemented in ensure accuracy is not compromised. using 200x150x100 for the cross-flow
parallel using the Message Passing The inlet region is resolved using domain and 50X50X100 for the jet
Interface (MPI). The computational 170x170x52, the nozzle is resolved section. The Reynolds number is
efficiency of the LB-LES solver is using 66x66x7, and the outflow 4700, based on the jet velocity and
considerable and achieves 4.42 x10-9 region is resolved using 202x202x234. the nozzle width D, and the jet cross-
Central Processing Unit (CPU) Figures 2a and 2b show, respectively, flow velocity ratio is 0.5. The cross-
seconds per time step, per grid point,
typical visualization of the vortex flow flow velocity profile is initialized with a
per processor on the IBM SP4. For a
generated by the synthetic jet and boundary layer thickness of 2D.
typical simulation of 20 forcing
forced square jet. The forcing Figure 3 also shows a comparison of
cycles, using 11 million grid points,
approximately 2,000 single-processor frequency for both cases is the same, predicted mean velocity and total
hours are needed on the IBM POWER and the main difference between kinetic energy with data at a specified
4 machine (MARCELLUS). these two cases is that
there is no mean flow
A key feature of all the studies b
in the synthetic jet
reported here is that the inlet pipe is
case.
fully resolved so that the flow at the
jet exit plane evolves naturally. This is A key feature
in contrast to many past studies where observed in both is the
the jet exit plane profile is typically effect of vortex
specified as a boundary condition. stretching and
Figure 2. Flow visualization of (a) forced square jet and
(b) synthetic jet. The color iso-surfaces indicate values of
constant vorticity. Green indicates azimuthal vorticity,
and red/blue indicates streamwise vorticity of equal and
opposite sign. Initially, azimuthally coherent vortices are
shed from the orifice, but undergo vortex switching and
stretching, eventually leading to breakdown in more
randomly oriented streamwise vortices.
6 SPRING 2003 NAVO MSRC NAVIGATOR
Figure 4. Flow visualization of the
jet in cross flow. The formation of
the hanging vortices and the
formation of the counter-rotating
pair in the downstream direction is
clearly seen. Recirculation
downstream of these structures
also forms, as indicated by the
streamlines.
location. Very good agreement is
obtained here and also at other
locations.1
A jet in cross-flow generates a
complex flow topology due to the
highly 3D nature of this flow. Past
studies have identified two structural
featuresin this flow: a horseshoe (or
kidney-shaped) structure and a Figure 4 also shows how the jet rolls which allows the jet exit profile to
Counter-Rotating Vortex Pair (CRVP)
up and creates the recirculation evolve naturally.
form in this flow.
region: an important mechanism for Good agreement with established
The current LB-LES captures both the mixing of jet and the cross-flow. data was obtained in the present
these features and also explains the Finally, Figure 5 shows a time study. These results establish LBE-
dynamics of the formation of these
sequence of the formation of these LES as an alternate method for
structures and their subsequent
flow features as the jet exits from the simulating turbulent shear flows.
breakdown. Figure 4 shows these
orifice and is turned downstream by
features quite clearly. Future application of this LBE-LES
the cross-flow.
approach will be in a hierarchical
The horseshoe vortices are tubelike
In summary, a new LES implementation simulation approach whereby
structures that form directly above
the exit on the lateral edges of the jet of the LBE method has been conventional finite-volume LES
and extend around the jet body developed and used to simulate a methods will be used to resolve the
and up along the lee side of the jet, 3D square jet and a 3D square jet large-scale flow features in the
approximately matching the path of in cross-flow. A localized dynamic combustor, while the LBE-LES
the jet. These tubes coincide with the subgrid closure is used to close the approach will be used to resolve the
location where the jet shear layer LES version of the LBE model. In finer scale features as in the
folds and eventually contribute to the these simulations the inflow is applied embedded synthetic jet and/or the
circulation of the CRVP. far upstream of the jet exit plane, flow inside the fuel injector.
X / D = 0
U (EXP)
V
W
2 K
U (LBE)
V
W
K
1
0
-1 0 1 2
Figure 3. Schematic of the jet in cross-flow and comparison with experimental data.
NAVO MSRC NAVIGATOR SPRING2003 7
1 2 3
4 5 6
Figure 5. Time sequence of the formation of the jet in cross-flow and the shedding of the hanging vortices as
the flow propagates downstream.
References
1.Feiz, H., H. Soo, and S. Menon, "LES of Turbulent Jets Using the Lattice Boltzmann Approach," AIAA Paper No. 2003-0780, 41st AIAA
Aerospace Sciences Meeting and Exhibit, Reno, NV, January 2003.
Contact Information
For further information contact Suresh Menon at [email protected]
8 SPRING 2003 NAVO MSRC NAVIGATOR
High Performance Computing and Simulation
for Advanced Armament Propulsion
Michael J. Nusca, U.S. Army Research Laboratory (ARL), Aberdeen Proving Ground, MD
The Army is exploring a variety of and repeatedly ignite the high-energy Computing (HPC) facilities. However,
armament propulsion options for and HLD solid propellant charge. the gun propulsion-modeling
indirect- and direct-fire weapons As modular and HLD propelling environment has historically been
(guns) for the legacy force and Future charges are being developed, one in which separate codes
Combat Systems (FCS). optimized, and ultimately mated to (some one-dimensional, some two-
As it transforms, the Army has systems such as indirect-fire cannon dimensional (2D)) are used, with no
identified requirements for and the continually evolving FCS, single multidimensional code able
hypervelocity projectile launch there is a critical need to have a to address the truly 3D details of all
systems for strategic Army missions. single, validated, maintainable these weapons systems.
Among these systems are those that computer code based on state-of-the-
This unfortunate situation renders
use solid propellant—granular form art Computational Fluid Dynamics
comparison of ballistic performance
loaded in modules (indirect-fire) or (CFD) as an evaluation and
cumbersome and inconclusive. In
disk and strip form for High-Loading- performance analysis tool.
contrast, the multiphase continuum
Density (HDL) cartridges (direct-fire) It has long been recognized that the equations that represent the physics
—augmented by ElectroThermal- availability of such a tool would of gun propulsion comprise a set
Chemical (ETC) technology. Two such provide the Army with the unique of general equations universally
armament propulsion systems are the capability to simulate current and applicable to all solid propellant
Army's Modular Artillery Charge emerging gun propulsion systems armament propulsion systems.
System (MACS) and HLD charges for
using computer simulations. These
In direct response to this situation the
the FCS.
simulations would serve to both
ARL began a development program
The MACS is being developed for streamline testing and aid in the
about eight years ago to revolutionize
indirect fire cannon on current 155 optimization of weapon performance.
the Army's ability to use HPC to
milimeter (mm) systems (e.g., the
Indeed, such a tool would dovetail simulate propelling charges. The
M109A6 Paladin and M198 Towed
nicely with the Army's initiative in the current author at ARL, with
Howitzer). The efficiency of the
creation of national High Performance consultation from noted industry/
MACS charge is dependent on proper
academic experts, has worked on
flamespreading through the
this project. The result is the
propellant modules, a process that
has been repeatedly demonstrated a Army's "next-generation," computer
scaleable, 3D, multiphase,
in gun firings, successfully
CFD code for armament propulsion
photographed using the Army
modeling.
Research Laboratory (ARL) 155
mm ballistics simulator, and The ARL NGEN3 code represents the
b
numerically modeled using the sole Department of Defense (DoD)
ARL Next Generation Three computer tool that is able to simulate
Dimensional (3D) interior ballistics the highly complex physics associated
code (NGEN3). The FCS requires with indirect- and direct-fire guns.
c
weapons systems exhibiting NGEN3 code development and
increased range, accuracy, application to the FCS is a
and highly repeatable projectile DoD HPC Challenge Project (FY01-
launch performance. 03) and is being exercised regularly
Figure 1. (a) Porosity Contours (red is
One of the technologies under with priority access to the Cray
dense material) at Initial Time,
investigation to achieve these goals is SV1ex at the Naval Oceanographic
(b) Porosity Contours at 6 ms, and
the ETC concept, in which electrically (c) Propellant Temperature Contours Office Major Shared Resource Center
generated plasma is injected into the (red is fully ignited propellant at 440K) (NAVO MSRC).
gun chamber in order to efficiently at 6 ms.
NAVO MSRC NAVIGATOR SPRING2003 9
