Portable Parallelization of Industrial Aerodynamic Applications (POPINDA): Results of a BMBF Project

231 Pages·1999·13.92 MB·English

by  Anton Schüller

Portable Parallelization of
Industrial Aerodynamic
Applications (POPINDA)
Edited by
Anton Schuller
Notes on Numerical Fluid Mechanics (NNFM) Volume 71
Series Editors: Ernst Heinrich Hirschel, Miinchen (General Editor)
Kozo Fujii, Tokyo
Werner Haase, Miinchen
Bram van Leer, Ann Arbor
Michael A. Leschziner, Manchester
Maurizio Pandolfi, Torino
Arthur Rizzi, Stockholm
Bernard Roux, Marseille
Volume 71: Portable Parallelization of Industrial Aerodynamic Applications (POPINDA) (A. Schuller,
Ed.)
Volume 70: Numerical Treatment of Multi-Scale Problems (w. Hackbusch I G. Wittum, Eds.)
Volume 69: Computational Mesh Adaptation (D. P. Hills I M. R. Morris I M. J. Marchant I P. Guillen, Eds.)
Volume 68: Recent Development of Aerodynamic Design Methodologies (K. Fujii I G. S. Dulikravich, Eds.)
Volume 67: Experimental and Numerical Investigation of Time Varying Wakes behind Turbine Blades
(c. H. Sieverding I G. CicateIli I J. M. Desse I M. Meinke I P. Zunino)
Volume 66: Numerical Flow Simulation I (E. H. Hirschel, Ed.)
Volume 65: Computation and Comparison of Efficient Turbulence Models for Aeronautics - ETMA
project (A. Dervieux I M. Braza I J.-O. Dussauge, Eds.)
Volume 64: Computation and Visualization of Three-Dimensional Vortical and Turbulent Flows
(R. Friedrich I P. Bontoux, Eds.)
Volume 63: Vortices and Heat Transfer (M. Fiebig I Nimai K. Mitra, Eds.)
Volume 62: Large-Scale Scientific Computations of Engineering and Environmental Problems
(M. Griebel I O. P. Iliev I S. D. Margenov I P. S. Vassilevski, Eds.)
Volume 61 Optimum Aerodynamic Design & Parallel Navier-Stokes Computations, ECARP-European
Computational Aerodynamics Research Project (J. Periaux I G. Bugeda I P. Chaviaropoulos I
K. Giannokoglou I S. Lanteri I B. Mantel, Eds.)
Volume 60 New Results in Numerical and Experimental Fluid Mechanics. Contributions to the 10th AG
STAB/DGLR Symposium Braunschweig, Germany 1996 (H. Komer I R. Hilbig, Eds.)
Volume 59 Modeling and Computations in Environmental Sciences. Proceedings of the First GAMM
Seminar at ICA Stuttgart, October 12-13, 1995 (R. Helmig I W. Jager I W. Kinzelbach I
P. Knabner I G. Wittum, Eds.)
Volume 58 ECARP - European Computational Aerodynamics Research Project: Validation of
CFD Codes and Assessment of Turbulence Models
(w. Haase I E. Chaput I E. Elsholz 1M. A. Leschziner I U. R. Muller, Eds.)
Volume 57 Euler and Navier-Stokes Solvers Using Multi-Dimensional Upwind Schemes and
Multigrid Acceleration. Results of the BRITEIEURAM Projects AERO-CT89-0003 and
AER2-CT92-00040, 1989-1995 (H. Deconinck I B. Koren, Eds.)
Volume 56 EUROSHOCK-Drag Reduktion by Passive Shock Control. Results of the Project
EUROSHOCK, AER2-CT92-0049. Supported by the European Union, 1993-1995
(E. Stanewsky I J. D6lery I J. Fulker I W. GeiBler, Eds.)
Volume 55 EUROPT - A European Initiative on Optimum Design Methods in Aerodynamics.
Proceedings of the Brite/Euram Project Workshop "Optimum Design in Aerodynamics",
Barcelona, 1992 (J. Periaux I G. Bugeda I P. K. Chaviaropoulos IT. Labrujere I
B. Stoufflet, Eds.)
Volume 54 Boundary Elements: Implementation and Analysis of Advanced Algorithms. Proceedings of
the Twelfth GAMM-Seminar, Kiel, January 19-21, 1996 (w. Hackbusch I G. Wittum, Eds.)
Volume 53 Computation of Three-Dimensional Complex Flows. Proceedings of the IMACS-COST
Conference on Computational Fluid Dynamics, Lausanne, September 13-15,1995
(M. Deville I S. Gavrilakis II. L. Ryhming, Eds.)
Volume 52 Flow Simulation with High-Performance Computers II. DFG Priority Research Programme
Results 1993-1995 (E. H. Hirschel, Ed.)
Volumes 1 to 51 are out of print.
The addresses of the Editors are listed at the end of the book.
Portable Parallelization of
Industrial Aerodynamic
Applications (POPINDA)
Results of a BMBF Project
Edited by
Anton Schuller
~
Vleweg
Die Deutsche Bibliothek - CIP-Einheitsaufnahme
Portable parallelizatioD of iDdastriai aerodynamic applicatioDS!
results of a BMBF project I Anton SchUller. - Braunschweig;
Wiesbaden: Vieweg, 1999
(Notes on numerical fluid mechanics; 71)
ISBN-13: 978-3-322-86578-6 e-ISBN-13: 978-3-322-86576-2
DOl: 10.1007/978-3-322-86576-2
All rights reserved
© Friedr. Vieweg & Sohn Verlagsgesellschaft mbH, BraunschweiglWiesbaden, 1999
Softcover reprint of the hardcover I st edition 1999
Vieweg ist a subsidiary company of Bertelsmann Professional Information.
No part of this publication may be reproduced, stored in a retrieval
system or transmitted, mechanical, photocopying or otherwise,
without prior permission of the copyright holder.
http://www.vieweg.de
Produced by W. LangelUddecke, Braunschweig
Printed on acid-free paper
ISSNOI79-9614
ISBN-13: 978-3-322-86578-6
Preface
This book contains the main results of the German project POPINDA. It surveys the state
of the art of industrial aerodynamic design simulations on parallel systems. POPINDA
is an acronym for Portable Parallelization of Industrial Aerodynamic Applications. This
project started in late 1993. The research and development work invested in POPINDA
corresponds to about 12 scientists working full-time for the three and a half years of the
project. POPINDA was funded by the German Federal Ministry for Education, Science,
Research and Technology (BMBF).
The central goals of POPINDA were to unify and parallelize the block-structured
aerodynamic flow codes of the German aircraft industry and to develop new algorithmic
approaches to improve the efficiency and robustness of these programs.
The philosophy behind these goals is that challenging and important numerical appli
cations such as the prediction of the 3D viscous flow around full aircraft in aerodynamic
design can only be carried out successfully if the benefits of modern fast numerical solvers
and parallel high performance computers are combined. This combination is a "conditio
sine qua non" if more complex applications such as aerodynamic design optimization or
fluid structure interaction problems have to be solved. When being solved in a standard
industrial aerodynamic design process, such more complex applications even require a
substantial further reduction of computing times.
Parallel and vector computers on the one side and innovative numerical algorithms such
as multigrid on the other have enabled impressive improvements in scientific computing
in the last 15 years.
A principle which will allow similar progress in the future is adaptitlity. Adaptive
methods compute an approximation to the solution on coarse grids and switch to finer
grids only on those parts of the computational domain, where the accuracy of the discrete
solution is not yet sufficient. When following this principle, the discretization in large parts
of the computational domain in a typical aerodynamic flow simulation can be carried
o~t on a fairly coarse grid. Finer grids are required near the aircraft and at certain
(a priori unknown) places of the flow domain. Since coarse grids implicitly mean low
computational effort, substantial reductions of computing times can be obtained. The
combination of adaptivity and parallelism is not at all straightforward. On the contrary,
there is a kind of conflict between these principles because the adaptive grid structures
evolve dynamically at run-time. Load-balancing and efficiency are thus crucial problems
for adaptive applications on parallel systems. The deVelopment of algorithms and tools
for adaptive refinement on parallel systems was therefore another important topic during
the last year of POPINDA.
I would like to thank all colleagues and scientists involved in the project for the
fruitful and excellent cooperation thoughout the project and afterwards when preparing
the material for this book.
January, 1999 Anton Schuller
v
CONTENTS
Page
1 Overview 1
1.1 Basis, Goals and Results of POPINDA (A. Schuller) . . . . . . . . 1
1.1.1 Introduction and Summary ............... 1
1.1.2 Background...................... 3
1.1.3 Basis............................ 4
1.1.4 Approach and Ideas. . . . . . . . . . . . . . . . . . . . 5
1.1.5 Results........................... 9
1.1.6 Reasons for the Success of POPINDA. . . . . . 14
1.1.7 Impact and Outlook .... . . . . . . . . . . . 15
1.2 POPINDA - the Industrial Qualification (K. Becker) . 17
2 Parallelization and Benchmarking 23
2.1 Unified Block Structures - the Basis for Parallelization (K. Becker, H.M.
Bleecke, B. EisJeld, N. Kroll) .................... 23
2.1.1 Requirements for the Parallelization of Large CFD Codes . . . . .. 23
2.1.2 Parallelization Strategies . . . . . . . . . . . . . . . . . . . . . . .. 24
2.1.3 Parallelization of Block-Structured Flow Solvers within the POP INDA
Project. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 26
2.1.4 Basic Concept of Block Structure . . . . . . . . . . . . 28
2.1.5 Standardization of Production Codes . . . . . . . . . . 32
2.2 The High-Level Communications Library CLIC (H. RitzdorJ) 33
2.2.1 Introduction........................ 33
2.2.2 Overview on Functionality of the CLIC-3D . . . . . . . 34
2.2.3 CLIC-3D Design Issues. . . . . . . . . . . . . . . . . . 36
2.2.4 Analysis of the Block Structure . . . . . . . . . . . . . 37
2.2.5 Distribution of Alteration rughts on Block Boundaries 40
2.2.6 Special Communication Requirements on Block-Structured Grids . 41
2.2.7 Creation of Node Processes and Mapping of the Blocks . . . . . .. 47
2.2.8 Special Communication Tasks Performed on Node Processes . . .. 48
2.2.9 Parallel Output . . . . . . . . . . . . . . . . . . . . . . . . 48
2.2.10 Global Operations over All Node Processes . . . . . . . . . . . . .. 49
2.2.11 Future Tasks to Be Realized by the CLIC-3D . . . . . . . . . . .. 49
2.3 Porting CLIC from PARMACS to MPI (M. Hoffmann, U. ReijJig) . . . .. 51
2.3.1 The Objective. . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 51
2.3.2 The Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 52
2.3.3 Schematic Representation of Conversion by Means of PM2MPI. .. 52
2.3.4 The GMD Conversion Tool PM2MPI . . . . . . . . . . . . . . . .. 53
CONTENTS (continued) Page
2.3.5 Tools for Conversion . . . . . . . . . . . . . . . . . . . . . . . . .. 53
2.3.6 Further Developments and Improvements. . . . . . . . . . . . . .. 55
2.3.7 Results.................................. 56
2.4 FLOWer (N. Kroll, B. Eisfeld, H.M. Bleecke) .........•.•.... 58
2.4.1 Governing Equations . . . . . . . . . . . . . . . . . . . . . . . . .. 58
2.4.2 Spatial Discretization. . . . . . . . . . . . . . . . . . . . . . . . .. 60
2.4.3 Time Integration . . . . . . . . . . . . . . . . . . . . . . . . . . .. 64
2.4.4 Acceleration Techniques for Steady Calculations . . . . . . . . . .. 65
2.4.5 Exchange of Solution Data at Block Boundaries . . . . . . . . . .. 67
2.4.6 Parallelization of the FLOWer Code. . . . . . . . . . . . . . . . .. 68
2.5 NSFLEX-P (R.K. Hold) . . . . . . . . . . . . . . . . . . . . . . . . . . .. 72
2.5.1 Governing Equations .................. '. . . . . . .. 72
2.5.2 The Navier-Stokes Solver NSFLEX-P . . . . . . . . . . . . . . . .. 75
2.6 Benchmarks and Large Scale Examples (H. M. Bleecke, B. Eisfeld, R. Hein-
rich, H. RitzdorJ, W. Fritz, S. Leicher, P. Aumann) . . . . . . . . . . . .. 89
2.6.1 Benchmarks............................... 89
2.6.2 Large Scale Examples ......................... 97
3 Algorithmic Aspects 105
3.1 Singularities of Block-Structured Meshes - a Special Parallelizable Ap-
proach (K. Becker, P. Aumann) ........................ 105
3.2 Dual-Time Stepping Method (R. Heinrich) .................. 112
3.3 Scalability of Parallel Multigrid (A. Schiilier, B. Steckel) .......... 118
3.3.1 Introduction ............................... 118
3.3.2 LiSS - a Package for the Parallel Solution of Partial Differential
Equations ................................ 119
3.3.3 Multigrid Treatment of Block Boundaries ............... 119
3.3.4 The Solution on the Coarsest Grid .................. 120
3.3.5 Conclusions............................... 122
3.4 Convergence for Increasing Numbers of Blocks (N. Kroll, S. Leicher, C. W.
Oosteriee, B. Steckel) ...................•.......... 124
3.4.1 Introduction............................... 124
3.4.2 Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
3.4.3 FLOWer................................. 125
3.4.4 LiSS ..............•.................... 131
3.5 New Smoothers for Higher Order Upwind Discretizations of Convection
Dominated Problems like the Euler Equations (C. W. Oosteriee, F.J. Gas-
par, T. Washio, R. Wienands) ..........•.............. 135
3.5.1 Introduction ............................... 135
3.5.2 The Discretization and the Solution Method . . . . . . . . . . . . . 136
3.5.3 Fourier Analysis Results . . . . . . . . . . . . . . . . . . . . . . . . 138
3.5.4 Numerical Results .......... ................. 141
3.5.5 Conclusions........................... .... 145
3.6 Krylov Subspace Acceleration for Linear and Nonlinear Multigrid Schemes
(C. W. Oosteriee, T. Washio) ......................... 146
3.6.1 Introduction............ ................... 146
VIII
CONTENTS (continued) Page
3.6.2 The Krylov Acceleration for Linear Multigrid Methods ....... 147
3.6.3 The Krylov Acceleration for Nonlinear Multigrid Methods ..... 153
3.6.4 Conclusions............................... 162
3.7 Multiple Semi-Coarsening for 3D Singularly Perturbed Scala.r Partial Dif-
ferential Equations (T. Washio, C. W. OosterIee) . .............. 165
3.7.1 Introduction ............................... 165
3.7.2 The 3D Solution Method ........................ 166
3.7.3 3D Numerical Results ......................... 175
3.7.4 Conclusions............................... 177
4 Adaptive Local Refinements 181
4.1 Why to Use Adaptive Grids? (C. W. OosterIee, A. SchiilIer, U. Trottenber:g}181
4.1.1 Future Applications ........................... 181
4.1.2 Idea ................................... 182
4.1.3 A Simple Example ........................... 183
4.1.4 Multigrid on Adaptive Grids ...................... 184
4.1.5 Refinement Criteria. . . . . . . . . . . . . . . . . . . . . . . . . . . 185
4.1.6 Discretization at Boundaries of Refinement Areas .......... 187
4.1. 7 Problems of the Paralle1ization for Block-Structured Multigrid ... 187
4.2 Self-Adaptive Local Refinements Supported by the CLIC-3D Library (H.
Ritzdorj) . .................................... 190
4.2.1 Introduction ............................... 190
4.2.2 Overview of CLIC Functions Supporting Self-Adaptive Local Re-
finements ................................ 191
4.2.3 Adaptive Multigrid (MLAT) on Block-Structured Grids ....... 192
4.2.4 The Refinement Criteria of CLIC-3D . . . . . . . . . . . . . . . . . 193
4.2.5 Creation of a New "Refined" Block Structure ............ 194
4.2.6 Transfer of Grid Function Values ................... 196
4.2.7 Future Tasks to Be Realized for Local Refinements by the CLIC-3D 198
4.3 Load-Balancing Strategies (H. Holthoff) ................... 200
4.3.1 Introduction ............................... 200
4.3.2 Communication Model . . . . . . . . . . . . . . . . . . . . . . . . . 201
4.3.3 Example................................. 203
4.4 Experiences LiSS (H. Ritzdorj, A. SchiiJler, B. Steckel, K. Stiiben, J. Wu) 206
4.4.1 Introduction ............................... 206
4.4.2 Applications and Results ........................ 206
4.5 Experiences FLOWer (H. Bleecke, H. RitzdorJ) ............... 212
4.5.1 Local Refinement Procedure ...................... 212
4.5.2 First Results of the Local Refinement Procedure . . . . . . . . . . . 213
5 Special Aspects and Related Activities 215
5.1 Software Engineering and Software Quality Issues (B. EisJeld, J. Raddatz,
H. M. Bleecke) ................................. 215
5.2 Real Applications on Parallel Systems - the RAPS Initiative (A. Schiiller) 218
5.2.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
5.2.2 Background............................... 219
IX
CONTENTS (continued) Page
5.2.3 Benchmarking Parallel Computers. . . . . .............. 219
5.2.4 The RAPS Approach . . . . . . . . . . . . . . . . . . . . . . . . . . 220
5.2.5 Exploitation of Results . . . . . . . . . . . . . . . . . . . . . . . . . 221
5.2.6 Industrial Impact and Knowledge Flow . . . . . . . . . . . . . . . . 221
5.3 MEGAFLOW (N. Kroll) . . . . . . . . . . . . . . . ............. 222
x
Chapter 1
Overview
1.1 Basis, Goals and Results of POPINDA
Anton Schuller
GMD - German National Research Center for Information Technology
D-53754 Sankt Augustin, Germany
e-mail: [email protected]
Abstract: In this section, we give a survey on the basis, the goals and the results of
the POPINDA project. In POPINDA several large aerodynamic codes for aircraft design
have been parallelized. The basis for the parallelization is the high-level communications
library CLIC, which has also been developed in POPINDA. CLIC takes care of all com
munications tasks occurring in applications employing block-structured grid. Using this
powerful tool, it has been possible to parallelize the codes in a few days.
Further goals have been to develop new fast parallelizable algorithms for the solution
of the corresponding aerodynamic problems with a focus on adaptive grid refinement.
POPINDA is a German national project that has been funded by the German Federal
Ministry for Education, Science, Research and Technology (BMBF).
1.1.1 Introduction and Summary
New Technology Standard in Aerospace Industry
The use of computer simulations for the specification and design of modern aircraft has
a decisive impact on the competitiveness in the aerospace industrial market. Increasing
demands for cost efficiency, speed, safety, comfort, noise reduction and environmental
protection necessitate simulations with increasing complexity. Optimized codes running
on high performance computers, i.e. parallel computers, must be used to meet those
requirements.
In the BMBF cooperation project POPINDA (Portable Parallelization of Industrial
Aerodynamic Applications), the innovative step into parallel computing has been com
pleted: over a period of more than 3 years, the partners DASA Airbus Bremen, DASA
LM with groups in Manching and Munich, DLR Braunschweig and GMD Sankt Augustin
were involved in the collaborative development of the parallel programs FLOWer and
NSFLEX-P for the computation of flow fields around aircraft. Additional partners were
IBM Heidelberg and ORCOM Freiberg. The NEC research Lab in Sankt Augustin was
1
A. Schüller (ed.), Portable Parallelization of Industrial Aerodynamic Applications (POPINDA)
© Friedr. Vieweg & Sohn Verlagsgesellschaft mbH, Braunschweig/Wiesbaden 1999