Table Of ContentLecture Notes in Earth System Sciences
Series Editors
P. Blondel, Bath, UK
J. Reitner, Göttingen, Germany
K. Stüwe, Graz, Austria
M. H. Trauth, Potsdam, Germany
D. A. Yuen, Minnesota, USA
Founding Editors
G. M. Friedman, Brooklyn and Troy, USA
A. Seilacher, Tübingen, Germany and Yale, USA
For further volumes:
http://www.springer.com/series/10529
David A. Yuen • Long Wang
Xuebin Chi • Lennart Johnsson
Wei Ge • Yaolin Shi
Editors
GPU Solutions to Multi-scale
Problems in Science
and Engineering
123
Editors
David A. Yuen Lennart Johnsson
Department of Earth Sciences and Computer Science
Minnesota Supercomputing Institute University of Houston
University of Minnesota Houston, TX
Minneapolis, MN USA
USA
Wei Ge
and Institute of Process Engineering
Chinese Academy of Sciences
School of Environmental Sciences Beijing
China University of Geosciences People’s Republic of China
Wuhan
People’s Republic of China Yaolin Shi
Laboratory of Computational Geodynamics
Long Wang Chinese Academy of Sciences
Network Information Center Beijing
Beijing People’s Republic of China
People’s Republic of China
Xuebin Chi
Supercomputing Center
Beijing
People’s Republic of China
ISSN 2193-8571 ISSN 2193-858X (electronic)
ISBN 978-3-642-16404-0 ISBN 978-3-642-16405-7 (eBook)
DOI 10.1007/978-3-642-16405-7
Springer Heidelberg New York Dordrecht London
Library of Congress Control Number: 2012952572
Ó Springer-Verlag Berlin Heidelberg 2013
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or
information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed. Exempted from this legal reservation are brief
excerpts in connection with reviews or scholarly analysis or material supplied specifically for the
purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the
work. Duplication of this publication or parts thereof is permitted only under the provisions
of the Copyright Law of the Publisher’s location, in its current version, and permission for use must
always be obtained from Springer. Permissions for use may be obtained through RightsLink at the
Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt
from the relevant protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for
any errors or omissions that may be made. The publisher makes no warranty, express or implied, with
respect to the material contained herein.
Printed on acid-free paper
Springer is part of Springer Science?Business Media (www.springer.com)
Preface
The past several years have witnessed a great transformative scenario occurring in
the computational science by the extremely rapid incursion made by GPU and
many-core architecture into the arena of high-performance computing (HPC). At
the Supercomputing 2009 meeting in Portland, a group of us (X. Chi, D. A. Yuen,
and H. Tufo from University of Colorado) gathered together and formulated plans
for a GPU conference to be held in China. First, it was supposed to be held in
Shanghai as proposed by Jifeng Yao from Shanghai Supercomputing Center, but
because of the EXPO 2010, we had to move it to Harbin, which turns out to be a
more delightful locale during the summer. Our timing for this conference was
quite propitious, as the platforms armed with GPU and fast networking carried the
day and captured three out of the top five slots of the TOP-500 list in Super-
computing 2010 in New Orleans. This auspicious event justified our thinking back
in 2009 about the potential importance of GPU computing.
The first International Workshop of GPU Solutions to Multiscale Problems in
Science and Engineering (GMP-SMP2010) kicked off July 26, 2010 in Harbin, the
capital of Heilongjiang province in Northeast (Dongbei) China. The workshop was
organized by the Supercomputing Center of Computer Network Information
Center, Chinese Academy of Sciences (CAS) in Beijing, the Graduate University
of CAS, and the Chinese Society of Theoretical and Applied Mechanics. Nearly
100 computational experts and scholars from the world’s well-known universities
and institutes such as the University of Houston, the National Astronomical
Observatory of Japan, Hong Kong Baptist University, the University of Chicago,
the Tokyo Institute of Technology, Japan, Brown University, the University of
Amsterdam, the University of Erlangen-Nuremberg, National Center of Atmo-
spheric Center, University of Minnesota, Macalester College, University of Bonn,
the Chinese Academy of Science, Tsinghua University, Peking University, Fudan
University, China, University of Science and Technology of China (USTC),
attended this international workshop.
The leadoff talk was given by Professor Xuebin Chi, the Director of the
Supercomputing Center of CNIC, CAS. He stressed strongly that a revolutionary
change is brewing in the field of technologies and applications of high-
v
vi Preface
performance computing due to the rapid development of GPU and many-core
technology. He further expressed the hope that the discussion on GPU applications
to multiscale problems in science and engineering be extremely fruitful. Following
Professor Chi’s speech, Professor David Yuen from University of Minnesota also
gave an exciting opening talk, encouraging strongly vigorous participation by the
students from China and the international contingent.
During the two-day workshop period, the attendees discussed topics on GPU
solutions to multiscale problems in science and engineering. The workshop con-
sisted of three sessions, the keynote lecture session, the invited lecture session, and
the student and poster session. The keynote lectures are ‘‘Development and
application of a HPC system for multiscale discrete simulation Mole-8.5’’ given
by Professor Wei Ge from IPE, CAS, ‘‘Acceleration for energy efficient, cost
effective HPC’’ given by Professor Lennart Johnsson from the University of
Houston, USA, ‘‘Practical Random Linear Network Coding on GPUs’’ given by
Professor Xiaowen Chu from the Hong Kong Baptist University, China and
‘‘GRAPE and GRAPE-DR’’ given by Professor Jun Makino from the National
Astronomical Observatory of Japan, Japan. In the subsequent talks, issues on
seeking GPU solutions to multiscale problems were addressed from different
viewpoints, such as focusing on high-performance computing methods and algo-
rithms, efficient software implementation techniques, the construction of scientific
computing environment, the mainstream development trends, as well as other
GPU-related issues in scientific computing and visualization technology. In the
second afternoon students, both graduates and undergraduates from USA and
China, gave a bulk of the presentations as well as interesting posters.
In the closing banquet, Professor David Yuen, on behalf of the organizing
committee, announced that the conference proceedings will be published soon and
the workshop will be held again in 2011, and Lanzhou was mentioned as a
potential place. Plans for a book from the workshop were also laid out at the
closing banquet on July 28.
As far as we are aware of, the few extant books on GPU and multi-core
computing (e.g., Kirk and Hwu 2010; Kurzak et al. 2011) are written by computer
scientists. Thus, there is now a dire need for a book with a strong applicational
bent in order to encourage more people to join the GPU game.
Our papers are divided into two types: long expository papers with ample
illustrations and examples and short contributions with a particular applicational
focus in mind. The book is divided into eight sections. In section 1 we begin with
this preface. Then we follow with an article by Matt Knepley and David Yuen -
which addresses the reasons why scientists and engineers should be considering
GPU. This is followed by a chapter by Lang, Wang, and Yuen offering photos of
the workshop itself. This book is rather unique at this time has articles, spanning
many different areas in science and engineering, which show the health of this
burgeoning field. This book is to be contrasted with the recent book edited by
Kurzak et al. (2011), which focused mainly on hardware and algorithms and is
devoted to a computer science audience rather than people in computational
science.
Preface vii
This book covers aspects of hardware and green computing in section 2. We
discuss in section 3 software libraries from both China and the USA., namely
PARRAY by Chen Yifeng and PETSc by the group at Argonne. In section 4 we
shows the industrial applications on GPU. There Wei Ge’s group performed an
outstanding job in getting a super performance from a GPU–CPU system using
thousands of GPU. In section 5, we shows the inroads made by GPU in density-
functional theory and electronic structures. Section 6 deals with geophysical and
fluid dynamical applications. We see the 145 Tflop performance on a weather code
from the Japanese group at Tokyo Institute of Technology. There is also appli-
cation of GPU on 3D elastic-wave propagation by Taro Okamoto and also by S.
Song and his colleagues. In section 7, on algorithms and solvers have a thorough
discussion of multigrid solver as applied to industrial problems by H. Koestler
from Germany. In the final section on visualization, we present a variety of
application on imaging and microtomography using GPU.
We plan to communicate broadly about the potential of GPU and many-core
computing to the scientific and engineering community out there and not restricted
to computer scientists. We hope this book will give the future perspectives of GPU
to scientists and engineers and will stimulate further growth in this field. We thank
both the Chinese Academy of Sciences and the OCI program of National Science
Foundation for their generous support. We are very grateful for the help provided
by Xianyu Lang, Qing Zhao, and Yichen Zhou in preparation of this volume.
Minnesota, USA David A. Yuen
Beijing, China Long Wang
Xuebin Chi
References
Kirk DB, Hwu WW (2010) Programming massively parallel processors: a hands-on-approach.
Elsevier, Amsterdam, p 256
Kurzak J, Bader DA, Dongarra J (eds) (2011) Scientific computing with multicore and accel-
erators. CRC Press, Boca Raton, p 480
Contents
Part I Introductory Material
1 Why Do Scientists and Engineers Need GPU’s Today? . . . . . . . . 3
Matthew G. Knepley and David A. Yuen
2 Happenings at the GPU Conference . . . . . . . . . . . . . . . . . . . . . . 13
Xian-yu Lang, Long Wang and David A. Yuen
Part II Hardware and Installations
3 Efficiency, Energy Efficiency and Programming of Accelerated
HPC Servers: Highlights of PRACE Studies . . . . . . . . . . . . . . . . 33
Lennart Johnsson
4 GRAPE and GRAPE-DR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Junichiro Makino
Part III Software Libraries
5 PARRAY: A Unifying Array Representation
for Heterogeneous Parallelism. . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Yifeng Chen, Xiang Cui and Hong Mei
6 Practical Random Linear Network Coding on GPUs . . . . . . . . . . 115
Xiaowen Chu and Kaiyong Zhao
7 Preliminary Implementation of PETSc Using GPUs. . . . . . . . . . . 131
Victor Minden, Barry Smith and Matthew G. Knepley
ix
x Contents
Part IV Industrial Applications
8 Multi-scale Continuum-Particle Simulation on CPU–GPU
Hybrid Supercomputer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Wei Ge, Ji Xu, Qingang Xiong, Xiaowei Wang, Feiguo Chen,
Limin Wang, Chaofeng Hou, Ming Xu and Jinghai Li
9 GPU Best Practices for HPC Applications at Industry Scale . . . . 163
Peng Wang and Stan Posey
10 Simulation of 1D Condensing Flows with CESE Method
on GPU Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
Wei Ran, Wan Cheng, Fenghua Qin and Xisheng Luo
11 Two-Way Coupled Sprays and Liquid Surface: A GPU-Based
Multi-Scale Fluid Animation Method . . . . . . . . . . . . . . . . . . . . . 187
Guijuan Zhang, Gaojin Wen and Shengzhong Feng
12 High Performance Implementation of Binomial Option
Pricing Using CUDA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
Yechen Gui, Shenzhong Feng, Gaojin Wen, Guijuan Zhang,
Yanyi Wan and Tao Liu
13 Research of Acceleration MS-Alignment Identifying
Post-Translational Modifications on GPU . . . . . . . . . . . . . . . . . . 215
Zhai Yantang, Tu Qiang, Lang Xianyu,
Lu Zhonghua and Chi Xuebin
Part V Chemical Physical Applications
14 GPU Tuning for First-Principle Electronic
Structure Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Yue Wu, Weile Jia, Lin-Wang Wang, Weiguo Gao,
Long Wang and Xuebin Chi
15 Nucleation and Reaction of Dislocations in Some Metals
and Intermetallic Compound TiAl. . . . . . . . . . . . . . . . . . . . . . . . 247
D. S. Xu, H. Wang and R. Yang
Contents xi
Part VI Geophysical and Fluid Dynamical Application
16 Large-Scale Numerical Weather Prediction
on GPU Supercomputer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
Takayuki Aoki and Takashi Shimokawabe
17 Targeting Atmospheric Simulation Algorithms for Large,
Distributed-Memory, GPU-Accelerated Computers . . . . . . . . . . . 271
Matthew R. Norman
18 Investigation of Solving 3D Navier–Stokes Equations
with Hybrid Spectral Scheme Using GPU . . . . . . . . . . . . . . . . . . 283
Ying Xu, Lei Xu, D. D. Zhang and J. F. Yao
19 Correlation of Reservoir and Earthquake by Multi
Temporal-Spatial Scale Flow Driven Pore-Network
Crack Model in Parallel CPU and GPU Platform . . . . . . . . . . . . 295
B. J. Zhu, C. Liu, Y. L. Shi and D. A. Yuen
20 A Full GPU Simulation of Evolving Fracture Networks
in a Heterogeneous Poro-Elasto-Plastic Medium
with Effective-Stress-Dependent Permeability . . . . . . . . . . . . . . . 305
Boris Galvan and Stephen Miller
21 GPU Implementation of Multigrid Solver for Stokes Equation
with Strongly Variable Viscosity . . . . . . . . . . . . . . . . . . . . . . . . . 321
Liang Zheng, Taras Gerya, Matthew Knepley, David A. Yuen,
Huai Zhang and Yaolin Shi
22 High Rayleigh Number Mantle Convection on GPU . . . . . . . . . . 335
David A. Sanchez, Christopher Gonzalez, David A. Yuen,
Grady B. Wright and Gregory A. Barnett
23 High-Order Discontinuous Galerkin Methods
by GPU Metaprogramming . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
Andreas Klöckner, Timothy Warburton and Jan S. Hesthaven
24 Accelerating Large-Scale Simulation of Seismic Wave
Propagation by Multi-GPUs and Three-Dimensional
Domain Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
Taro Okamoto, Hiroshi Takenaka, Takeshi Nakamura
and Takayuki Aoki
xii Contents
25 Support Operator Rupture Dynamics on GPU . . . . . . . . . . . . . . 391
Shenyi Song, Yichen Zhou, Tingxing Dong and David A. Yuen
Part VII Algorithms and Solvers
26 A Geometric Multigrid Solver on GPU Clusters . . . . . . . . . . . . . 407
Harald Koestler, Daniel Ritter and Christian Feichtinger
27 Accelerating 2-Dimensional CFD on Multi-GPU
Supercomputer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423
Sen Li, Xinliang Li, Long Wang, Zhonghua Lu and Xuebin Chi
28 Efficient Rendering of Order Independent Transparency
on the GPUs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437
Fang Liu
29 Performance Evaluation of Fast Fourier Transform
Application on Heterogeneous Platforms . . . . . . . . . . . . . . . . . . . 457
Xiaojun Li, Yang Gao, Xinyu Ma and Ying Liu
30 Accurate Evaluation of Local Averages on GPGPUs . . . . . . . . . . 487
Dmitry A. Karpeev, Matthew G. Knepley and Peter R. Brune
31 Accelerating Swarm Intelligence Algorithms
with GPU-Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503
Robin M. Weiss
32 Asynchronous Parallel Logic Simulation on Modern
Graphics Processors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517
Yangdong Deng, Yuhao Zhu and Wang Bo
33 Implementations of Main Algorithms for Generalized
Symmetric Eigenproblem on GPU Accelerator . . . . . . . . . . . . . . 543
Yonghua Zhao, Fang Liu, Yangang Wang and Xuebin Chi
34 Using Mixed Precision Algorithm for LINPACK Benchmark
on AMD GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555
Xianyi Zhang, Yunquan Zhang and Lei Wang
35 Parallel Lattice Boltzmann Method on CUDA Architecture . . . . . 561
Weibing Feng, Wu Zhang, Bing He and Kai Wang
