Table Of Contentintroduction to
Scientific
and technical
computing
introduction to
sciEnTiFic
and TEchnical
compuTing
Edited by
Frank T. WillmorE • Eric JankoWski
coray colina
Boca Raton London New York
CRC Press is an imprint of the
Taylor & Francis Group, an informa business
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2017 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed on acid-free paper
Version Date: 20160525
International Standard Book Number-13: 978-1-4987-4504-8 (Paperback)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been
made to publish reliable data and information, but the author and publisher cannot assume responsibility for the valid-
ity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright
holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this
form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may
rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or uti-
lized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopy-
ing, microfilming, and recording, or in any information storage or retrieval system, without written permission from the
publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://
www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923,
978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For
organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.
Library of Congress Cataloging‑in‑Publication Data
Names: Willmore, Frank T., author. | Jankowski, Eric, author. | Colina,
Coray, author.
Title: Introduction to scientific and technical computing / Frank T.
Willmore, Eric Jankowski, and Coray Colina.
Description: Boca Raton : Taylor & Francis, CRC press, 2017. | Includes
bibliographical references and index.
Identifiers: LCCN 2016006492 | ISBN 9781498745048 (alk. paper)
Subjects: LCSH: Engineering--Data processing. | Science--Data processing. |
Research--Data processing.
Classification: LCC TA345 .W545 2017 | DDC 502.85--dc23
LC record available at http://lccn.loc.gov/2016006492
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
Foreword .......................................................................................................................................vii
Preface ..............................................................................................................................................ix
Editors ..............................................................................................................................................xi
Contributors .................................................................................................................................xiii
1. Operating Systems Overview ..............................................................................................1
Erik E. Santiso
2. Machine Numbers and the IEEE 754 Floating-Point Standard ...................................31
Frank T. Willmore
3. Developing with Git and Github ......................................................................................39
Chris Ertel
4. Introduction to Bash Scripting ..........................................................................................55
Erik E. Santiso
5. Debugging with gdb ............................................................................................................71
Frank T. Willmore
6. Makefiles, Libraries, and Linking ....................................................................................83
Frank T. Willmore
7. Linking and Interoperability .............................................................................................91
Frank T. Willmore
8. Build Management with CMake .....................................................................................105
Ryan L. Marson and Eric Jankowski
9. Getting Started with Python 3 .........................................................................................119
Brian C. Barnes and Michael S. Sellers
10. Prototyping ..........................................................................................................................137
Charles Lena
11. Introduction to High-Performance Computing Systems ...........................................145
Todd Evans
12. Introduction to Parallel Programming with MPI ........................................................157
Jerome Vienne
13. Introduction to OpenMP ..................................................................................................173
Yaakoub El Khamra
v
vi Contents
14. Checkpointing Code for Restartability with HDF5 ....................................................195
Frank T. Willmore
15. Libraries for Linear Algebra .............................................................................................205
Victor Eijkhout
16. Parallel Computing with Accelerators ...........................................................................219
İnanç Şenocak and Haoqiang Jin
17. Testing and Verification ....................................................................................................235
Paul Kwiatkowski
18. Validation of Computational Models and Codes ........................................................261
Christopher R. Iacovella, Christoph Klein, Janos Sallai, and Ahmed E. Ismail
19. Software Licensing and Distribution.............................................................................275
Paul Saxe
Index .............................................................................................................................................281
Foreword
In the relatively brief history of electronic computers—roughly seven decades—we have
seen exponential increases in capability and reductions in cost and size, enabled by tre-
mendous advances in technology. These exponential rates of change in capability and cost-
efficiency produced the PC revolution, and now the “post-PC” revolution of “smartphones”
and other smart mobile devices, and are now driving us toward an “Internet of Things”
in which computing devices of all types will be ever-present in our homes, cities, trans-
portation, and perhaps even clothes and bodies. It is easy to understand how computing
could now be taken for granted. Computing devices are now pervasive; they are not only
on our desks and in our laps, but in our cars and appliances, in our pockets, and even on
our wrists. Most people interact with computers daily, hourly, or even constantly for enjoy-
ment or for productivity, without ever really pausing to wonder how they work. In fact,
the term “computer” isn’t even applied generally to most of the computers in our lives. This
is perfectly fine, for the most part. However, computers are also important instruments of
science and engineering, necessary for helping us make new discoveries and innovations.
For scientists, engineers, and other technical professionals in universities, labs, and com-
panies, computers are increasingly powerful tools for research and development (R&D).
Sometimes, very powerful computers—often called “supercomputers’’—make headlines
for their role in enabling groundbreaking research. Mostly, however, scientific computing
systems are out of the public awareness, even more invisible than the pervasive devices
we use daily.
The tremendous success of computing in supporting our businesses, running our infra-
structure, and enriching our lives has created a huge demand for computing professionals
in companies that provide these products and services. The business and consumer mar-
kets need experts in cloud computing, data analytics, web services, mobile applications,
devices, networking, and so on. The explosion in business- and consumer-focused com-
puting during the past two decades—since the World Wide Web made computing access
desirable to everyone—has dominated the market for programmers, and thus changed the
focus on computer science programs. The days of scientific computing skills and technolo-
gies being the foundation of computer science curricula are long gone, even though comput-
ing is also more important than ever in science and engineering. The majority of computer
science curricula has understandably evolved with markets for computing technologies. In
2016, essentially all college and graduate students can take courses for programing in Java
or for understanding web services, but few can take science-focused programming classes
to learn the new version of Fortran or how to use C/C++ for scientific applications. Fewer
still can take classes to learn how to use OpenMP and MPI to create high- performance
scientific and engineering applications that run on parallel computing systems.
For these reasons, I created and started teaching a course on parallel computing for
scientists and engineers when I worked at the San Diego Supercomputer Center (SDSC)
in the late 1990s, and taught it at both the University of California, San Diego (UCSD) and
SDSU. The class was based on a 2 to 3 day workshop we offered to SDSC users, most of
whom were faculty and postdocs who needed supercomputers in their research but had
no formal education on how to use them. Shortly after I founded the Texas Advanced
Computing Center (TACC) at The University of Texas (UT) in 2001, we started teaching
a similar class at UT—and then discovered the need for additional, more basic scientific
vii
viii Foreword
computing classes on programing in Fortran and C, performance optimization, visualiza-
tion, data management, and other scientific computing skills. At TACC, we were at the
forefront of computational science, and we volunteered our time to share our expertise
by teaching classes so students could learn these skills, which were not addressed by the
computer science curriculum. Our “scientific computing curriculum” courses were elec-
tives for students, but they increased in popularity as students in science and engineer-
ing programs at UT realized the need for such skills and understanding to pursue their
degrees, especially in graduate research. The classes filled a gap in the curriculum for
future scientists and engineers in training, and our course slides filled the gap normally
addressed by textbooks.
I am pleased to see that the curriculum pendulum is starting to swing back in the United
States, with more universities now offering scientific computing classes: sometimes in sci-
ence or engineering departments, sometimes in computer science departments, sometimes
in new scientific computing/computational science programs. Public appreciation of the
importance of computational science also seems to be on the upswing. The advent of the
“Big Data” era has impacted science as well as business; data-driven scientific discover-
ies such as the Higgs boson and confirmation of gravitational waves have made main-
stream media headlines, as have research advances in medical research. The White House
“National Strategic Computing Initiative,” announced in June 2015, has helped to increase
public awareness of the need for “exascale” computing systems for new scientific discov-
eries, industry competitiveness, and national security. However, the general shortage of
scientific computing classes at universities remains, as does the scarcity of high-quality,
up-to-date textbooks. I am thus very proud that one of the very talented TACC (former)
staff who volunteered his time to teach these skills has led a team to prepare a compre-
hensive, up-to-date scientific computing textbook. For this effort, Frank Willmore deserves
far more appreciation from scientists and engineers than he is likely to receive, as is usual
for a textbook author. Nonetheless, I have high hopes that this textbook will enrich the
education of many students, and enable many new classes to be taught at universities
nation- and worldwide. Better trained scientists and engineers will use computing more
effectively to do better science and engineering, producing discoveries and innovations in
research, industry, and national security for which we will all benefit—even if we don’t all
appreciate it, all the time.
Jay Boisseau
Vizias Computing
Austin, Texas
Preface
In the fall of 2014, my coeditors Coray Colina, Eric Jankowski, and I organized a group
of sessions on the topic of software engineering for the molecular sciences at the annual
meeting of the American Institute of Chemical Engineers. Soon thereafter, we received
an inquiry from the publisher asking if we would be interested in producing a book on
the topic. I had previously taught a course on Scientific and Technical Computing at the
University of Texas at Austin, and it had been on my mind to produce a book suitable to
accompany that course. My approach was to give my students at least a taste of proficiency
in a variety of topics and to find 20% of the information on key topics that were used 80%
of the time. In editing this volume, we set out to recruit authors who were both sufficiently
knowledgeable in an area and proficient communicators; people who were passionate
about these topics and who could explain them well. We asked these authors to write their
responses from the following perspective: “If you could give a junior researcher only a
single lecture or chapter of notes on a topic, what you would tell them?”
There are two ways in which you can read this book: (1) Cover to cover, and you will gain
enough knowledge to be a well-rounded computational researcher and will have a good
idea of what the computing terrain looks like and will know where to turn; or (2) Choose
chapters as a quick-start guide for various technologies as you discover you need them.
When the time comes to go further, follow the references as a starting point. If you are
already an enthusiastic scientist or engineer and don’t have a lot of experience coding, this
should at least be a quick guide to the tools that your colleagues use and give you enough
of a taste that you can speak with them intelligently about computation.
This is a technology book, and the process of composing it involved a poignant use of
technology. Some chapters were written using MS Word, some with LaTeX, and others
with Google Docs. The journey of writing the book has involved the challenges of manag-
ing a project team across three continents and several time zones. Documents were shared
using Google Drive and meetings were held using Skype. Technology, being a universal
constant in a world that does not share a common language, traditions, and so on, brings
us together.
In any new endeavor, we struggle to find guides, ostensible mentors, as well as known
wizards—people and resources who can and will help us to accomplish what we set out
to do. I owe the deepest debt of gratitude to my team of colleagues at the Texas Advanced
Computing Center. In particular, Luke Wilson and John Lockman were my copilots the
two times I taught the course, and I relied heavily on their depth of experience. A shout out
to Yaakoub El Khamra for his never-ending generosity both at work and in contributing
his chapter on OpenMP; Carlos Rosales for helping me find answers to the many technical
questions; Robert McLay for taking me ever deeper into systems configuration; and John
Cazes for keeping me on track through it all. Finally, I thank my coeditors and contributors
for believing in my project and bringing this book into existence. I could not have done it
without you!
Frank T. Willmore
Level Zero Design
ix
