Table Of ContentTHE LINEAR ALGEBRA A BEGINNING GRADUATE STUDENT
OUGHT TO KNOW
The Linear Algebra a Beginning
Graduate Student Ought to Know
Second Edition
by
JONATHAN S. GOLAN
University of Haifa, Israel
AC.I.P. Catalogue record for this book is available from the Library of Congress.
ISBN-10 1-4020-5494-7 (PB)
ISBN-13 978-1-4020-5494-5 (PB)
ISBN-10 1-4020-5495-5 (e-book)
ISBN-13 978-1-4020-5495-2 (e-book)
Published by Springer,
P.O. Box 17, 3300 AADordrecht, The Netherlands.
www.springer.com
Printed on acid-free paper
All Rights Reserved
© 2007 Springer
No part of this work may be reproduced, stored in a retrieval system, or transmitted in any form or by
any means, electronic, mechanical, photocopying, microfilming, recording or otherwise, without written
permission from the Publisher, with the exception of any material supplied specifically for the purpose
of being entered and executed on a computer system, for exclusive use by the purchaser of the work.
To my grandsons: Shachar, Eitan, and Sarel
(cid:157)(cid:159)(cid:172) (cid:167)(cid:165)(cid:161)(cid:163)(cid:159) (cid:167)(cid:165)(cid:161)(cid:166) (cid:169)(cid:180)
(' , )
(cid:162)(cid:177)(cid:178)(cid:174) (cid:163)(cid:166)(cid:179)(cid:168)
Contents
1 Notation and terminology 1
2 Fields 5
3 Vector spaces over a field 17
4 Algebras over a field 33
5 Linear independence and dimension 49
6 Linear transformations 79
7 The endomorphism algebra of a vector space 99
8 Representation of linear transformations by matrices 117
9 The algebra of square matrices 131
10 Systems of linear equations 169
11 Determinants 199
12 Eigenvalues and eigenvectors 229
13 Krylov subspaces 267
vii
viii Contents
14 The dual space 285
15 Inner product spaces 299
16 Orthogonality 325
17 Selfadjoint Endomorphisms 349
18 Unitary and Normal endomorphisms 369
19 Moore-Penrose pseudoinverses 389
20 Bilinear transformations and forms 399
A Summary of Notation 423
Index 427
For whom is this book written?
Crow’s Law: Do not think what you want to think until you
know what you ought to know.1
Linear algebra is a living, active branch of mathematical research which
is central to almost all other areas of mathematics and which has impor-
tant applications in all branches of the physical and social sciences and in
engineering. However, in recent years the content of linear algebra courses
required to complete an undergraduate degree in mathematics – and even
moresoinotherareas–atallbutthemostdedicateduniversities,hasbeen
depleted to the extent that it falls far short of what is in fact needed for
graduatestudyandresearchorforreal-worldapplication. Thisistruenot
onlyintheareasoftheoreticalworkbutalsointheareasofcomputational
matrix theory, which are becoming more and more important to the work-
ingresearcheraspersonalcomputersbecomeacommonandpowerfultool.
Students are not only less able to formulate or even follow mathematical
proofs, they are also less able to understand the underlying mathematics
of the numerical algorithms they must use. The resulting knowledge gap
has led to frustration and recrimination on the part of both students and
faculty alike, with each silently – and sometimes not so silently – blaming
the other for the resulting state of affairs. This book is written with the
intention of bridging that gap. It was designed be used in one or more of
several possible ways:
(1) As a self-study guide;
(2) As a textbook for a course in advanced linear algebra, either at the
upper-class undergraduate level or at the first-year graduate level; or
(3) As a reference book.
It is also designed to be used to prepare for the linear algebra portion of
prelim exams or PhD qualifying exams.
This volume is self-contained to the extent that it does not assume any
previous knowledge of formal linear algebra, though the reader is assumed
tohavebeenexposed,atleastinformally,tosomebasicideasortechniques,
such as matrix manipulation and the solution of a small system of linear
equations. It does, however, assume a seriousness of purpose, considerable
1This law, attributed to John Crow of King’s College, London, is quoted by R. V.
JonesinhisbookMost Secret War.
ix
x For whom is this book written?
motivation, and modicum of mathematical sophistication on the part of
the reader.
The book also contains a large number of exercises, many of which are
quite challenging, which I have come across or thought up in over thirty
years of teaching. Many of these exercises have appeared in print before,
in such journals as American Mathematical Monthly, College Mathemat-
ics Journal, Mathematical Gazette, or Mathematics Magazine, in various
mathematicscompetitionsorcirculatedproblemcollections,orevenonthe
internet. Some were donated to me by colleagues and even students, and
some originated in files of old exams at various universities which I have
visited in the course of my career. Since, over the years, I did not keep
track of their sources, all I can do is offer a collective acknowledgement to
allthosetowhomitisdue. Goodproblemformulators,liketheGodofthe
abbot of Citeaux, know their own. Deliberately, difficult exercises are not
marked with an asterisk or other symbol. Solving exercises is an integral
part of learning mathematics and the reader is definitely expected to do
so, especially when the book is used for self-study.
Solving a problem using theoretical mathematics is often very differ-
ent from solving it computationally, and so strong emphasis is placed on
the interplay of theoretical and computational results. Real-life imple-
mentation of theoretical results is perpetually plagued by errors: errors in
modelling, errors in data acquisition and recording, and errors in the com-
putationalprocessitselfduetoroundoffandtruncation. Therearefurther
constraints imposed by limitations in time and memory available for com-
putation. Thusthemosteleganttheoreticalsolutiontoaproblemmaynot
lead to the most efficient or useful method of solution in practice. While
no reference is made to particular computer software, the concurrent use
of a personal computer equipped symbolic-manipulation software such as
Maple, Mathematica, Matlab or MuPad is definitely advised.
In order to show the “human face” of mathematics, the book also in-
cludes a large number of thumbnail photographs of researchers who have
contributed to the development of the material presented in this volume.
Acknowledgements. Most of the first edition this book was written
while the I was a visitor at the University of Iowa in Iowa City and at the
University of California in Berkeley. I would like to thank both institu-
tions for providing the facilities and, more importantly, the mathematical
atmosphere which allowed me to concentrate on writing. This edition was
extensively revised after I retired from teaching at the University of Haifa
in April, 2004.
I have talked to many students and faculty members about my plans for
this book and have obtained valuable insights from them. In particular, I
would like to acknowledge the aid of the following colleagues and students
who were kind enough to read the preliminary versions of this book and
For whom is this book written? xi
offer their comments and corrections: Prof. Daniel Anderson (University
ofIowa),Prof.AdiBen-Israel(RutgersUniversity),Prof.RobertCacioppo
(Truman State University), Prof. Joseph Felsenstein (University of Wash-
ington),Prof.RyanSkipGaribaldi(EmoryUniversity),Mr.GeorgeKirkup
(University of California, Berkeley), Prof. Earl Taft (Rutgers University),
Mr. Gil Varnik (University of Haifa).
Photo credits. The photograph of Dr. Shmuel Winograd is used with
the kind permission of the Department of Computer Science of the City
UniversityofHongKong. ThephotographsofProf.Ben-Israel,Prof.Blass,
Prof. Kublanovskaya, and Prof. Strassen are used with their respective
kind permissions. The photograph of Prof. Greville is used with the
kind permission of Mrs. Greville. The photograph of Prof. Rutishauser
is used with the kind permission of Prof. Walter Gander. The photograph
of Prof. V. N. Faddeeva is used with the kind permission of Dr. Vera Si-
monova. The photograph of Prof. Zorn is used with the kind permission
of his son, Jens Zorn. The photograph of J. W. Givens was taken from
a group photograph of the participants at the 1964 Gatlinburg Conference
on Numerical Algebra. All other photographs are taken from the MacTu-
tor History of Mathematics Archive website (http://www-history.mcs.st-
andrews.ac.uk/history/index.html),theportraitgalleryofmathematicians
at the Trucsmatheux website (http://trucsmaths.free.fr/), or similar web-
sites. Tothebestknowledgeofthemanagersofthosesites,andtothebest
of my knowledge, they are in the public domain.
1
Notation and terminology
Setswillbedenotedbybraces, { }, betweenwhichwewilleitherenumer-
atetheelementsofthesetorgivearulefordeterminingwhethersomething
is an element of the set or not, as in {x|p(x)}, which is read “the set of
all x suchthat p(x)”. If a isanelementofaset A wewrite a∈A; if
itisnotanelementof A, wewrite a∈/ A. Whenoneenumeratestheele-
mentsofaset,theorderisnotimportant. Thus {1,2,3,4} and {4,1,3,2}
both denote the same set. However, we often do wish to impose an order
on sets the elements of which we enumerate. Rather than introduce new
andcumbersomenotationtohandlethis,wewillmaketheconventionthat
when we enumerate the elements of a finite or countably-infinite set, we
willassumeanimpliedorder,readingfromlefttoright. Thus,theimplied
order on the set {1,2,3,...} is indeed the usual one. The empty set,
namely the set having no elements, is denoted by ∅. Sometimes we will
usetheword“collection”asasynonymfor“set”,generallytoavoidtalking
about “sets of sets”.
A finite or countably-infinite selection of elements of a set A is a list.
Members of a list are assumed to be in a definite order, given by their
indices or by the implied order of reading from left to right. Lists are
usually written without brackets: a ,...,a , though, in certain contexts,
1 n
it will be more convenient to write them as ordered n-tuples (a ,...,a ).
1 n
Note that the elements of a list need not be distinct: 3,1,4,1,5,9 is a list
of six positive integers, the second and fourth elements of which are equal
to 1. A countably-infinite list of elements of a set A is also often called
1
