Table Of ContentMatrix Algebra: Exercises and Solutions
Springer Science+Business Media, LLC
David A. Harville
Matrix Algebra:
Exercises and Solutions
" Springer
David A. Harville
Mathematical Sciences Department
IBM TJ. Watson Research Center
Yorktown Heights, NY 10598-0218
USA
Library ofCongress Cataloging-in-Publieation Data
Harville, David A.
Matrix algebra: exereises and solutions / David A. Harville.
p. em.
lncludes bibliographieal referenees and index.
ISBN 978-0-387-95318-2 ISBN 978-1-4613-0181-3 (eBook)
DOI 10.1007/978-1-4613-0181-3
1. Matrices-Problems, exereises, etc. I. Title.
QAI88 .H38 2001
519.9'434--iic21 2001032838
Printed on acid-free paper.
© 2001 Springer Science+Business Media New York
Originally published by Springer-Verlag New York, Ine in 2001
AII rights reserved. This work may not be translated or copied in whole or in part without the written
permission of the publisher Springer Science+Business Media, LLC, except for brief excerpts
inconnection with reviews or scholarly analysis. Use in connection with any form of information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed is forbidden,
The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the
former are not especially identified, is not to be taken as a sign that such names, as understood by the
Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.
Production managed by Yong-Soon Hwang; manufacturing supervised by Jeffrey Taub.
Photocomposed copy prepared from the author's LaTeX file.
9 8 765 432 I
ISBN 978-0-387-95318-2
Preface
This book comprises well over three-hundred exercises in matrix algebra and their
solutions. The exercises are taken from my earlier book Matrix Algebra From a
Statistician's Perspective. They have been restated (as necessary) to make them
comprehensible independently of their source. To further insure that the restated
exercises have this stand-alone property, I have included in the front matter a
section on terminology and another on notation. These sections provide definitions,
descriptions, comments, or explanatory material pertaining to certain terms and
notational symbols and conventions from Matrix Algebra From a Statistician's
Perspective that may be unfamiliar to a nonreader of that book or that may differ in
generality or other respects from those to which he/she is accustomed. For example,
the section on terminology includes an entry for scalar and one for matrix. These
are standard terms, but their use herein (and in Matrix Algebra From a Statistician's
Perspective) is restricted to real numbers and to rectangular arrays of real numbers,
whereas in various other presentations, a scalar may be a complex number or more
generally a member of a field, and a matrix may be a rectangular array of such
entities.
It is my intention that Matrix Algebra: Exercises and Solutions serve not only
as a "solution manual" for the readers of Matrix Algebra From a Statistician's
Perspective, but also as a resource for anyone with an interest in matrix algebra
(including teachers and students of the subject) who may have a need for exercises
accompanied by solutions. The early chapters of this volume contain a relatively
small number of exercises-in fact, Chapter 7 contains only one exercise and
Chapter 3 only two. This is because the corresponding chapters of Matrix Alge
bra From a Statistician's Perspective cover relatively standard material, to which
many readers will have had previous exposure, and/or are relatively short. It is
vi Preface
the final ten chapters that contain the vast majority of the exercises. The topics
of many of these chapters are ones that may not be covered extensively (if at all)
in more standard presentations or that may be covered from a different perspec
tive. Consequently, the overlap between the exercises from Matrix Algebra From
a Statistician's Perspective (and contained herein) and those available from other
sources is relatively small.
A considerable number of the exercises consist of verifying or deriving results
supplementary to those included in the primary coverage of Matrix Algebra From
a Statistician's Perspective. Thus, their solutions provide what are in effect proofs.
For many of these results, including some of considerable relevance and interest
in statistics and related disciplines, proofs have heretofore only been available (if
at all) through relatively high-level books or through journal articles.
The exercises are arranged in 22 chapters and within each chapter, are numbered
successively (starting with 1). The arrangement, the numbering, and the chapter
titles match those in Matrix Algebra From a Statistician's Perspective. An exercise
from a different chapter is identified by a number obtained by inserting the chapter
number (and a decimal point) in front of the exercise number.
A considerable effort was expended in designing the exercises to insure an
appropriate level of difficulty-the book Matrix Algebra From a Statistician's
Perspective is essentially a self-contained treatise on matrix algebra, however it
is aimed at a reader who has had at least some previous exposure to the subject
(of the kind that might be attained in an introductory course on matrix or linear
algebra). This effort included breaking some of the more difficult exercises into
relatively palatable parts and/or providing judicious hints.
The solutions presented herein are ones that should be comprehensible to those
with exposure to the material presented in the corresponding chapter of Matrix
Algebra From a Statistician's Perspective (and possibly to that presented in one
or more earlier chapters). When deemed helpful in comprehending a solution,
references are included to the appropriate results in Matrix Algebra From a Statis
tician's Perspective-unless otherwise indicated a reference to a chapter, section,
or subsection or to a numbered result (theorem, lemma, corollary, "equation",
etc.) pertains to a chapter, section, or subsection or to a numbered result in Matrix
Algebra From a Statistician's Perspective (and is made by following the same con
ventions as in the corresponding chapter of Matrix Algebra From a Statistician's
Perspective). What constitutes a "legitimate" solution to an exercise depends of
course on what one takes to be "given". If additional results are regarded as given,
then additional, possibly shorter solutions may become possible.
The ordering of topics in Matrix Algebra From a Statistician's Perspective is
somewhat nonstandard. In particular, the topic of eigenvalues and eigenvectors is
deferred until Chapter 21, which is the next-to-Iast chapter. Among the key results
on that topic is the existence of something called the spectral decomposition. This
result if included among those regarded as given, could be used to devise alternative
solutions for a number of the exercises in the chapters preceding Chapter 21.
However, its use comes at a "price"; the existence of the spectral decomposition
can only be established by resort to mathematics considerably deeper than those
Preface vii
underlying the results of Chapters 1-20 in Matrix Algebra From a Statistician's
Perspective.
I am indebted to Emmanuel Yashchin for his support and encouragement in the
preparation of the manuscript for Matrix Algebra: Exercises and Solutions. I am
also indebted to Lorraine Menna, who entered much of the manuscript in IhTPC,
and to Barbara White, who participated in the latter stages of the entry process.
Finally, I wish to thank John Kimmel, who has been my editor at Springer-Verlag,
for his help and advice.
Contents
Preface v
Some Notation ........................................................................................... xi
Some Terminology xvii
1 Matrices
2 Submatrices and Partitioned Matrices 7
3 Linear Dependence and Independence 11
4 Linear Spaces: Rowand Column Spaces 13
5 Trace of a (Square) Matrix 19
6 Geometrical Considerations 21
7 Linear Systems: Consistency and Compatibility ............................ .. 27
8 Inverse Matrices 29
9 Generalized Inverses 35
10 Idempotent Matrices 49
x Contents
11 Linear Systems: Solutions ........... ...................................................... 55
12 Projections and Projection Matrices 63
13 Determinants ...................................................................................... 69
14 Linear, Bilinear, and Quadratic Forms 79
15 Matrix Differentiation 113
16 Kronecker Products and the Vec and Vech Operators 139
17 Intersections and Sums of Subspaces 161
18 Sums (and Differences) of Matrices 179
19 Minimization of a Second-Degree Polynomial (in n Variables)
Subject to Linear Constraints ........................................................... 209
20 The Moore-Penrose Inverse 221
21 Eigenvalues and Eigenvectors 231
22 Linear Transformations 251
References 265
Index 267
Some Notation
{xd A row or (depending on the context) column vector whose ith element
is Xi
{aij} A matrix whose ijth element is aij (and whose dimensions are arbitrary
or may be inferred from the context)
A' The transpose of a matrix A
AP The pth (for a positive integer p) power of a square matrix A; i.e., the
matrix product AA ... A defined recursively by setting A 0 = I and taking
Ak = AAk- 1 (k = 1, ... , p)
C(A) Column space of a matrix A
R(A) Row space of a matrix A
Rmxn The linear space comprising all m x n matrices
Rn The linear space Rnx I comprising all n-dimensional column vectors
or (depending on the context) the linear space R I xn comprising all n
dimensional row vectors
speS) Span of a finite set S of matrices; Sp({AI, ... , Ad), which represents
the span of the set {AI, ... , Ad comprising the k matrices AI, ... , Ab
is generally abbreviated to Sp(AI, ... , Ak)
C Writing SeT (or T ~ S) indicates that a set S is a (not necessarily
proper) subset of a set T
dim (V) Dimension of a linear space V
rank A The rank of a matrix A
rank T The rank of a linear transformation T
