Undergraduate Matrix Theory Linear Algebra

352 Pages·2015·1.04 MB·English

Undergraduate
Matrix Theory
and
Linear Algebra
2 3
a a (cid:1)(cid:1)(cid:1) a
11 12 1n
6 7
66 a a (cid:1)(cid:1)(cid:1) a 77
6 21 22 2n 7
66 ... ... ... ... 77
4 5
a a (cid:1)(cid:1)(cid:1) a
m1 m2 mn
John S. Alin
Lin(cid:12)eld College
Colin L. Starr
Willamette University
December 15, 2015
ii
Contents
1 SYSTEMS OF LINEAR EQUATIONS 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Systems of Linear Equations and Matrices . . . . . . . . . . . . . . . . . . . 6
1.3 Properties of Matrix Operations . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.4 Equivalent Systems of Equations and Row Operations . . . . . . . . . . . . . 25
1.5 Gaussian Elimination and the Reduced Echelon Form . . . . . . . . . . . . . 31
1.6 Homogeneous Systems of Equations and Solution Sets . . . . . . . . . . . . . 40
1.7 The LU-Factorization (optional) . . . . . . . . . . . . . . . . . . . . . . . . . 44
1.8 Applications - Kirchhoff’s Laws (Optional) . . . . . . . . . . . . . . . . . . . 51
1.9 Applications - Stochastic Matrices (optional) . . . . . . . . . . . . . . . . . . 55
2 VECTOR SPACES 61
2.1 Introduction: Vectors in R2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
2.2 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
2.3 Abstract Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
2.4 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
2.5 Linear Combinations and the Span of a Set of Vectors . . . . . . . . . . . . . 79
2.6 Linear Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
2.7 Basis and Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
2.8 The Rank of a Matrix/Consistency in Systems of Equations . . . . . . . . . 96
2.9 Dimension of Solution Space of Homogeneous Systems . . . . . . . . . . . . 100
3 LINEAR TRANSFORMATIONS AND MATRICES 105
3.1 De(cid:12)nitions and Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.2 Properties of Linear Transformations . . . . . . . . . . . . . . . . . . . . . . 109
3.3 The Matrix of a Linear Transformation . . . . . . . . . . . . . . . . . . . . . 113
3.4 Properties of Matrices and Matrix Multiplication . . . . . . . . . . . . . . . 118
3.5 Change of Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
3.6 Row and Column Operations and Change of Basis . . . . . . . . . . . . . . . 127
4 DETERMINANTS 137
4.1 De(cid:12)nitions and Elementary Properties . . . . . . . . . . . . . . . . . . . . . 137
4.2 Determinant of a Product, Adjoints, etc. . . . . . . . . . . . . . . . . . . . . 143
4.3 Cramer’s Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
4.4 Some Geometrical Aspects of 3-D Vectors (optional) . . . . . . . . . . . . . . 153
iii
iv CONTENTS
4.5 Computational Complexity (optional) . . . . . . . . . . . . . . . . . . . . . . 164
5 SIMILARITY, EIGENVALUES and DIAGONALIZATION 171
5.1 Inner Product, Orthogonality, Gram-Schmidt Process . . . . . . . . . . . . . 171
5.2 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . . . . . . . 179
5.3 Similarity to a Diagonal Matrix . . . . . . . . . . . . . . . . . . . . . . . . . 185
5.4 Symmetric and Hermitian Matrices . . . . . . . . . . . . . . . . . . . . . . . 189
5.5 The Cayley-Hamilton Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 197
5.6 Quadratic Forms (optional) . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
5.7 Applications: Inertia Tensor and Principal Axes (optional) . . . . . . . . . . 211
6 THE JORDAN CANONICAL FORM AND APPLICATIONS 221
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
6.2 The Jordan Canonical Form . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
6.3 Systems of Constant Coefficient ODEs . . . . . . . . . . . . . . . . . . . . . 236
7 Linear Programming 247
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
7.2 The Simplex Method in Equation Form . . . . . . . . . . . . . . . . . . . . . 253
7.3 The Simplex Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
7.4 Applications - Game Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
A Sets 271
B Functions 275
C Fields 279
D Facts about polynomials 283
D.1 Factoring Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
E Mathematical Induction 289
F Additional Proofs 293
Hints, Comments, and Solutions for Selected Exercises 299
Index 343
Chapter 1
SYSTEMS OF LINEAR
EQUATIONS
1.1 Introduction
We are embarking on a tour of matrix theory and linear algebra. These topics have become
extremely important not only in mathematics but also in a wide variety of other disciplines
such as science, engineering, economics, and management. The study of matrices began in
the mid-nineteenth century and came to its present form during the period 1920-1940. We
will not dwell on the history of the subject1.
To some extent, the power of matrix theory comes from the notation that is used. The
ability to treat an array of numbers as a single quantity is a powerful tool and it yields
signi(cid:12)cant results. While the details require hard work, the methods that follow are worth-
while. We will take an abstract approach to these topics in order to make our accumulated
methods and knowledge applicable in a broader scope.This will enable us to (cid:12)ll in some holes
left in previous courses. In several undergraduate courses material is omitted because of the
unavailability of certain aspects of matrix theory. In calculus, it is customary to skip a dis-
cussion of optimizing functions of several variables - this is covered in Section 5.6. Systems
of differential equations are often omitted from a (cid:12)rst course in differential equations because
of difficulties in dealing with the Jordan canonical form - this topic is introduced in Section
6.3. Physics instructors in courses on classical mechanics usually cannot cover the material
on principal axes because of the lack of techniques on diagonalizing symmetric matrices - we
outline this in Section 5.7.
In order to (cid:12)ll in these holes we will need to develop our theory in sufficient generality so
that these problems can be solved. This means that our development must be \general" or
\abstract." Students sometimes have difficulty with abstraction at (cid:12)rst exposure. Working
with axioms, de(cid:12)nitions, theorems, and proofs can be confusing. The mathematician uses
abstraction to simplify (and generalize) an investigation, not to complicate it. An apprecia-
tion of the power of the method of abstraction will be a secondary bene(cid:12)t of this course of
study.
1For an interesting account of the development of this area of mathematics, the reader is referred to A
History of Mathematics by Carl B. Boyer, New York, 1968.
1
2 CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS
Thestudyofmatrixtheoryandlinearalgebrabeginswithinvestigationsintothesolutions
of systems of simultaneous linear equations. Students encounter systems of equations at an
earlypointintheirmathematicalcareers. Forinstance,inhighschoolalgebra,oneencounters
\story" problems such as: The sum of Tom’s age and his older brother Dick’s age is 17 and
the difference is 7. How old is each?
If x represents Dick’s age and y represents Tom’s age, then from the statement of the
problem, we obtain the following system of linear equations:
x + y = 17
(1.1.1)
x (cid:0) y = 7
Adding the equations, we obtain the equation 2x = 24 and we see that x = 12. Substi-
tution into the (cid:12)rst equation gives 12+y = 17, and we obtain y = 5. So, we see that Tom’s
age is 5 and Dick’s age is 12, and we say that the solution of the system of equations 1.1.1 is
x = 12 and y = 5, or we might say that the solution is the pair of numbers (12;5) and that
the solution set is f(12;5)g.
It seems that we have found a rich source of problems. Systems of equations like those
in 1.1.1 can be constructed with more variables and more equations. How do we (cid:12)nd the
solutions? Are there any? Are there many? A lot of questions arise, and we will be able to
answer most of them!
GEOMETRIC CONSIDERATIONS
To begin our investigations, it is worthwhile to consider the geometrical aspects of systems
of equations of the type 1.1.1. Each equation describes a straight line in the xy-plane, and
since the line given by x+y = 17 has slope (cid:0)1 and the one given by x(cid:0)y = 7 has slope
1, the lines are not parallel and so they must intersect (see Figure 1.1.). The solution is the
unique pair of numbers that describe the point of intersection of the lines.
y
17 x + y = 17
x - y = 7
5
x
7 12 17
Figure 1.1: Ages of Dick and Tom
By considering other systems of equations we can see the possibilities for solution sets of
similar systems of linear equations. In the system
1.1. INTRODUCTION 3
x + y = 3
(1.1.2)
x + y = 5;
we see that the lines are parallel (both have slope (cid:0)1) and distinct (see Figure 1.2), and so
the two lines have no points in common. This is another way of saying that the system 1.1.2
has no simultaneous solution or that the solution set is the empty set. Such a system is said
to be inconsistent.
y
5
3 x + y = 5
x + y = 3
x
3 5
Figure 1.2: Parallel Lines
The third possibility is that the lines described by the two equations are exactly the
same. For example, consider the system:
x (cid:0) y = 1
(1.1.3)
(cid:0)2x + 2y = (cid:0)2:
Figure 1.3 shows the graph. Since the two lines are the same, any point on the line is a
solution. It follows that the solution set can be written as f(x;y)jx(cid:0)y = 1g or alternately
f(x;x(cid:0)1)jx is any real numberg. Such a system is said to be redundant.
y
x(cid:0)y = 1
(cid:0)2x+2y = (cid:0)2
1 x
Figure 1.3: Identical Lines
4 CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS
R A
1
I
1
I
10 V 1 I R2 3 R
2 4
2
I
1
B
R
3
Figure 1.4: Circuit Diagram
Notice that system 1.1.1 has exactly one solution, that 1.1.2 has no solutions, and that
1.1.3 has in(cid:12)nitely many solutions.
Systems of linear equations can be found in many disciplines - physics, chemistry, engi-
neering, economics, etc. In courses that involve electricity and magnetism one encounters
systems of simultaneous linear equations that arise from certain electrical networks obeying
Kirchhoff’s laws. Those networks having only voltage sources (batteries, for example) and
resistors will produce a system of linear equations with the currents in each branch of the
networks as the unknowns. Previous experience with Kirchhoff’s laws will not be needed in
the following discussion. We will present an example and this will give the general idea of
how these systems arise.
KIRCHHOFF’S LAWS
Brie(cid:13)y, Kirchhoff’s laws state that around any closed loop in an electrical network, the sum
of the voltages produced by the energy sources in the loop equals the sum of the voltage
drops across the resistors in the loop, and that at any branch point in the network, the sum
of the currents is zero. Using (cid:6)-notation, we can express these laws symbolically by:
∑ ∑
V = IR
loop loop
∑
I = 0;
point
where V represents the voltage, I the current, and R the resistance in a given resistor.
Kirchhoff’s laws will be investigated more fully in Section 1.8; here we will consider one
example. In reality, the orientation of the assigned currents must be considered, as well as
the orientation of the power sources in the circuit. The sum of the voltage drops must be
taken \algebraically;" that is, with allowance for the algebraic sign. Consider the circuit in
Figure 1.4.
By Kirchhoff’s laws, after some rearrangement, the following system of linear equations
is obtained:
1.1. INTRODUCTION 5
Loop 1: R I + R I + R I = 10
1 1 2 2 3 1
Loop 2: (cid:0) R I + R I = 0 (1.1.4)
2 2 4 3
Point A: I (cid:0) I (cid:0) I = 0
1 2 3
Point B produces essentially the same equation as Point A and so it need not be consid-
ered. Let us assume certain speci(cid:12)c values for the resistance; say R = 2;R = 5;R = 3;
1 2 3
and R = 2. Combining the two terms involving I , the system of equations [1.1.4] becomes:
4 1
5I + 5I = 10
1 2
(cid:0) 5I + 2I = 0 (1.1.5)
2 3
I (cid:0) I + I = 0
1 2 3
Dividing the (cid:12)rst equation by 5 and subtracting it from the last equation, we get
I + I = 2
1 2
5I (cid:0) 2I = 0 (1.1.6)
2 3
(cid:0) 2I + I = (cid:0)2
2 3
(Note that we also multiplied the second equation by (cid:0)1.) Now dividing the second
equation by 5, adding twice the second equation to the last, and dividing by (cid:0)2, we obtain
I + I = 2
1 2
I (cid:0) 2I = 0 (1.1.7)
2 5 3
9 I = 1
10 3
Solving for the unknown currents, we obtain I = 10=9;I = 4=9; and I = 14=9:
3 2 1
Section 1.1 Exercises
In Exercises 1-7 solve the systems of equations:
1. (a) x+y = 3 (b) 2x+4y = 6
x(cid:0)y = 2 (cid:0)x+2y = 3
2. (a) 2x+3y = 4 (b) 2x(cid:0)4y = 8
x(cid:0)2y = 8 x+3y = (cid:0)1
3. x + y + z = 4
y (cid:0) z = 6
x + z = 2
4. 2x + y + 2z = 3
x + z = (cid:0)2
(cid:0)x + y (cid:0) z = 6
6 CHAPTER 1. SYSTEMS OF LINEAR EQUATIONS
5. w (cid:0) x + y (cid:0) z = 7
x + 2y + z = 3
y (cid:0) z = 6
y + z = (cid:0)1
6. In the electrical circuit in Figure 1.4, assume that the resistances are given by R =
1
2;R = 3;R = 1; and R = 1. Solve for the currents I ;I ; and I .
2 3 4 1 2 3
7. In the electrical circuit in Figure 1.4, assume that the resistances are given by R =
1
5;R = 10;R = 5; and R = 6. Solve for the currents I ;I ; and I :
2 3 4 1 2 3
8. Find all solutions of: 2x + 3y (cid:0) z = 12
x (cid:0) 2y + 2z = 7:
9. Consider the system of linear equations:
ax + y = 2
x + 2y = (cid:0)3
where x and y are the unknowns and a is some unknown constant. Find values of
a (if possible) for which the system of equations has: a) exactly one solution, b) no
solutions, c) in(cid:12)nitely many solutions.
10. Consider the system of linear equations:
x + y = a
ax + y = 2:
Find the values of a for which the system of equations has: a) no solution, b) a unique
solution. Could the system have in(cid:12)nitely many solutions?
11. The sum of three numbers is 32. The quotient of two of them is 3 and the third number
is 3/5 of the sum of the other two. Find the three numbers.
12. A couple has two children. The youngest was born when the oldest was four, and two
years from now the sum of the children’s ages will be 20. How old is each child?
1.2 Systems of Linear Equations and Matrices
The discussion and examples in Section 1.1 should suggest that systems of linear equations
arise naturally in problems and that it is important to be able to solve such systems. While
the problems in Section 1.1 involved only two or three unknowns, it is not hard to imagine
problems involving many unknowns. For example, a Kirchhoff’s laws problem with 16 loops
could have 36 or more variables. In this section we will discuss general systems of linear
equations and show how a system of linear equations may be regarded as a single \matrix"
equation.