Matrices, transposes, and inverses

Math 40, Introduction to Linear Algebra

Wednesday, February 1, 2012

Matrix-vector multiplication: two views

1st perspective: A
x is linear combination of columns of A

44
11
22 33 11 22
33 4
33 == 44 + 33
+ + 22
+ =
22 11 55 22 11 55 21
22
A
x
2nd perspective: A
x is computed as dot product of rows of A with vector x

4 4
1 2 3 4
3 = 2 4 =
2 1 5 dot product of 1 and 3 21
2 5 2
A
x
Notice that # of columns of A = # of rows of x.
This is a requirement in order for matrix multiplication to be defined.
Matrix multiplication

What sizes of matrices can be multiplied together?

the matrix product AB
For m x n matrix A and n x p matrix B,
is an m x p matrix.
mxn nxp
inner
parameters
must match
outer parameters become
parameters of matrix AB

If A is a square matrix and k is a positive integer, we define

Ak = A A A
k factors

Properties of matrix multiplication

Most of the properties that we expect to hold for matrix multiplication do....

A(B + C) = AB + AC
(AB)C = A(BC)
k(AB) = (kA)B = A(kB) for scalar k

.... except commutativity!!

In general, AB = BA.
Matrix multiplication not commutative
Problems with hoping AB and BA are equal:
In general,
BA may not be well-defined. AB = BA.
(e.g., A is 2 x 3 matrix, B is 3 x 5 matrix)

Even if AB and BA are both defined, AB and BA may not be

the same size.
(e.g., A is 2 x 3 matrix, B is 3 x 2 matrix)

Even if AB and BA are both defined and of the same size, they
still may not be equal.

1 1 1 2 2 4 3 3 1 2 1 1
= = =
1 1 1 2 2 4 3 3 1 2 1 1

Truth or fiction?

Question 1 For n x n matrices A and B, is

A2 B 2 = (A B)(A + B) ?

No!! (A B)(A + B) = A2 + AB BA B 2

=0

Question 2 For n x n matrices A and B, is (AB)2 = A2 B 2 ?

No!! (AB)2 = ABAB = AABB = A2 B 2

Matrix transpose
Definition The transpose of an m x n matrix A is the n x m matrix
AT obtained by interchanging rows and columns of A,
i.e., (AT )ij = Aji i, j.

Example

1 5
1 3 5 2 3 3
A= AT =
5 3 2 1 5 2
Transpose operation can be viewed as 2 1
flipping entries about the diagonal.

Definition A square matrix A is symmetric if AT = A.

Properties of transpose
apply twice -- get back
(1) (AT )T = A to where you started

(2) (A + B)T = AT + B T

(3) For a scalar c, (cA)T = cAT

(4) (AB)T = B T AT
To prove this, we show that
[(AB)T ]ij =
Exercise ..
.
Prove that for any matrix A, ATA is symmetric.
= [(B T AT )]ij
Special matrices

Definition A matrix with all zero entries is called 0 0 0 0
A = 0 0 0 0
a zero matrix and is denoted 0.
0 0 0 0

Definition A square matrix is upper-triangular

2 1
4 5
if all entries below main diagonal are zero. A = 0 6 0
0 0 3
analogous definition for a lower-triangular matrix
3
8 0 0 0
Definition A square matrix whose off-diagonal 0 2 0 0
A=
0

entries are all zero is called a diagonal matrix. 0 4 0
0 0 0 1

Definition The identity matrix, denoted In, is the
1 0 0
I3 = 0 1 0
n x n diagonal matrix with all ones on the diagonal.
0 0 1

Identity matrix

Definition The identity matrix, denoted In, is the
1 0 0

I3 = 0 1 0
n x n diagonal matrix with all ones on the diagonal.
0 0 1

Important property
of identity matrix
If A is an m x n matrix, then
ImA = A and AIn = A.
If A is a square matrix, then
IA = A = AI.
TheThe notion
inverse
The of aofof
inverse inverse
matrix
a matrix
Exploration Lets think
Lets about inverses first in theincontext of real num-num-
Exploration
Exploration Consider theabout
think set ofinverses
real numbers,
first andcontext
the say that we have
of real
bers. Say we have equation
the
bers.equation
Say we have equation
3x = 3x23x =
=2
2
and weand
wantwetowant to solve
solve for
x. for x. so, multiply both sides by What1
todo we do?
and we want to for
solve To x.
do To do so, multiply both sides3 by 1obtain
3 to obtain
We multiply both 1 sides 1 of1 the 1equation by 12 to obtain
(3x) =(3x)(2)= (2) = = x = 3x. = 2 .
3 13 3 31 3 3 2
For R, For
1 (3x) = (2) of =
1 =
x1 (3)
1. = .1.
3 isR,
the13 multiplicative 3 inverseinverse
is the multiplicative 3of 3 since 3 (3) =
3 since 3
multiplicative inverse 3
1
of 3 since (3) = 1
Now consider 3 following
the
Now consider systemsystem
the following of of NoticeNotice that that
we can we rewrite
can rewrite thesethese
The Now,
equationsinverse
equations
The consider
inverse ofthea linear
matrix
of a system
matrix Notice
equations as weascan rewrite equations as
equations
that

Exploration 3x 3x
5x
Lets 2 Lets think about inverses first5
= 5x
6
think =
about6 inverses first 3
in the 3 5 x xof1 real 6 num-6
1
Exploration 1 2
incontext
the 1context
= =of real num-
bers. Say we
2x have+2x3xequation
+= 3x
1 = 1 2 2
3 3x2 x2 1 1
bers. Say 1 we have 1
2 2
equation
3x = 3x 2 =2 A A x x b b
which we which
wantwetowant solve tofor
solvex for
and x x and
. x .
Howfordox.we 1 isolate 2 the vector x bysides
itselfby on1 LHS?
1 2
and weand want wetowant solveto solve for To x. doTo so,do multiply both
so, multiply both sides 3 byto 1obtain
x 1
x1 3 to obtain
How
How do we isolate do we isolate
1 the 1vector the vector
1 x1= x by x = by
itself
2 on itselfthe LHS?LHS?
on the
(3x) =(3x)(2)= (2) = 2 = x 2
x = x. = . 2
3 3 3 3 3 3 3

1 2 5
For R, Multiply
Multiply 1
For
3 is the
R,
both both
1sides
is theofsides
multiplicative
matrixof matrix
multiplicativeinverse
equation equation
of
inverse 3
by since
of by
3
3
2 3
5
3 (3)
since = (3): = 1.
1 1.
: 3

3
3

The notion 3 5 of inverse

3 5 3 5 x1 3 5 x 1 3=
3 56 6
5
Now consider the following
Now consider 3 system
the following
2 of of
system =
NoticeNotice that2 we 3
that can
we rewrite
can rewrite thesethese
2 3 2 32 x32 x2 2 3 1 1
equationsequations
Now, consider the linear system equations as we
equations ascan rewrite equations as
1 0 Notice x that 13
1 0 x1 1
= 3
13
3x1 3x 5x12 = 5x
62=6 0 1 =x2 3 5 9 5 x1 x1 6 6
0 1 x2 9 = =
2x1 + 2x3x12 + 12 = 1
= 3x I 2 2 3 3x2 x2 1 1
I
x1 A13A x x b b
which wetowant to solve 1 for x1xand xx1 2. =
13
which we want solve
How for do x we and
isolate 2 . the
vector
=x 2 x by9 itself on LHS?
x 9
x
the vectorxx1= by itself on the
2 1
How doHow we do we isolate
isolate the vector 3 x 5 = x by itself on the LHS?LHS? 6
? x2 1 x 2 = ?
Thus, the solution to () is 2x1 = 313 x and2 x 2 = 9.
9.
1
Thus, the solution to () is x 1 = 13
both sides ofmatrix equation and x
2 =
3 5
Multiply Multiply
bothwant sides ofequal
matrix equation by I 3 by5 2 3 :
this to identity matrix, 2 3 :
Lecture
Math

40,Spring

7
3 5 3 5 12, x 1
Kindred
Prof.
3 56 6 Page 1
Lecture 7 3 5 Math3 40, 5 Spring 12,x1 3 5
Prof. Kindred3= 5 Page 1
2 3 =
2 3 2 32 x32 2x2 3 2 3 2 3 1
1

13
1 0 1x01 x1 13 =
x1 1 0 x01 1 =x3 5 9 6 13
= 0 1 x2 = 29 3 1 = 9
2
x2 0 1 x2 I
I
x1 13
x1 =
13
=x2 9
x2 9
Matrix inverses

Definition A square matrix A is invertible (or nonsingular) if matrix

B such that AB = I and BA = I. (We say B is an inverse of A.)

Example

2 7 4 7
A= is invertible because for B = ,
1 4 1 2

2 7 4 7 1 0
we have AB = = =I
1 4 1 2 0 1

4 7 2 7 1 0
and likewise BA = = = I.
1 2 1 4 0 1

The notion of an inverse matrix only applies to square matrices.

- For rectangular matrices of full rank, there are one-sided inverses.

- For matrices in general, there are pseudoinverses, which are a generalization to matrix inverses.

1 1
Example Find the inverse of A = . We have
1 1

1 1 a b 1 0 a+c b+d 1 0
= = =
1 1 c d 0 1 a+c b+d 0 1
= a + c = 1 and a + c = 0 Impossible!

1 1
Therefore, A = is not invertible (or singular).
1 1

Take-home message: Not all square matrices are invertible.

Lecture 7 Math 40, Spring 12, Prof. Kindred Page 1

Important questions:
When does a square matrix have an inverse?
If it does have an inverse, how do we compute it?
Can a matrix have more than one inverse?

Theorem. If A is invertible, then its inverse is unique.

Proof. Assume A is invertible. Suppose, by way of contradiction, that the
inverse of A is not unique, i.e., let B and C be two distinct inverses of A.
Then, by defn of inverse, we have
BA = I = AB (1)
and CA = I = AC. (2)
It follows that
B = BI by defn of identity matrix
= B(AC) by (2) above
= (BA)C by associativity of matrix mult.
= IC by (1) above
= C. by defn of identity matrix
Thus, B = C, which contradicts the previous assumption that B = C.
So it must be that case that the inverse of A is unique.

The inverse of a matrix A is unique,

Take-home message:
and we denote it A1.

Theorem (Properties of matrix inverse).

(a) If A is invertible, then A1 is itself invertible and (A1)1 = A.

Lecture 7 Math 40, Spring 12, Prof. Kindred Page 2

(b) If A is invertible and c = 0 is a scalar, then cA is invertible and
(cA)1 = 1c A1.

(c) If A and B are both n n invertible matrices, then AB is invertible

and (AB)1 = B 1A1.
socks and shoes rule similar to transpose of AB
generalization to product of n matrices
(d) If A is invertible, then AT is invertible and (AT )1 = (A1)T .

To prove (d), we need to show that there is some matrix such that
AT = I and AT = I.

Proof of (d). Assume A is invertible. Then A1 exists and we have

(A1)T AT = (AA1)T = I T = I
and
AT (A1)T = (A1A)T = I T = I.
So AT is invertible and (AT )1 = (A1)T .

Question: If A and B are invertible n n matrices,

what can we say about A + B?

There is no guarantee A + B is invertible even if A and B themselves are

invertible! In other words, we cannot say that (A + B)1 = A1 + B 1.

How do we compute the inverse of a matrix, if it exists?

Lecture 7 Math 40, Spring 12, Prof. Kindred Page 3

Inverse of a 2 2 matrix: Consider the special case where A is a 2 2
matrix with A = [ ac db ]. If ad bc = 0, then A is invertible and its inverse is

1 d b
A1 = .
ad bc c a

Exercise: Check that AA1 = [ 10 01 ] = A1A.

2 1
Example For A = , we have
3 3
1

1 3 1 1
A1 = = 3 .
3 3 2 1 23
We can easily check that
1

2 1 1 3 1 0
AA1 = 2 =
3 3 1 3 0 1
and
1
1 2 1 1 0
A1A = 3 = .
1 23 3 3 0 1

How do we find inverses of matrices that are larger than 2 2 matrices?

Theorem. If some EROs reduce a square matrix A to the identity matrix

I, then the same EROs transform I to A1.

EROs -1
A I I A

If we can transform A into I, then we will obtain A1. If we cannot do so,

then A is not invertible.

Lecture 7 Math 40, Spring 12, Prof. Kindred Page 4

 1 3 1
Example: Find the inverse of the matrix A = 3 6 0 .
1 0 1

1 3 1 1 0 0 1 3 1 1 0 0
R2 +3R1
3 6 0 0 1 0 0 3 3 3 1 0
R3 +R1
1 0 1 0 0 1 0 3 2 1 0 1

1 3 1 1 0 0
R1
0 3 3 3 1 0
R3 R2
0 0 1 2 1 1

1 0 2 2 1 0
R +R2
1 0 3 3 3 1 0
R3
0 0 1 2 1 1

1 0 2 2 1 0
13 R2
0 1 1 1 13 0
0 0 1 2 1 1

1 0 0 2 1 2
R1 2R3
0 1 0 1 23 1
R2 +R3
0 0 1 2 1 1
Thus, A is invertible and its inverse is

2 1 2
A1 = 1 23 1 .
2 1 1

Why does this work? = discussion next class

Lecture 7 Math 40, Spring 12, Prof. Kindred Page 5