System of Linear Equations

Friday, May 15, 2015 1:01 PM

System of Linear Equations Matrix Notation

1. Coeff. Matrix: (I) →

(I)

2. Augmented Matrix: (I) →

Variables
: Coeff.
: Const. terms

Solution possibilities:
○ None (Inconsistent) Elementary Row Operation (EROs)
○ Unique (Consistent)
○ Many (Consistent) 1. Row Replacement crj + ri
Row(i) := Row(i) + c × Row(j)

2. Row Interchange ri ↔ rj
Swap the position of Row(i) & Row(j)

3. Row scaling cri

Row(i) := c × Row(i)

THM. 1:
• 2 matrix are row-equivalent if they differ by a sequences
of EROs
• Linear sys. corresponding to row-equiv. augmented
matrix will have same sol. set

MATH 2111 Page 1

Row Echelon Form (REF & RREF)
Saturday, May 16, 2015 11:40 AM

Definitions Row Reduce Algorithm

1. Leading entry: Phase 1: Forward elimination → REF

Leftmost non-zero entry in a non-zero row 1. Stop when all visible entries zeros, or no visible entry

e.g. 2. Locate leftmost non-zero col (pivot col.)

2. Row Echelon Form (REF):
○ All zero rows are in bottom of matrix
○ Leading entry moves to right by at least 1 col when
going down a row
○ All entries in a col below leading entry are zeros
e.g.
Select row containing non-zero entry in that col
Use Row Interchange to move it to top
3. Use row replacement to make all entries below selected entry
(pivot pos.) be 0
4. Cover selected row
Repeat step 1 - 3 for submatrix below this row

Phase 2: Backward elimination REF → RREF

3. Reduced Row Echelon Form (RREF): 5. Use Row Scaling to scale all leading entries to 1
○ All leading entries are 1
○ Each leading entry 1 is the only non-zero entry in its 6. From rightmost leading entries to left:
column Use Row Replacement to make all entries above them 0

e.g.
e.g.

THM. 2: Each matrix is row-equiv. to one & only one

RREF

4. Pivot position:
Location corresponding to leading 1 in RREF of a
matrix

5. Pivot column:
Column containing pivot pos.

e.g.

Pivot cols = 1, 2, 4

MATH 2111 Page 2

Linear System Consistency
Saturday, May 16, 2015 12:29 PM

THM. 3 Basic & Free variables

Linear sys. is consistent • Basic var. correspond to pivot col

⇔ Last col. of augmented matrix is NOT pivot col • Free var correspond to non-pivot col

e.g. NOTE: Common practice is assigning free param. to free var

e.g.

Augmented matrix:

Consistency Basic var.: x1, x2, x4 ; Free var.: x3

⇔ Last col is NOT pivot col Solution (assign free param. To free var.):

Rank of Matrix
No. of pivot pos. in matrix

THM. 4:
○ Linear sys. [A | b] is consistent ⇔ RankA = Rank[A | b]
○ [A | b] is consistent with n var.:
 No. of basic var. = rankA
 No. of free var. = n - rankA

MATH 2111 Page 3

Linear Concepts: Combination, Span &
Independence
Tuesday, May 12, 2015 5:04 PM

Linear Combination (l.c.) Linear Independence & Dependence

Equation:
: l.c. of vectors in S
Linear Independent (l.i.)
⇔ Equation has unique zero solution:

Linear Dependent (l.d.)

Span
⇔ Equation has non-zero solution:

THM. 6:

THM. 5:
○ If S is l.d.
○ If p > n ⇒ S is l.d.
⇔ Augment matrix is consistent
(⇔ Last col. is NOT pivot)

MATH 2111 Page 4

Matrix-Vector Product
Saturday, May 16, 2015 3:47 PM

Definition THM. 7

(m×n matrix) (m×n matrix)

Following equations will have same sol. set (x1, …, xn):

a.
b.
c. Linear sys. with augmented matrix:
e.g.

THM. 8
A: m×n matrix
Followings are equivalent:
a. consistent

"Linear" Properties b. Each is l.c of A's cols

c. A's cols span Rm
A: m×n matrix d. A has pivot pos. in EVERY ROWS

a.
b.

MATH 2111 Page 5

Homogeneous & Non-homogeneous System
Saturday, May 16, 2015 4:36 PM

Homogeneous System Non-homogeneous System

A: m×n matrix ; A: m×n matrix ;

: Trivial solution
: Non-trivial solution THM. 9:
: Consistent sys, with sol.
: General sol (associated homogeneous sys.)
Possibilities Solution set No. of free var.
Solution set of :
Unique sol 0
Many sol. ≥1 e.g.
: Basic sols.

e.g.

MATH 2111 Page 6

Linear Transformation
Saturday, May 16, 2015 5:26 PM

Matrix Transformation Linear Transformation (in general)

is called linear if:

a.
b.
Domain Rn
Object Or: (a), (b) ⇔

Codomain: Rm e.g.
: Image of under T, i.e.
is NOT linear transformation:
Range of T = S

Standard Matrix of Linear Transformation is NOT linear transformation:

linear transformation

⇔
Standard matrix:
⇔
⇔
NOTE:
⇔

This must be true

However, take a1 = b2 = 0, a2 = b1 = 1, we have k1k2 = 0, which
can't be true

is linear transformation
e.g. Find standard matrix of linear transformation that does:

One-to-one & Onto Properties

Standard matrix:

One-to-one: ⇔
Onto:
NOTE: Some special standard matrix:
• Rotation on R2 by θ anticlockwise:
THM. 10:
Linear transformation (Standard matrix A) is:
• Reflection on R2 about line passing through O & making 1. One-to-one
angle θ with (+) x-axis: ⇔ has only trivial solution
⇔ All A's cols are pivot, all vars are basic
⇔ A's cols are l.i

MATH 2111 Page 7

 ⇔ A's cols are l.i
2. Onto
⇔ A's cols span Rm
⇔ A has pivot pos. on EVERY rows

MATH 2111 Page 8

Matrix Algebra
Friday, May 15, 2015 1:02 PM

Matrix Operators
Basic Matrix Definitions
1. Equality
1. m×n matrix:
⇔

2. Addition
A, B same size m×n
○ Square matrix: Size n×n C = A + B: cij := aij + bij
○ Main diagonal: a11, a22, …, ann
3. Scalar Multiplication
C = kA: cij := k.aij
2. Column matrix:
4. Matrix Multiplication
: m×p matrix
3. Row matrix: : p×n matrix

4. Triangular matrix: Upper/Lower : m×n matrix

○ Upper: All entries below main diagonal = 0

○ Lower: All entries above main diagonal = 0

e.g.

5. Diagonal matrix: Square, Off-diagonal entries = 0

6. Zero matrix:

Individual entry calculation:

7. Identity matrix: (Size: n×n)

5. Power of Matrix:
A0 := In
Properties of Matrix Operators
Ak := A × A × … × A (k times)

i. A + B = B + A
6. Matrix Transpose:
ii. A + (B + C) = (A + B) + C
Changing all rows of A into cols
iii. A + O = A

A: m×n matrix
iv. r(A + B) = rA + rB
AT: n×m matrix: (AT)ij = Aji
(r + s)A = rA + sA
v. r(sA) = (rs)A

MATH 2111 Page 9

 v. r(sA) = (rs)A

vi. A(BC) = (AB)C

vii. A(B + C) = AB + AC
(B + C)A = BA + CA
viii. r(AB) = (rA)B = A(rB)
ix. ImA = A = AIn

x. (AT)T = A
xi. (A + B)T = AT + BT
xii. (rA)T = r(AT)
xiii. (AB)T = BTAT (order REVERSED)

NOTE: For Matrix Multiplication in general:

• NO commutative: AB ≠ BA
• NO cancellation: AB = AC B≠C
• A, B ≠ O but AB = O
• A2 = A but A ≠ I, O

MATH 2111 Page 10

Invertible Matrix
Sunday, May 17, 2015 4:01 PM

Definition Finding Matrix Inverse (THM. 1)

A: n×n (square) matrix A: n×n matrix

C: CA = In = AC If A-1 exists [A | In] → [In | A-1] by EROs
A is called invertible/non-singular
C = A-1: Inverse of A

e.g. e.g.

doesn't exist

NOTE: When A-1 exists:

• A-1 is unique
• A-k = A-1 × A-1 × … × A-1 (k times)

Properties of Invertible Matrices

A invertible Cancellation laws recovered

i. always has unique sol
ii. CA = O C=O
AD = O D=O
iii. C1A = C2A C1 = C2
AD1 = AD2 D 1 = D2

A, B: invertible n×n matrices

iii. (A-1)-1 = A
iv. (order REVERSED)
v. (AT)-1 = (A-1)T

MATH 2111 Page 11

Elementary Matrix
Sunday, May 17, 2015 4:54 PM

Elementary Matrix (E) Sequence of EROs (THM. 3)

Matrix obtained by performing single ERO on Im A → E1A → E2(E1)A → … → (Ep…E2E1)A = B

Im → E1Im → E2(E1)Im → … → (Ep…E2E1)Im = P

THM. 2: A is row equiv. to B: B = PA ; A = P-1B

A: m×n matrix i.e. If [A | Im] → [B | P] by EROs ⇒ B = PA

A → B by ERO
Im → E by same ERO e.g. Find invertible P such that B = PA; B = RREF(A)

B = EA (E must be on the LEFT of A)

A = E-1B (E is invertible)

MATH 2111 Page 12

Determinants
Friday, May 15, 2015 1:02 PM

Definitions Row, Column, Operations & Determinants (THM. 3)

A: n×n (square) matrix A: n×n (square) matrix

Aij: (n-1)×(n-1) submatrix obtained by deleting ith row & When A → B by:
jth col of A i. Row (kri + rj)/Col (kci + cj) Replacement:
Cij = (-1)i+jAij: (i, j)-cofactor detB = detA

• Calculate determinant by Cofactor Expansion ii. Row (ri ↔ rj)/Col (ci ↔ cj) Interchange:
○ Down the kth row: detB = -detA

iii. Row (kri)/Col (kci) Scaling (by factor of k):

○ Down the p col: th detB = k.detA

NOTE: A → REF(A), RREF(A) by EROs

○ det(REF(A)), det(RREF(A)) easily computed (triangular
matrix det = Product of main diagonal entries)
• Simple formula JUST for (2x2), (3x3) determinant:
○ Keep track of changes by EROs Calculate detA from
det(REF(A)), det(RREF(A))

e.g.

det = a1b2c3 + b1c2a3 + c1a2b3

- a3b2c1 - b3c2a1 - c3a2b1

: det = a1b2 - a2b1

e.g.

Calculate by cofactor expansion down row 2:

detA = a21C21 + a22C22 + a23C23

* NOTE: Determinants of some special matrices

• 4×4 Vandermonde Matrix:

Calculate by simple formula:

detA = 1(-1)(-3) + 2.1.2 + 3.1.1 - 2(-1)3 - 1.1.1 - (-3).1.2
= 21
• 4×4 Hilbert matrix:
THM. 4:
A: (n×n) triangular matrix
detA = Product of entries on main diagonal

MATH 2111 Page 13

 Properties of Determinants

A: n×n matrix

i. detAT = detA
ii. det(AB) = detAdetB
iii. det(kA) = kndetA

MATH 2111 Page 14

Cramer's Rule
Monday, May 18, 2015 12:28 AM

(A: n×n matrix)

When detA ≠ 0, the unique sol of sys is:

(i.e. Replace A's ith col by )

MATH 2111 Page 15

Determinants & Area, Volume, Linear Transformation
Monday, May 18, 2015 12:35 AM

Area & Volume Determinants & Linear Transformation (THM. 5)

1. n-dim parallelotope : Linear transformation on R2

Sides
For any parallelogram S in R2:
Area of T(S) = |detA| × Area of S
NOTE:
 n = 2: vol2 = Area of parallelogram
 n = 3: vol3 = Volume of parallelepiped

2. 2D Triangle
(x1, y1), (x2, y2), (x3, y3)

NOTE:
 3 points collinear ⇔ det = 0
 Eq. of straight line passing through (x1, y1), (x2, y2):

3. Tetrahedron
(xi, yi, zi) i = 1, 2, 3, 4

NOTE:
 4 points coplanar ⇔ det = 0
 Eq. of plane passing through (xi, yi, zi), i = 1, 2, 3:

MATH 2111 Page 16

The Invertible Matrix Theorem
Sunday, May 17, 2015 4:54 PM

A: n×n (square) matrix

Following are equivalent:

a. A is invertible
b. AT is invertible
c. detA ≠ 0

d. C (size n×n): CA = In
e. D (size n×n): AD = In
(One-sided inverse is enough)

f. A In (EROs)
g. A has n pivot pos.
h. A's cols are l.i. & span Rn Basis for Rn

i. has only trivial sol

j. consistent

k. is one-to-one
l. is onto

MATH 2111 Page 17

Vector Spaces
Tuesday, May 12, 2015 3:57 PM

Vector Space Examples of Vector Space

1. Components: 1. : Zero vector space

○ Collection of Vector V & Scalar R
○ Vector addition (+) & scalar multiplication (.) 2.

2. Conditions: 3. : Collection of polynomials with degree and real coeff:

i. Closure:

ii. Existence of zero & negative vector:

"Zero vector":
iii. Commutative:
4. Function space: Collection of all real-valued function on a set D
iv. Associative:

v. Distributive:

"Zero vector":
vi. Compatibility:

vii. Normalization:

Examples of Subspace

Subspace H of V 1. is a subpace of V

1. Definition:
2. is a subspace of
○
○ H is a vector space
○ Vector addition & scalar multiplication 3. is a subspace of if
induced from V
4. is a subspace of
2. Checking conditions: function space
i. Closure:

ii. Existence of zero & negative vector:

((iii), (iv), (v), (vi), (vii) automatically valid)

MATH 2111 Page 18

Linear Combination & Span
Tuesday, May 12, 2015 5:04 PM

Linear Combination (l.c.) Span

: l.c. of vectors in S
NOTE:
e.g. Is a l.c. of S contains infinitely many vectors
is collection of all possible l.c. of every finite subset of S

THM. 1:

is a subspace of V

e.g. Is a l.c. of

is not l.c. of S

MATH 2111 Page 19

Linearly Independent/Dependent Set
Tuesday, May 12, 2015 6:58 PM

Definitions Examples

1. Polynomial space
Equation: is l.i:

Linear Independent (l.i.) Equating coeff. of , we get:

⇔ Equation has unique zero solution:

is l.d.:

Linear Dependent (l.d.)

⇔
⇔ Equation has non-zero solution: Equating coeff. of , we get:

NOTE: When S has infinitely many vectors: ⇔

○ S is l.i. ⇔ Every finite subsets of S is l.i
○ S li l.d. ⇔ Exists a finite subset of S being l.d. 2. Function space:
is l.d.:

(with all different) is l.i.:

Eq. Sys. has unique solution

Notice that when , eq. sys. is satisfied

MATH 2111 Page 20

Subspaces of Matrix
Tuesday, May 12, 2015 5:58 PM

Null Space Column Space

A: m × n matrix : m × n matrix

i.e. is solution set of homogenous sys.

NOTE: is a subspace of
NOTE: is a subspace of

THM. 2: THM. 3:
⇔ is consistent

THM 4:
⇔ is onto ⇔ Every A's rows has pivot pos.
e.g.

e.g.

is inconsistent

Row Space

: m × n matrix

i.e.

NOTE: is a subspace of

THM. 5:

e.g.

MATH 2111 Page 21

General Linear Transformation
Tuesday, May 12, 2015 6:53 PM

Definition THM. 6:
kerT is a subspace of V
V, W: 2 vector spaces over scalar R ImT is a subspace of W

is linear if: THM. 7:

a. T is one-to-one ⇔
b. T is onto ⇔

Kernel of T:

Image/Range of T:

NOTE:
• When V = W, T is called a linear operator on V
• When :
○ T is given by matrix transformation
○

MATH 2111 Page 22

Basis
Tuesday, May 12, 2015 11:00 PM

Basis Bases for Null Space

H: Subspace of V When has:

Indexed set is called a basis for H if: ○ Unique zero solution:
i. B is linearly independent ○ Non-zero solution: ( : Basic solution set)
ii. SpanB = H
e.g.
REMARKS:
• B is "minimal" vector collection that can span H
•

e.g.

THM. 8: Basis for NulA:

is n×n invertible matrix
for

Bases for Row Space Basses for Column Space

A→ REF(A) Method 1:
Span(RowA) = Span(REF(A)) Method 2:

B = { Non-zero rows of REF(A) } e.g.

e.g.

Pivot columns: 1st, 2nd, 3rd

Basis for ColA:

⇒ Basis for RowA:

MATH 2111 Page 23

Coordinate Vector Relative to a Basis
Wednesday, May 13, 2015 12:20 AM

Definition B-coordinate mapping

be a basis for H be a basis for

is the B-coordinate vector of f is:

○ Linear transformation
○ One-to-one
NOTE:
○ Onto
• but
• The order of must match the order of in is also linear transformation
the basis
IMPLICATION:
Any linear problem in V can be translated to a
REMEMBER: B must be l.i & SpanB = H corresponding problem in (solved by powerful matrix
theories)

e.g. e.g.
Is l.d.
for
in ?
Find Let
For each , we need to find such that: By B-coordinate mapping, W is sent to:

Let , we have 5 vectors in must be l.d.

Under inverse B-coordinate mapping, we obtain a
(k = 1,2,3) dependent relation in W
W is l.d
Let
We can solve all 3 equations in the following
ways:

(A is l.i. A must be row equivalent to I3)

for

MATH 2111 Page 24

MATH 2111 Page 25
Dimension of Vector Space
Wednesday, May 13, 2015 12:20 AM

Uniqueness of number of vectors of Basis Dimension of Vector Space

THM. 9: B is basis of V; |B| = n

be a basis for V is called n-dimensional: dimV = n
Any subset in H containing vectors must be l.d.
NOTE:
THM 10: • dim{0} = 0
• If V can't be spanned by finite vector sets:
dimV = ∞ (infinite dimensional)
B, C are two bases for V |B| = |C|, or n = m
e.g.

THM. 11:
Any l.i set in V can be extended to a basis for V
Characterization of a basis S of vector space V
If H is a subspace of V:
○
Any 2 of the following conditions:
○
a. |S| = n = dimV
b. S is l.i.
NOTE: How to extend a l.i set S to a V's basis:
c. SpanS = V
1. If SpanS = V, S is a basis
2. If not:
e.g. Describe all possible subspace of R 3
3. New set is l.i
Let H subspace of R3 ⇒ dimH = 0, 1, 2, 3
4. Go back to (1) with . This addition must
dimH = 0 ⇒ stop somewhere as l.i. set in V can't contain more
dimH = 1 than dimV = n vectors (Thm. 9)

is a line passing through

origin, containing Dimensions of NulA, RowA, ColA
dimH = 2
dim(NulA): Nullity of A
is a dim(NulA) = n - rankA
plane through origin, containing
dimH = 3 ⇒
dim(RowA): Row rank of A
dim(ColA): Column rank of A
dim(RowA) = dim(ColA) = rankA

(rankA = No. of pivot col. = No. of basic var.)

THM 14
Let be a matrix transformation

MATH 2111 Page 26

Eigenvalues & Eigenvectors
Tuesday, May 12, 2015 7:13 PM

Definition Eigenspace

A: n × n matrix •
When has solutions is eigenspace of A corresponding to eigenvalue λ
○ λ: Eigenvalue of A • Non-zero vectors in Eλ are eigenvectors of A
○ : Eigenvector of A corresponding to λ

NOTE:
• must be non-zero
• λ can be any number
Characteristic Equation

e.g.
λ is eigenvalue of n × n mat. A
⇔ has non-zero, multiple solutions
is an eigenvector of ⇔ det(A - λI) = 0
corresponding to eigenvalue λ = -4
THM. 1:
Eigenvalues of a triangular matrix are entries on its main diagonal

e.g.

MATH 2111 Page 27

Diagonalization
Wednesday, May 13, 2015 6:37 PM

Definition Steps of Diagonalizing Matrix A=PDP -1

1. Find eigenvalues λ of A

is invertible 2. For each of A, find a set of basic solution (by solving

: Eigenvectors corresponding )
to a. If total no. of vectors collected from all λ = n ⇒ A is
⇒ A is called diagonizable diagonalizable (THM. 4)
b. Otherwise, A is not diagonalizable
THM. 2:
• Eigenvectors corresponding to distinct 3. Construct P by putting collected vectors as its columns
eigenvalues are automatically l.i.
• For each : Set of basic solution 4. Construct D using the corresponding λ, following the same
is l.i. order chosen to construct P

e.g.

Multiplicity of an Eigenvalue

λ is of multiplicity m
⇔ λ occurs m times as root of characteristic eq.

e.g.

λ1 = 5: Multiplicity = 3
λ2 = 1: Multiplicity = 2
λ3 = -3: Multiplicity = 1

THM. 3:
n×n matrix A is diagonizable
⇔ A has n l.i eigenvectors
⇔ , or:
 Characteristic eq. factorizes completely
into linear factors:
e.g. (x - 3), (x - 5)2, …
(x2 - 5), (x3 -x + 1), …
 For each i:
dimNul(A - λiI)= Multiplicity of λ

Computation of Matrix Power

e.g. Find B such that

Let A=PDP-1

MATH 2111 Page 28

Let A=PDP-1
If we have C3 = D, then for B=PCP-1: B3 = PC3P-1 = PDP-1 = A

MATH 2111 Page 29

Applications of Eigenvalues & Eigenvectors
Thursday, May 14, 2015 12:15 AM

Discrete Dynamical System System of Ordinary Differential Equations

City: Constant pop. = 30000:

• 15000 Independent (I)
• 9000 Democrats (D) Rewrite the system using matrix notation:
• 6000 Liberals (L)
Each year:
• 20%D + 10%L → I
Diagonalization:
• 20%I + 10%L → D
• 10%I + 10%D → L
Find political composition of city in the long run

Use matrix notation to state the problem:

Take (or ):

⇔
⇔
⇔

Let

Solve each differential equation by method of integrating factors:

Write :
Characteristic eq.:
Substitute the above result to , we get the solution of :
⇔
⇔
⇔

* NOTE: Method of integrating factors

We have:

MATH 2111 Page 30

When k is large:

MATH 2111 Page 31

Inner Product Spaces
Tuesday, May 12, 2015 7:13 PM

Basic Definitions Properties

1. Inner Product on Rn
○ Normal definition:
i.
ii.
iii.
○ Matrix definition: iv.
⇔

2. Length/Norm of vectors:

: Unit vectors

3. Angle between 2 vectors:

MATH 2111 Page 32

Orthogonality
Friday, May 15, 2015 10:45 AM

Basic Definitions Orthogonal & Orthonormal Sets

1. Orthogonal: 1. Orthogonal Set:

○ ⇔ Any 2 vectors in S are always orthogonal to each other
○ ⇔
( is common perpendicular to S) 2. Orthonormal Set:
○ S is orthogonal
2. Orthogonal Complement: ○ Each vector in S is of unit length

is orthogonal complement of S in Rn e.g.

THM. 1: is always a subspace of Rn is orthogonal

THM. 2: is orthonormal



THM. 4:
THM. 3: Any orthogonal set S of non-zero vectors is always l.i.
A: m×n matrix
THM. 5:
is orthogonal set

Orthogonal & Orthonormal Basis THM. 5':

is orthonormal set
Basis for subspace W is called:
○ Orthogonal basis ⇔ Orthogonal set
○ Orthonormal basis ⇔ Orthonormal set
e.g. Determine if
e.g.
is orthonormal basis for Rn
is orthogonal basis for R2 Because (orthogonal set), we can use THM. 5 to verify
whether
is orthonormal basis for R2 If , we should have:

THM. 6:
• n cols of m×n matrix U are orthonormal ⇔ UTU = In
• n cols & n rows of n×n square matrix A are
orthonormal ⇔ ATA = In (orthogonal matrix)

THM. 7:
Linear transformation
Standard m×n matrix U with orthonormal cols.

a. (preserving length)
b. (preserving inner product)
c. ⇔ (preserving
orthogonality)

MATH 2111 Page 33

Orthogonal Projections
Friday, May 15, 2015 12:16 PM

Definitions THM. 9

1. Orthogonal Projections of onto W:

: Orthogonal basis for W "=" holds ⇔

is called the "best approx." of by vectors in W

NOTE: ⇔

2. Orthogonal Decomposition:

Standard Matrix of Orthogonal Projection (THM. 10)

3. Distance of to W: : Orthonormal basis for W

e.g. Let . Find Take :

First, check whether is an orthogonal basis for W:

○ Orthogonal set (also l.i.)
○ by def.

MATH 2111 Page 34

Gram-Schmidt Orthogonalization Process
Friday, May 15, 2015 12:56 PM

Gram-Schmidt Orthogonalization Process Example

1. Input: l.i
. Find orthogonal basis for ColA
2. Output:
orthogonal
First, find a basis for ColA (by choosing pivot columns of A):

3. Process:

Then, convert it to orthogonal basis using Gram-Schmidt process:

NOTE:
•

• If obtain for some k, i.e.

Scale
will be l.d.

Scale

Orthogonal basis for Col A =

MATH 2111 Page 35

Approximation Problems
Friday, May 15, 2015 12:57 PM

Least-Squares Solution Least-Squares Problems

Given 1. Linear Regression

Find such that: Fit a straight line y=mx +c to data
○
○ min

⇔
Solution:

(THM. 9)
Use normal equation:

Since we take:

⇔ (THM. 3)
e.g. (2, 1), (5, 2), (7, 3), (8, 3)
⇔
⇔
⇔

THM. 10:
Least-squares solutions of are solutions of
Normal equation:

⇔ ⇔
e.g. Find least-squares solution of:

2. Polynomial Curve Fitting

e.g. Finding a polynomial curve of degree 2 which best fits
the following data:

Solve:

THM. 11:
A: n×p matrix, l.i. cols Use normal equation:
W = ColA
Standard matrix of orthogonal projection onto W,
, is:

MATH 2111 Page 36

MATH 2111 Page 37