Nave Gauss Elimination

Ch.9

Nave Gauss Elimination

Linear Algebra Review

Elementary Matrix Operations Needed for

Elimination Methods:
Multiply an equation in the system by a non-
zero real number.
Interchange the positions of two equation in
the system.
Replace an equation by the sum of itself and a
multiple of another equation of the system.

Nave Gauss Elimination

Similar to Elimination of Unknowns

1. Forward Elimination
2. Backward Substitution

Nave because we dont consider division by zero

to be a possibility
a11 x1 + a12 x2 + a13 x3 = b1
a21 x1 + a22 x2 + a23 x3 = b2
a31 x1 + a32 x2 + a33 x3 = b3

1
 Nave Gauss Elimination
Similar to Elimination of Unknowns

1. Forward Elimination of Unknowns

1. Reduce the coeficient matrix [A] to an upper
triangular system
2. Eliminate x1 from the 2nd to nth Eqns.
3. Eliminate x2 from the 3rd to nth Eqns.
4. Continue process until the nth equation has only 1
Non-Zero coefficient

a11 a12 a13 x1 b1 a11 a12 a13 x1 b1

a
21 a22 a23 x2 = b2 0 a'22

a '23 x2 = b'2

a31 a33 x3 b3 a' '33 x3 b' '3
0
a32 0

Nave Gauss Elimination

1. Forward Elimination
a11 x1 + a12 x2 + a13 x3 = b1 (1)
a21 x1 + a22 x2 + a23 x3 = b2 (2)
a31 x1 + a32 x2 + a33 x3 = b3 (3)
Eliminate x1 from equation (2). Multiply (1) by a21/a11, then subtract the result from (2)

a a a a
21 a11 x1 + 21 a12 x2 + 21 a13 x3 = 21 b1
a11 a11 a11 a11
a21 x1 + a22 x2 + a23 x3 = b2
a a a
a22 21 a12 x2 + a23 21 a13 x3 = b2 21 b1
a11 a11 a11

a'22 x2 + a '23 x3 = b'2 (2' )

Nave Gauss Elimination

1. Forward Elimination
a11 x1 + a12 x2 + a13 x3 = b1 (1) Pivot Equation

a21 x1 + a22 x2 + a23 x3 = b2 (2) Elimination Row

a31 x1 + a32 x2 + a33 x3 = b3 (3)
Eliminate x1 from equation (2). Multiply (1) by a21/a11, then subtract the result from (2)

a a a a
21 a11 x1 + 21 a12 x2 + 21 a13 x3 = 21 b1
a11 a11 a11 a11
Pivot Element
a21 x1 + a22 x2 + a23 x3 = b2
a a a
a22 21 a12 x2 + a23 21 a13 x3 = b2 21 b1
a11 a11 a11

a'22 x2 + a '23 x3 = b'2 (2' )

2
 Nave Gauss Elimination
1. Forward Elimination
a11 x1 + a12 x2 + a13 x3 = b1 (1)
+ a'22 x2 + a'23 x3 = b'2 (2' )
a31 x1 + a32 x2 + a33 x3 = b3 (3) Elimination Row
Eliminate x1 from (3). Multiply (1) by a31/a11, then subtract the result from (3)

a a a a
31 a11 x1 + 31 a12 x2 + 31 a13 x3 = 31 b1
11
a a11 a11 a11
a31 x1 + a32 x2 + a33 x3 = b3
a31 a31 a31
a32 a a12 x2 + a33 a a13 x3 = b3 a b1
11 11 11

a'32 x2 + a '33 x3 = b'3

Nave Gauss Elimination

1. Forward Elimination
a11 x1 + a12 x2 + a13 x3 = b1 (1)
+ a'22 x2 + a'23 x3 = b'2 (2' ) Pivot Equation

+ a'32 x2 + a '33 x3 = b'3 (3' ) Elimination Row

Eliminate x2 from (3). Multiply (2) by a32/a22, then subtract the result from (3)

a' a' a'

32 a '22 x2 + 32 a'23 x3 = 32 b'2
a '22 a '22 a '22
a '32 x2 + a'33 x3 = b'3
a' a'
+ a'33 32 a'23 x3 = b'3 32 b'2
a'22 a '22

a' '33 x3 = b' '3

Nave Gauss Elimination

1. Forward Elimination
a11 x1 + a12 x2 + a13 x3 = b1 (1)
a'22 x2 + a '23 x3 = b'2 (2' )
+ a ' '33 x3 = b' '3 (3' ' )
Solve for x3

b' '3
x3 =
a ' '33

3
 Nave Gauss Elimination
2. Backwards substitution:
a11 x1 + a12 x2 + a13 x3 = b1 (1)
a'22 x2 + a '23 x3 = b'2 (2' )
+ a ' '33 x3 = b' '3 (3' ' )
b' '
x3 = 3
a ' '33
From (2)
b'2 a '23 x2
x2 =
a '22

b1 a12 x2 a13 x3
From (1) x3 =
a11

Nave Gauss Elimination

In General, the last equation should reduce to:
bn(n 1)
xn = ( n 1)
ann
n
bi(i 1) a(
j =i +1
ij
i 1)
xj
xi = (i 1)
a ii

General form is how we will numerically implement.

Note: Since we normalize with the pivot element, if it is zero, we have a

problem Nave method

Nave Gauss Elimination

Example:
2 x1 + x2 + 3 x3 = 1 (1)
4 x1 + 4 x2 + 7 x3 = 1 (2)
2 x1 + 5 x2 + 9 x3 = 3 (3)

4
 Nave Gauss Elimination Numerically
Implementing
a11 a12 a13 x1 b1 a11 a12 a13 x1 b1
a
21 a22 a23 x2 = b2 0 a'22

a '23 x2 = b'2

a31 a33 x3 b3 a' '33 x3 b' '3
0
a32 0

3 Main Loops: Forward Elimination

1. Pivot Row from 1st row to the n-1 row, move down, we will
call the pivot row, row k.
2. Elimination Row Rows below Pivot row, where eliminations
take place (top down), call this the ith row.
3. Element transform Loop columns, jth column. Move left to
right.

Nave Gauss Elimination Numerically

Implementing
a11 a12 a13 x1 b1 a11 a12 a13 x1 b1
a
21 a22 a23 x2 = b2 0 a'22

a '23 x2 = b'2

a31 a33 x3 b3 a' '33 x3 b' '3
0
a32 0

What we do: Forward Elimination

A. Normalization Step: multiply the kth row elements akj by (-aik/akk)
B. Add the result of step A, to aij
aik
a'ij = aij akj Instead of saving aij
akk we save as aij

a21
a'22 = a22 a12
a11 aik
C. Calculate the new bs or right hand side terms b'i = bi bk
akk
a21
b'2 = b2 b1
a11

Nave Gauss Elimination Numerically

Implementing
a11 a12 a13 x1 b1 a11 a12 a13 x1 b1
0
a'22 a '23 x2 = b'2 0 a22

a23 x2 = b2

0 0 a' '33 x3 b' '3 0 0 a33 x3 b3

What we do: Back Substitution

For the nth Row:
bn
xn =
ann

Now, work backwards row by row, right to left (n-1 row and n
column) n
bi a x
j = i +1
ij j

xi =
aii

5
 Nave Gauss Elimination Pseudocode
%Forward Elimination to build an upper triangular matrix
for k=1:n-1
for i=k+1:n
factor = a(i,k)/a(k,k); %normalzing factor (Step A)
for j=k+1:n %move accross the columns loop
a(i,j) = a(i,j) - factor*a(k,j); %(Step B)
end
b(i)=b(i)-factor*b(k); %(Step C)
end
end
%Backward Substitution
x(n)=b(n)/a(n,n); %solve for the last x value
for i=n-1:-1:1
sum = 0;
for j=i+1:n
sum = sum + a(i,j)*x(j);
end
x(i)=(b(i)-sum)/a(i,i);
end

Problems with Nave Elimination Methods

1. Division by zero
if a11 = 0, then the 1st elimination step yields division by
zero.
pivoting technique will be used to avoid this problem
2. R.O. Error
Every result is dependant on previous results RO error
can propagate
Rule of Thumb if n > 100
Double precision will help
3. Ill Conditioned Systems (D~0)
Small changes in the coefficient (aij) matrix result in large
changes in the solution
Or, alternatively a wide range of answers (xis) satisfy the
equations
RO error can produce small changes in coefficients that
can lead to large errors, (Check by slightly changing the
coefficients and seeing the effect on the results)

Problems with Nave Elimination Methods

4. Singular Systems: (D=0)
One or more equations are identical
We have (n-1) equations and n unknowns

QUICK way to check D

After the forward elimination evaluate the determinant of
the modified coefficient matrix
a11 a12 a13 x1 b1
0
a'22 a '23 x2 = b'2
0 0 a' '33 x3 b' '3

a11 a12 a13

D = 0 a '22 a'23 = a11 a '22 a' '33
0 0 a ' '33

6
 Problems with Nave Elimination Methods
When checking D, how small is too small? Solution:
Standardize the determinant.

Scale Equations such that the maximum coefficient

for any equation is 1.
3x1 + 2 x2 = 12
D = 11
x1 + 3 x2 = 3

Divide (1) by 3 and (2) by 3.

2
x1 + x2 = 4
3
D = 1.22
1
x1 + x2 = 1
3

Problems with Nave Elimination Methods

Now for a 2x2 matrix D takes on Values:
a11 x1 + a12 x2 = b1
a21 x1 + a22 x2 = b 2
D = a11a22 a12 a21
D 2

Methods for Improving Solutions

1. Use More Significant digits
2. Partial Pivoting
Avoid division by zero or vary small numbers
a) Before normalizing in Gauss elimination, find the
largest element (absolute value)in the first
column
b) Reorder the equations so that the largest
element is the pivot element
c) Repeat for each elimination step I.e., 2nd
application would find the largest element in the
2nd column (below the 1st Equation) and seek the
largest pivot element.

7
Methods for Improving Solutions Partial Pivoting
code
p=k; %assume row with largest coefficient
big=abs(a(k,k)) %assume the diagnol term is largest

for ii = k+1:n %move down the rows to check elements

dummy=abs(a(ii,k));
if dummy > big %if the elent is bigger swap it out
big=dummy;
p=ii; %rename the largest row
end
end %end for loop

%if p is not equal to k, we need to swap row k with row p

%if p is equal to k, then we don't do anything
if p~=k
for jj=k:n %move across columns to swap coefficient values
dummy=a(p,jj); %temporarily store the element
a(p,jj)=a(k,jj);
a(k,jj)=dummy;
end
dummy = b(p); %now sway right hand side values
b(p) = b(k);
b(k) = dummy;
Show Matlab example
end

Methods for Improving Solutions

3. Scaling helps make pivoting decisions

3x1 + 70,000 x2 = 40,000 What problem could

x1 + .2 x2 = 3 Arise here

scale

0.0000428 x1 + 1x2 = 0.5714

x1 + .2 x2 = 3
Pivot

x1 + .2 x2 = 3
0.0000428 x1 + 1x2 = 0.5714

Methods for Improving Solutions

3. Scaling
We have seen that adding and subtracting of numbers with
very different magnitudes can result in RO error
Scale all rows so that the maximum coefficient value in any
row is one.
NOTE: scaling by very large numbers can potentially
introduce RO error

Suggestion:
Employ scaling only to make a decision regarding pivoting
Comparison & row switching are not subject to RO error
Complete solution using original coefficients

8
 Gauss-Jordan Elimination
Variation of Gauss Elimination
When an unknown is eliminated it is eliminated from all
equations, not just subsequent ones (Diagonal Matrix Results)
All rows are normalized by their pivot element
Identity Matrix results a a a b
[A]
11 12 13 1
Is augmented
a21 a22 a23 b2
a31 a33 b3
[I ]{x} = {b'} a32

1 0 0 x1 b'1
0 1 0 x = b'
2 2
0 0 1 x3 b'3
Almost identical to Gauss Elimination but, more operations are
required
No back substitution step

Methods for Improving Solutions Partial Pivoting

Overhead Gauss Jordan example