Simultaneous Linear Equations

Topic: Gaussian Elimination

Dr. Nasir M Mirza
Numerical Methods
Email: nasirmm@yahoo.com
Gaussian Elimination
Basic Principles
The general description of a set of linear equations in
the matrix form: [A][X] = [B]
Where, [A] is ( m x n ) matrix, the [X] is a ( n x 1 )
vector, and the [B] is a (m x 1 ) vector.
How we solve such a system:
Write the equations in natural form
Identify unknowns and order them
Isolate unknowns
Write equations in matrix form
Matrix Representation
(
(
(
(
(
(

=
(
(
(
(
(
(

(
(
(
(
(
(

n n nn n n n
n
n
n
b
b
b
b
x
x
x
x
a a a a
a a a a
a a a a
a a a a

3
2
1
3
2
1
3 2 1
3 33 32 31
2 23 22 21
1 13 12 11
m n mn m m m
n n
n n
n n
b x a x a x a x a
b x a x a x a x a
b x a x a x a x a
b x a x a x a x a
= + + + +
= + + + +
= + + + +
= + + + +

3 3 2 2 1 1
3 3 3 33 2 32 1 31
2 2 3 23 2 22 1 21
1 1 3 13 2 12 1 11 The set of n linear
equations with n
unknowns:
The matix form:
Types of Matrix Formulation
m n mn m m m
n n
n n
n n
b x a x a x a x a
b x a x a x a x a
b x a x a x a x a
b x a x a x a x a
= + + + +
= + + + +
= + + + +
= + + + +

3 3 2 2 1 1
3 3 3 33 2 32 1 31
2 2 3 23 2 22 1 21
1 1 3 13 2 12 1 11
If m = n The solution of [A][x] =[b] with n unknowns and m eqs.
If m > n The system is over-determined (Least Square Problems)
If m < n The system is underdetermined (Optimization Problems)
Suppose we have m
equations and n unknown in
a linear system.
| || | | |
(
(
(
(
(
(

=
(
(
(
(
(
(

(
(
(
(
(
(

=
m m mn m m m
n
n
n
b
b
b
b
x
x
x
x
a a a a
a a a a
a a a a
a a a a
b X A

3
2
1
3
2
1
3 2 1
3 33 32 31
2 23 22 21
1 13 12 11
The Nonsingular Case (m = n)
If the number (m) of equations and the number (n) of unknowns are
equal (m = n), then the coefficient matrix A is square so that the
solution can be written as x = A
1
b; Here the matrix A is not
singular.
There are MATLAB commands for one example are:
A = [1 2; 3 4]; b = [-1; -1];
x = inv(A)*b
The result will be
x = 1.0000
-1.0000
What if A is square, but singular?
A = [1 2;2 4]; b = [-1;-1];
x = inv(A)*b
Result will be
Warning: Matrix is singular to working precision.
x = -Inf
-Inf
Matrix Representation
(
(
(
(

=
(
(
(
(

(
(
(
(

n
2
1
n
2
1
nn n2 n1
n 2 22 21
1n 12 11
b
b
b
x
x
x
a a a
a a a
a a a

A system of algebraic equations has the following form:

n n nn n n n
n n
n n
b x a x a x a x a
b x a x a x a x a
b x a x a x a x a
= + + + +
= + + + +
= + + + +




3 3 2 2 1 1
2 2 3 23 2 22 1 21
1 1 3 13 2 12 1 11
[A][x] = [b]
where the
coefficients a
ij
and
the constants b
j
are known, and x
i
represent the
unknowns.
In matrix
notation the
equations are
written as:
Consistency & Uniqueness of Solution
[A][x] = [b]
The problem is consistent, if a solution exists for the
problem.
A system of n linear equations in n unknowns has a
unique solution, provided that the determinant of the
coefficient matrix is nonsingular, i.e., if |A| 0.
The rows and columns of a nonsingular matrix are
linearly independent in the sense that no row or column
is a linear combination of other rows (or columns).
Consistency & Uniqueness of Solution
If the coefficient matrix is singular, the equations may
have an infinite number of solutions, or no solutions at
all, depending on the constant vector.
As an example, take the equations:
2x + y = 3 ; 4x + 2y = 6
The second equation can be obtained by multiplying
the first equation by two,
Any combination of x and y that satisfies the first
equation is also a solution of the second equation.
The number of such combinations is infinite.
Consistency & Uniqueness of Solution
On the other hand, look at the following equations:
2x + y = 3; 4x + 2y = 0
They have no solution because the second equation is
equivalent to 2x + y = 0,
It contradicts the first one.
Therefore, any solution that satisfies one equation
cannot satisfy the other one.
No consistency.
Rank of Matrix and consistency
Rank of a matrix is the number of linearly independent column
vectors in the matrix.

For n x n matrix, A:
If rank(A) = n and is consistent, A has an unique solution exists;
If rank(A) = n and is inconsistent, A has no solution
If rank(A) < n and system is consistent, A has an infinite number of
solutions.
The problem is consistent, if a solution exists for the problem.
The problem is inconsistent, if there is no solution for the problem.
Rank of Matrix and consistency
Therefore, for an n x n system, rank(A) = n
automatically guarantees that the system is
consistent.

The columns of A are linearly independent;
The rows of A are linearly independent;
rank(A) = n
det(A) 0 and [A]
-1
exists; or [x] = [A]
-1
[b]
The solution to [A][x]=[b] exists and it is unique.
Matrix Definition
Consider
y = -2.0 x + 6
y = 0.5 x + 1
2 unknowns x , y and
rank is 2

Consider
y = -2 x + 6
y = -2 x + 5
2 unknowns x , y and rank
is 1 and is inconsistent

)
`

=
)
`

1
6
1 5 . 0
1 2
y
x
)
`

=
)
`

5
6
1 2
1 2
y
x
Gaussian Elimination
Gaussian Elimination is one of the most popular techniques for solving
simultaneous linear equations of the form: [A][X] =[b]

The method consists of 2 steps:
1. Forward Elimination of Unknowns;
2. Back Substitution.

Let us learn the method first by examples:
m n mn m m m
n n
n n
n n
b x a x a x a x a
b x a x a x a x a
b x a x a x a x a
b x a x a x a x a
= + + + +
= + + + +
= + + + +
= + + + +

3 3 2 2 1 1
3 3 3 33 2 32 1 31
2 2 3 23 2 22 1 21
1 1 3 13 2 12 1 11
Example 1:
Consider following three linear equations with three unknowns:
) 3 ( 74 . 5 06 . 1 13 . 5 69 . 3
) 2 ( 69 . 4 75 . 3 78 . 0 46 . 1
) 1 ( 76 . 1 28 . 4 06 . 3 37 . 2
3 2 1
3 2 1
3 2 1
= + +
= +
= +
x x x
x x x
x x x
) 4 ( 7426 . 0 8059 . 1 2911 . 1
3 2 1
= + x x x
) 5 ( 6058 . 3 366 . 6 6650 . 2 0
69 . 4 75 . 3 780 . 0 46 . 1
0842 . 1 6366 . 2 885 . 1 46 . 1
3 2
3 2 1
3 2 1
= +
= + +
= +
x x
x x x
x x x
First step is to divide by 2.37 so that the coefficient of x
1
is one. Thus
Now multiply this by -1.46 and adding it with Eq 2.
) 6 ( 4802 . 8 6038 . 5 8942 . 9 0
3 2
= + x x
Now multiply Eq.4 with 3.69 and adding it to Eq3.
Example 1:
Re-writing Equations 4, 5 and 6:
) 6 ( 4802 . 8 6038 . 5 8942 . 9 ) 0 (
) 5 ( 6058 . 3 3866 . 6 6650 . 2 ) 0 (
) 4 ( 7426 . 0 8059 . 1 2911 . 1
3 2 1
3 2 1
3 2 1
= +
= +
= +
x x x
x x x
x x x
) 7 ( 3530 . 1 3965 . 2 0
3 2
= + x x
8671 . 21 1077 . 18 ) 0 ( 0
3 2
= + + x x
The second step is to divide by -2.6650 so that the coefficient of x
2
is one. Thus
Now multiply this by -9.8942 and adding it to Eq 6.
dividing it by 18.1077, we get
) 8 ( 2076 . 1 ) 0 ( 0
3 2
= + + x x
Example 1:
Final form of Eq. 4, 7 and 8 is
) 8 ( 2076 . 1 0 0
) 7 ( 3530 . 1 3965 . 2 0
) 4 ( 7426 . 0 8059 . 1 2911 . 1
3
3 2
3 2 1
= + +
= +
= +
x
x x
x x x
5410 . 1 2076 . 1 3965 . 2 3530 . 1
3965 . 2 3530 . 1
3 2
= + =
+ = x x
Now x
3
is known and we can back-substitute it into Eq. 7 to find x
2
.
Similarly, we can find x
1
using values of x
2
and x
3
from Eq. 4
Therefore, the solution is given as
x
1
= 0.9338 ; x
2
= 1.5410 ; x
3
= 1.2076
9338 . 0
2076 . 1 3965 . 2 5410 . 1 2911 . 1 7426 . 0
3965 . 2 2911 . 1 7426 . 0
3 2 1
=
+ =
+ = x x x
Forward Elimination
Linear Equations:
A set of n equations and n
unknowns
n n nn n n n
n n
n n
n n
b x a x a x a x a
b x a x a x a x a
b x a x a x a x a
b x a x a x a x a
= + + + +
= + + + +
= + + + +
= + + + +

3 3 2 2 1 1
3 3 3 33 2 32 1 31
2 2 3 23 2 22 1 21
1 1 3 13 2 12 1 11
| || | | |
(
(
(
(
(
(

=
(
(
(
(
(
(

(
(
(
(
(
(

=
n n nn n n n
n
n
n
b
b
b
b
x
x
x
x
a a a a
a a a a
a a a a
a a a a
b X A

3
2
1
3
2
1
3 2 1
3 33 32 31
2 23 22 21
1 13 12 11
Forward Elimination
Transform to an Upper Triangular Matrix
Step 1: Eliminate x
1
in 2
nd
equation using Eq. 1 as the pivot
equation; so divide Eq. 1 by a
11
and multiply by a
21
.
) (
1
21
11
a
a
Eqn

Which will change the Eq. 1 as following:

1
11
21
1
11
21
2 12
11
21
1 21
... b
a
a
x a
a
a
x a
a
a
x a
n n
= + + +
) 1 ( ...
1 1 3 13 2 12 1 11
b x a x a x a x a
n n
= + + + +
Forward Elimination
Both equations are here:
Subtract Equation 1 from Eq.2
1
11
21
2 1
11
21
2 2 12
11
21
22
... b
a
a
b x a
a
a
a x a
a
a
a
n n n
=
|
|
.
|

\
|
+ +
|
|
.
|

\
|

'
2
'
2 2
'
22 1
... ) 0 ( b x a x a x
n n
= + + +
n n n
a
a
a
a a
a
a
a
a a
1
11
21
2
'
2
12
11
21
22
'
22


=
=

Or
) 2 ( ...
) 1 ( ...
2 2 3 23 2 22 1 21
1
11
21
1
11
21
2 12
11
21
1 21
b x a x a x a x a
b
a
a
x a
a
a
x a
a
a
x a
n n
n n
= + + + +
= + + +
Where,
Forward Elimination
Repeat this procedure for the remaining equations to reduce the set of
equations as
) ( ... ) 0 (
) 3 ( ... ) 0 (
) 2 ( ... ) 0 (
) 1 ( ...
' '
3
'
3 2
'
2 1
'
3
'
3 3
'
33 2
'
32 1
'
2
'
2 3
'
23 2
'
22 1
1 1 3 13 2 12 1 11
n b x a x a x a x
b x a x a x a x
b x a x a x a x
b x a x a x a x a
n n nn n n
n n
n n
n n
= + + + +
= + + + +
= + + + +
= + + + +

Forward Elimination
Step 2: Eliminate x
2
in the 3
rd
equation:
It is equivalent to eliminating x
1
in the 2
nd
equation using
equation 2 as the pivot equation.
) (
2
3
32
22
a
a
Eqn
Eqn
'

'

Forward Elimination
Then we have the following form:
) ( ...
) 3 ( ...
) 2 ( ...
) 1 ( ...
' '
3
'
3
'
3
'
3 3
'
33
'
2
'
2 3
'
23 2
'
22
1 1 3 13 2 12 1 11
n b x a x a
b x a x a
b x a x a x a
b x a x a x a x a
n n nn n
n n
n n
n n
= + +
= + +
= + + +
= + + + +

Forward Elimination
Continue this procedure by using the third equation as
the pivot equation and so on.
At the end of (n-1) Forward Elimination steps, the
system of equations will look like:
'
2
'
2 3
'
23 2
'
22
... b x a x a x a
n n
= + + +
"
3
"
3
"
33
... b x a x a
n n
= + +
( )
( ) 1
1

=
n
n n
n
nn
b x a



1 1 3 13 2 12 1 11
... b x a x a x a x a
n n
= + + + +
Forward Elimination
At the end of the Forward Elimination steps:
(
(
(
(
(
(

=
(
(
(
(
(
(

(
(
(
(
(
(

) - (n
n n
3
2
1
n
nn
n
n
n
b
b
b
b
x
x
x
x
a
a a
a a a
a a a a
1
"
3
'
2
1
) 1 (
"
3
"
33
'
2
'
23
'
22
1 13 12 11

Back Substitution
The goal of Back Substitution is to solve each of the
equations using the upper triangular matrix.
(
(
(

=
(
(
(

(
(
(

3
2
1
3
2
1
33
23 22
13 12 11
x
x
x

0 0
0
b
b
b
a
a a
a a a
Example of a system of 3 equations:
Here x3 = b3/a33; x2 = [ b2 (a23)(x3) ]/a22
x2 = {b2 (b3)(a23)/(a33)}/(a22);
x1 = [ b1 (a12)(x2) (a13)(x3) ]/a11;
x1 = [ b1 (a12)(x2) (a13)(x3) ]/a11
x1 = [ b1 (a12)(b2 b3(a23)(a33))/a22 (a13)(b3)/a33 ]/a11 ;
Back Substitution
Start with the last equation because it has only one unknown,
we get the following:
) 1 (
) 1 (

=
n
nn
n
n
n
a
b
x
Solve the second from last equation (n-1)
th
using x
n
solved for
previously.
This solves for x
n-1
.
Back Substitution
Representing Back Substitution for all equations by formula
( ) ( )
( ) 1
1
1 1

+ =

=
i
ii
n
i j
j
i
ij
i
i
i
a
x a b
x

For i=n-1, n-2,.,1
and
) 1 (
) 1 (

=
n
nn
n
n
n
a
b
x
Example & Computer Program
% Example on Gauss elimination
A =[4 -2 1; -2 4 -2; 1 -2 4 ];
b = [11 -16 17]';
format long
[x,det] = gauss(A,b)
% here gauss function is called
% check the accuracy
Y = A*x - b
Computer Program (continued)
function [x,det] = gauss(A,b)
% Solves A*x = b by Gauss elimination and computes det(A).
if size(b,2) > 1; b = b'; end % b must be column vector
n = length(b);
for k = 1:n-1 % Elimination phase
for i= k+1:n
if A(i,k) ~= 0
lambda = A(i,k)/A(k,k);
A(i,k+1:n) = A(i,k+1:n) - lambda*A(k,k+1:n);
b(i)= b(i) - lambda*b(k);
end
end
end
if nargout == 2; det = prod(diag(A)); end
for k = n:-1:1 % Back substitution phase
b(k) = (b(k) - A(k,k+1:n)*b(k+1:n))/A(k,k);
end
x = b;
Results of computer program
x =
1
-2
3

det =

36

Y =
0
0
0
Example: Rocket Velocity
The upward velocity of a rocket is given
at three different times
Time, t Velocity, v
s m/s
5 106.8
8 177.2
12 279.2
The velocity data is approximated by a polynomial as:
( ) 12. t 5 ,
3 2
2
1
s s + + = a t a t a t v
Find: The Velocity at t=6,7.5,9, and 11 seconds.
Example: Rocket Velocity
Assume:
( ) 12. t 5 , a t a t a t v s s + + =
3 2
2
1
(
(
(

=
(
(
(

(
(
(

3
2
1
3
2
3
2
2
2
1
2
1
1
1
1
v
v
v
a
a
a

t t
t t
t t
3
2
1
Results in a matrix template of the form:
Using date from the time / velocity table, the matrix becomes:
(
(
(

=
(
(
(

(
(
(

2 . 279
2 . 177
8 . 106
1 12 144
1 8 64
1 5 25
3
2
1
a
a
a

Example: Rocket Velocity
Forward Elimination: Step 1
=
(

) 64 (
25
1
2
Row
Row
Result is
(
(
(

=
(
(
(

(
(
(

2 . 279
21 . 96
81 . 106
a
a
a

1 12 144
56 . 1 8 . 4 0
1 5 25
3
2
1
(
(
(

=
(
(
(

(
(
(

2 . 279
2 . 177
8 . 106
1 12 144
1 8 64
1 5 25
3
2
1
a
a
a

Example: Rocket Velocity
=
(

) 144 (
25
1
3
Row
Row
(
(
(

=
(
(
(

(
(
(

0 . 336
21 . 96
8 . 106
a
a
a

76 . 4 8 . 16 0
56 . 1 8 . 4 0
1 5 25
3
2
1
Forward Elimination: Step 1
(
(
(

=
(
(
(

(
(
(

2 . 279
21 . 96
81 . 106
a
a
a

1 12 144
56 . 1 8 . 4 0
1 5 25
3
2
1
Example: Rocket Velocity
=
(

) 8 . 16 (
8 . 4
2
3
Row
Row
(
(
(

=
(
(
(

(
(
(

735 . 0
21 . 96
8 . 106
a
a
a

7 . 0 0 0
56 . 1 8 . 4 0
1 5 25
3
2
1
This is now ready for Back Substitution
Forward Elimination: Step 2
(
(
(

=
(
(
(

(
(
(

0 . 336
21 . 96
8 . 106
a
a
a

76 . 4 8 . 16 0
56 . 1 8 . 4 0
1 5 25
3
2
1
Example: Rocket Velocity
Back Substitution: Solve for a
3
using the third equation
735 . 0 7 . 0
3
= a
7 0
735 0
.
.
a
3
=
050 1. a
3
=
Example: Rocket Velocity
Back Substitution: Solve for a
2
using the second equation
21 . 96 56 . 1 8 . 4
3 2
= a a
8 . 4
56 . 1 21 . 96
3
2

+
=
a
a
( )
8 4
050 1 56 1 21 96
. -
. . . -
a
2
+
=
70 19. a
2
=
Example: Rocket Velocity
Back Substitution: Solve for a
1
using the first equation
8 . 106 5 25
3 2 1
= + + a a a
25
5 8 . 106

3 2
1
a a
a

=
( )
25
050 . 1 70 . 19 5 8 . 106
1

= a
2900 . 0
1
= a
Example: Rocket Velocity
Solution:
The solution vector is
(
(
(

=
(
(
(

050 . 1
70 . 19
2900 . 0
3
2
1
a
a
a
The polynomial that passes through the three data points is
then given as :
( )
12 5 , 050 . 1 70 . 19 2900 . 0
2
3 2
2
1
s s + + =
+ + =
t t t
a t a t a t v
Example: Rocket Velocity
Solution: Substitute each value of t to find the corresponding
velocity
( ) ( ) ( )
. s / m 1 . 165
050 . 1 5 . 7 70 . 19 5 . 7 2900 . 0 5 . 7 v
2
=
+ + =
( ) ( ) ( )
. s / m 8 . 201
050 . 1 9 70 . 19 9 2900 . 0 9 v
2
=
+ + =
( ) ( ) ( )
. s / m 8 . 252
050 . 1 11 70 . 19 11 2900 . 0 11 v
2
=
+ + =
( ) ( ) ( )
. / 69 . 129
050 . 1 6 70 . 19 6 2900 . 0 6
2
s m
v
=
+ + =
Pitfalls of method
Division by zero: May occur in the forward elimination
steps. Consider the set of equations:
-- Round-off error: Prone to round-off errors.
(
(
(

=
(
(
(

(
(
(

6
901 . 3
7
5 1 5
6 099 . 2 3
0 7 10
3
2
1
x
x
x
Pitfalls: Example
Consider the system of equations:
Use five significant figures with chopping
(
(
(

=
(
(
(

(
(
(

6
901 . 3
7
5 1 5
6 099 . 2 3
0 7 10
3
2
1
x
x
x
(
(
(

3
2
1
x
x
x
(
(
(

6
901 . 3
7

At the end of Forward Elimination
(
(
(

15005 0 0
6 001 . 0 0
0 7 10
(
(
(

3
2
1
x
x
x
(
(
(

15004
001 . 6
7
=

Pitfalls: Example
Back Substitution
99993 . 0
15005
15004
3
= = x
5 . 1
001 . 0
6 001 . 6
3
2
=

=
x
x
3500 . 0
10
0 7 7
3 2
1
=
+
=
x x
x
(
(
(

=
(
(
(

(
(
(

15004
001 . 6
7
15005 0 0
6 001 . 0 0
0 7 10
3
2
1
x
x
x
Pitfalls: Example
Compare the calculated values with the exact solution
| |
(
(
(

=
(
(
(

=
99993 . 0
5 . 1
35 . 0
3
2
1
x
x
x
X
calculated
| |
(
(
(

=
(
(
(

=
1
1
0
3
2
1
x
x
x
X
exact
Improvements
Increase the number of significant digits
Decreases round off error
Does not avoid division by zero
Gaussian Elimination with Partial Pivoting
Avoids division by zero
Reduces round off error
Partial Pivoting
The core formula used for Gauss elimination requires
division by a
kk
at the kth stage, where a
kk
is the
diagonal element in the kth row.
What if a
kk
= 0?
In such a case, it is customary to switch the kth row
and another row below it having the element of the
largest absolute value in the kth column.
This procedure, called partial pivoting,
It is recommended for reducing the round-off error
even in the case where the kth pivot a
kk
is not zero.
Let us consider the following example:
Partial Pivoting: Example
Consider the system of equations:
1
0 2
2 ) 0 (
3 2 1
3 2 1
3 2 1
= +
=
= + +
x x x
x x x
x x x
In matrix form:
(
(
(

=
(
(
(

(
(
(

+

1
0
2
1 1 1
1 1 2
1 1 0
3
2
1
x
x
x
(
(
(

3
2
1
x
x
x

Let us Solve this using Gauss Elimination with Partial
Pivoting;
Unknown are:
Partial Pivoting: Example
(
(
(

=
(
(
(

(
(
(

1
0
2
1 1 1
1 1 2
1 1 0
3
2
1
x
x
x
Step 1: Examining the values of the first column
|0|, |2|, and |1| or 0, 2, and 1.
The largest absolute value is 2, which means, to follow the
rules of Partial Pivoting, we switch row1 with row2 .

(
(
(

=
(
(
(

(
(
(

+ +

1
2
0
1 1 1
1 1 0
1 1 2
3
2
1
x
x
x
Partial Pivoting: Example
(
(
(

=
(
(
(

(
(
(

+ +

1
2
0
2 / 1 2 / 3 0
1 1 0
1 1 2
3
2
1
x
x
x
Step 2: now perform forward elimination
perform the following operations:
row2 (0/2) x row1 and row3 (1/2) x row1

(
(
(

=
(
(
(

(
(
(

+ +

1
2
0
1 1 1
1 1 0
1 1 2
3
2
1
x
x
x
Partial Pivoting: Example
(
(
(

=
(
(
(

(
(
(

+ +

1
2
0
2 / 1 2 / 3 0
1 1 0
1 1 2
3
2
1
x
x
x
Step 2: Here, instead of pivoting at a22 , we switch the
second row and the third row. The largest value is in 3
rd

row.
The element of the largest absolute value among the
elements is 3/2.

(
(
(

=
(
(
(

(
(
(

2
1
0
1 1 0
2 / 1 2 / 3 0
1 1 2
3
2
1
x
x
x
Partial Pivoting: Example
(
(
(

=
(
(
(

(
(
(

3 / 4
1
0
3 / 4 0 0
2 / 1 2 / 3 0
1 1 2
3
2
1
x
x
x
Step 2: now perform forward elimination:
Perform these operations:
row3 (2/3) x row2

(
(
(

=
(
(
(

(
(
(

2
1
0
1 1 0
2 / 1 2 / 3 0
1 1 2
3
2
1
x
x
x
Partial Pivoting: Example
Back Substitution: Solving the equations through back
substitution:
1
3 / 4
3 / 4
3
= = x
1
2 / 3
5 . 0 1
3
2
=
+
=
x
x
1
2
0
3 2
1
=
+ +
=
x x
x
(
(
(

=
(
(
(

(
(
(

3 / 4
1
0
3 / 4 0 0
2 / 1 2 / 3 0
1 1 2
3
2
1
x
x
x
[x
1
x
2
x
3
] = [1 1 1]
Conditioning
As the size of the matrix grows, the round-off errors
grow and propagate in matrix operations to such a
degree that zero may appear to be an absolutely small
number, or a nonzero number very close to zero may
appear to be zero.

In order to be able to give such a qualitative judgment
in the right way based on some quantitative analysis, we
should be equipped with theoretical knowledge as well
as practical experience.
There are several criteria by which we judge the degree
of ill-condition, such as how discrepant A A
1
is with the
identity matrix, how far det{A}det{A
1
} stays away from
one(1).
Conditioning
An obvious question is: what happens when the
coefficient matrix is almost singular;
That is if det(A) = |A| is very small?
In order to determine whether the determinant of the
coefficient matrix is small, we need a reference against
which the determinant can be measured.
This reference is called the norm of the matrix, denoted
by ||A||.
We can then say that the determinant is small if
||A||<< A
Where, ||A|| = max of (sum of aij for ith row) for all rows.

Conditioning
If the equations are ill-conditioned, small changes in the
coefficient matrix result in large changes in the solution.
As an illustration, consider the equations
2x + y =3; 2x + 1.001y = 0

that have the solution x = 1501.5, y = 3000.
Since |A| = 2(1.001) 2(1) = 0.002 is much smaller
than the coefficients, the equations are ill-conditioned.
The effect of ill-conditioning can be verified by changing
the second equation to 2x + 1.002y = 0 and re-solving
the equations.
The result is x = 751.5, y = 1500.
Note that a 0.1% change in the coefficient of y produced
a 100% change in the solution.
Numerical Solutions and conditioning
Numerical solutions of ill-conditioned equations are not
to be trusted.
The reason is that round-off errors during the solution
process are equivalent to introducing small changes into
the coefficient matrix.
This in turn introduces large errors into the solution;
The magnitude of which depends on the severity of ill-
conditioning.
In suspect cases the determinant of the coefficient
matrix should be computed so that the degree of ill-
conditioning can be estimated.
This can be done during or after the solution with only a
small computational effort.