You are on page 1of 34

Until now, we have assumed that the signals are

deterministic, i.e., each value of a sequence is


uniquely determined.
In many situations, the processes that generate
signals are so complex as to make precise
description of a signal extremely difficult or
undesirable.
A random or stochastic signal is considered to be
characterized by a set of probability density
functions.
Discrete-time Random Signals
Random (or stochastic) process (or signal)
A random process is an indexed family of random
variables characterized by a set of probability
distribution function.
A sequence x[n], <n< . Each individual sample x[n] is
assumed to be an outcome of some underlying random
variable X
n
.
The difference between a single random variable and a
random process is that for a random variable the
outcome of a random-sampling experiment is mapped
into a number, whereas for a random process the
outcome is mapped into a sequence.
Stochastic Processes

Probability density function of x[n]:

Joint distribution of x[n] and x[m]:

Eg., x
1
[n] = A
n
cos(wn+|
n
), where A
n
and |
n
are
random variables for all < n < , then x
1
[n] is a
random process.
Stochastic Processes (continue)
( ) n x p
n
,
( ) m x n x p
m n
, , ,
x[n] and x[m] are independent iff


x is a stationary process iff


for all k.
That is, the joint distribution of x[n] and x[m]
depends only on the time difference m n.
Independence and Stationary
( ) ( ) ( ) m x p n x p m x n x p
m n m n
, , , , , =
( ) ( ) m x n x p k m x k n x p
m n k m k n
, , , , , , = + +
+ +
Particularly, when m = n for a stationary process:



It implies that x[n] is shift invariant.
Stationary (continue)
( ) ( ) n x p k n x p
n k n
, , = +
+
In many of the applications of discrete-time signal
processing, random processes serve as models for
signals in the sense that a particular signal can be
considered a sample sequence of a random
process.
Although such a signals are unpredictable
making a deterministic approach to signal
representation is inappropriate certain average
properties of the ensemble can be determined,
given the probability law of the process.
Stochastic Processes vs.
Deterministic Signal
Mean (or average)


c denotes the expectation operator


For independent random variables

Expectation
{ } ( )
}


= =
n n n n x
dx n x p x x m
n
, c
( ) { } ( ) ( )
}


=
n n n n
dx n x p x g x g , c
{ } { } { }
m n m n
y x y x c c c =
Mean squared value



Variance


Mean Square Value and
Variance
( )
}


=
n n n n
dx n x p x x ,
2 2
} { c
{ }
)
`

=
2
n
x n n
m x x c var
Autocorrelation




Autocovariance

Autocorrelation and
Autocovariance
{ }
( )
} }




-
-
=
=
m n m n m n
m n xx
dx dx m x n x p x x
x x n,m
, , ,
c | } {
( )( ) { }
-
=
=
m n
m n
x x xx
x m x n xx
m m n,m
m x m x n,m
} {
} {
|
c
*
For a stationary process, the autocorrelation is
dependent on the time difference m n.
Thus, for stationary process, we can write




If we denote the time difference by k, we have

Stationary Process
{ }
( ) { }
2
2
x n x
n x x
m x
x m m
n
=
= =
c o
c
( ) ( ) { }
-
+
= = +
n k n xx xx
x x k n k n c | | ,
In many instances, we encounter random
processes that are not stationary in the strict
sense.
If the following equations hold, we call the
process wide-sense stationary (w. s. s.).
Wide-sense Stationary
{ }
( ) { }
2
2
x n x
n x x
m x
x m m
n
=
= =
c o
c
( ) ( ) { }
-
+
= = +
n k n xx xx
x x k n k n c | | ,
For any single sample sequence x[n], define their
time average to be



Similarly, time-average autocorrelation is
Time Averages
| | | |

=

+
=
L
L n
l
n x
L
n x
1 2
1
lim
| | | | | | | | n x m n x
L
n x m n x
L
L n
l
-
=

-

+
+
= +
1 2
1
lim
A stationary random process for which time
averages equal ensemble averages is called an
ergodic process:






Ergodic Process
| |
x
m n x =
| | | | | | m n x m n x
xx
| = +
-
It is common to assume that a given sequence is
a sample sequence of an ergodic random
process, so that averages can be computed from
a single sequence.
Ergodic Process (continue)
| |
| | ( )
| | | | | | | |

=
- -

=
+ = +
=
=
1
0
1
0
2
2
1
0
1
1
1
L
n
L
L
n
x x
L
n
x
n x m n x
L
n x m n x
m n x
L
n x
L
m

o
In practice, we cannot
compute with the limits, but
instead the quantities.
Similar quantities are often
computed as estimates of the
mean, variance, and
autocorrelation.
Property 1:
Properties of correlation and
covariance sequences
| | { }
| | ( )( ) { }
| | { }
| | ( )( ) { }
-
+
-
+
-
+
-
+
=
=
=
=
y n x m n xy
n m n xy
x n x m n xx
n m n xx
m y m x m
y x m
m x m x m
x x m
c
c |
c
c |
| | | |
| | | |
-
=
=
y x xy xy
x xx xx
m m m m
m m m
|
|
2
Property 2:




Property 3
Properties of correlation and
covariance sequences (continue)
| |
| | Variance 0
Value Squared Mean 0
2
2
= =
=
(

=
x xy
n xx
x E
o
|
| | | | | | | |
| | | | | | | | m m m m
m m m m
xy xy xx xx
xy xy xx xx
- -
- -
= =
= =

| | | |


Property 4:




Properties of correlation and
covariance sequences (continue)
| | | | | |
| | | | | | 0 0
0 0
2
2
yy xx xy
yy xx xy
m
m

| | |
s
s
| | | |
| | | | 0
0
xx xx
xx xx
m
m

| |
s
s
Property 5:
If
Properties of correlation and
covariance sequences (continue)
| | | |
| | | | m m
m m
xx yy
xx yy

| |
=
=
0
n n n
x y

=
Since autocorrelation and autocovariance
sequences are all (aperiodic) one-dimensional
sequences, there Fourier transform exist and are
bounded in |w|st.
Let the Fourier transform of the autocorrelation
and autocovariance sequences be
Fourier Transform Representation
of Random Signals
| | ( ) | | ( )
| | ( ) | | ( )
jw
xy xy
jw
xx xx
jw
xy xy
jw
xx xx
e m e m
e m e m
I I
u u

| |


Consider the inverse Fourier Transforms:
Fourier Transform Representation
of Random Signals (continue)
| | ( )
| | ( ) dw e e m
dw e e m
jwn jw
xx xx
jwn jw
xx xx
}
}

u =
I =
t
t
t
t
t
|
t

2
1
2
1
Consequently,





Denote
to be the power density spectrum (or power
spectrum) of the random process x.
Fourier Transform Representation
of Random Signals (continue)
| | { } | | ( )
| | ( ) dw e e
dw e n x
jwn jw
xx xx x
jw
xx xx
}
}

I = =
u = =
t
t
t
t
t
o
t
| c
2
1
0
2
1
0
2
2
( ) ( )
jw
xx xx
e w P u =
The total area under power density in [t,t] is the
total energy of the signal.
P
xx
(w) is always real-valued since |
xx
(n) is
conjugate symmetric
For real-valued random processes, P
xx
(w) = u
xx
(e
jw
)
is both real and even.
Power Density Spectrum
| | { } ( )dw w P n x
xx
}

=
t
t
t
c
2
1
2
Consider a linear system with frequency response
h[n]. If x[n] is a stationary random signal with
mean m
x
, then the output y[n] is also a stationary
random signal with mean m
x
equaling to


Since the input is stationary, m
x
[nk] = m
x
, and
consequently,
Mean and Linear System
| | | | { } | | | | { } | | | | k n m k h k n x k h n y n m
x
k k
y
= = =


=

=
c c
| | ( )
x
j
k
x y
m e H k h m m
0
= =

=
If x[n] is a real and stationary random signal, the
autocorrelation function of the output process is





Since x[n] is stationary , c{x[nk]x[n+mr] }
depends only on the time difference m+kr.
Stationary and Linear System
| | | | | | { }
| | | | | | | |
| | | | | | | | { }

=
+ =

+ =
+ = +
k r
k r
yy
r m n x k n x r h k h
r m n x k n x r h k h
m n y n y m n n
c
c
c | ,
Therefore,




The output power density is also stationary.
Generally, for a LTI system having a wide-sense
stationary input, the output is also wide-sense
stationary.
Stationary and Linear System
(continue)
| |
| | | | | |
| | m
r k m r h k h
m n n
yy
k r
xx
yy
|
|
|
=
+ =
+

=
,
By substituting l = rk,




where

A sequence of the form of c
hh
[l] is called a
deterministic autocorrelation sequence.
Power Density Spectrum and
Linear System
| | | | | | | | | |
| | ( )

=
=
+ =
l
hh xx
l k
xx yy
l c l m
k l h k h k h l m m
|
| |
| | | | | |

=
+ =
k
hh
k l h k h l c
A sequence of the form of C
hh
[l] l = rk,


where C
hh
(e
jw
) is the Fourier transform of c
hh
[l].

For real h,

Thus
Power Density Spectrum and
Linear System (continue)
( ) ( ) ( )
jw
xx
jw
hh
jw
yy
e e C e u = u
| | | | | |
( ) ( ) ( )
jw jw jw
hh
hh
e H e H e C
l h l h l c
-
=
- =
( ) ( )
2
jw jw
hh
e H e C =
We have the relation of the input and the output
power spectrums to be the following:


Power Density Spectrum and
Linear System (continue)
( ) ( ) ( )
jw
xx
jw jw
yy
e e H e u = u
2
| | { } | | ( )
| | { } | | ( ) ( )
output the of power average total
2
1
0
input the of power average total
2
1
0
2
2
2
=
u = =
= u = =
}
}

dw e e H n y
dw e n x
jw
xx
jw
yy
jw
xx xx
t
t
t
t
t
| c
t
| c
Key property: The area over a band of
frequencies, w
a
<|w|<w
b
, is proportional to the
power in the signal in that band.
To show this, consider an ideal band-pass filter.
Let H(e
jw
) be the frequency of the ideal band
pass filter for the band w
a
<|w|<w
b
.
Note that |H(e
jw
)|
2
and u
xx
(e
jw
) are both even
functions. Hence,
Power Density Property
( ) ( ) ( ) ( )dw e e H dw e e H
b
a
a
b
w
w
jw
xx
jw
w
w
jw
xx
jw
} }
u + u =

2 2
2
1
2
1
t t
| | output in power average 0 =
yy
|
A white noise signal is a signal for which


Hence, its samples at different instants of time are
uncorrelated.
The power spectrum of a white noise signal is a
constant

The concept of white noise is very useful in
quantization error analysis.
White Noise (or White
Gaussian Noise)
( )
2
x
jw
xx
e o = u
| | | | m m
x xx
o o |
2
=
The average power of a white-noise is therefore


White noise is also useful in the representation
of random signals whose power spectra are not
constant with frequency.
A random signal y[n] with power spectrum u
yy
(e
jw
) can
be assumed to be the output of a linear time-invariant
system with a white-noise input.
White Noise (continue)
| | ( )
2 2
2
1
2
1
0
x x
jw
xx xx
dw dw e o o
t t
|
t
t
t
t
} }

= = u =
( ) ( )
2
2
x
jw jw
yy
e H e o = u
The cross-correlation between input and output of
a LTI system:





That is, the cross-correlation between the input
output is the convolution of the impulse response
with the input autocorrelation sequence.
Cross-correlation
| | | | | | { }
| | | | | |
| | | |

=
=

+ =
+ =
k
xx
k
xy
k m k h
k m n x k h n x
m n y n x m
|
c
c |
By further taking the Fourier transform on both sides
of the above equation, we have

This result has a useful application when the input is
white noise with variance o
x
2
.


These equations serve as the bases for estimating the
impulse or frequency response of a LTI system if it is
possible to observe the output of the system in response to
a white-noise input.
Cross-correlation (continue)
( ) ( ) ( )
jw
xx
jw jw
xy
e e H e u = u
| | | | ( ) ( )
jw
x
jw
xy x xy
e H e m h m
2 2
o o | = u = ,
Remained Materials Not Included
From Chap. 4, the materials will be
taught in the class without using
slides

You might also like