You are on page 1of 448

2012-09-05 Dan Ellis 1

ELEN E4810: Digital Signal Processing


Topic 1: Introduction
1. Course overview
2. Digital Signal Processing
3. Basic operations & block diagrams
4. Classes of sequences
2012-09-05 Dan Ellis 2
1. Course overview

Digital signal processing:


Modifying signals with computers

Web site:
http://www.ee.columbia.edu/~dpwe/e4810/

Book:
Mitra Digital Signal Processing
(3rd ed., 2005)

Instructor: dpwe@ee.columbia.edu
2012-09-05 Dan Ellis 3
Grading structure

Homeworks: 20%

Mainly from Mitra

Wednesday-Wednesday schedule

Collaborate, dont copy

Midterm: 20%

One session

Final exam: 30%

Project: 30%
2012-09-05 Dan Ellis 4
Course project

Goal: hands-on experience with DSP

Practical implementation

Work in pairs or alone

Brief report, optional presentation

Recommend MATLAB

Ideas on website

Dont copy! Cite your sources!


2012-09-05 Dan Ellis 5
Example past projects

Solo Singing Detection

Guitar Chord Classifier

Speech/Music Discrimination

Room sonar

Construction equipment monitoring

DTMF decoder

Reverb algorithms

Compression algorithms
o
n

w
e
b

s
i
t
e
W
2012-09-05 Dan Ellis 6
MATLAB

Interactive system for numerical


computation

Extensive signal processing library

Focus on algorithm, not implementation

Access:

Columbia Site License:


https://portal.seas.columbia.edu/matlab/

Student Version (need Sig. Proc. toolbox)

Engineering Terrace 251 computer lab


2012-09-05 Dan Ellis 7
Course at a glance
2012-09-05 Dan Ellis 8
2. Digital Signal Processing

Signals:
Information-bearing function

E.g. sound: air pressure variation at a


point as a function of time p(t)

Dimensionality:
Sound: 1-Dimension
Greyscale image i(x,y) : 2-D
Video: 3 x 3-D: {r(x,y,t) g(x,y,t) b(x,y,t)}
2012-09-05 Dan Ellis 9
Example signals

Noise - all domains

Spread-spectrum phone - radio

ECG - biological

Music

Image/video - compression

.
2012-09-05 Dan Ellis 10
Signal processing

Modify a signal to extract/enhance/


rearrange the information

Origin in analog electronics e.g. radar

Examples

Noise reduction

Data compression

Representation for recognition/


classification
2012-09-05 Dan Ellis 11
Digital Signal Processing

DSP = signal processing on a computer

Two effects: discrete-time, discrete level


x(t)
x[n]
2012-09-05 Dan Ellis 12
DSP vs. analog SP

Conventional signal processing:

Digital SP system:
Processor p(t) q(t)
Processor p(t) q(t) A/D D/A
p[n] q[n]
2012-09-05 Dan Ellis 13
Digital vs. analog

Pros

Noise performance - quantized signal

Use a general computer - flexibility, upgrde

Stability/duplicability

Novelty

Cons

Limitations of A/D & D/A

Baseline complexity / power consumption


2012-09-05 Dan Ellis 14
DSP example

Speech time-scale modification:


extend duration without altering pitch

M
2012-09-05 Dan Ellis 15
3. Operations on signals

Discrete time signal often obtained by


sampling a continuous-time signal

Sequence {x[n]} = x
a
(nT), n=-1,0,1,2

T= samp. period; 1/T= samp. frequency


2012-09-05 Dan Ellis 16
Sequences

Can write a sequence by listing values:

Arrow indicates where n=0

Thus,
{x[n]} = {. . . , 0.2, 2.2, 1.1, 0.2, 3.7, 2.9, . . .}

2012-09-05 Dan Ellis 17


Left- and right-sided

x[n] may be defined only for certain n:

N
1
n N
2
: Finite length (length = )

N
1
n: Right-sided (Causal if N
1
0)

n N
2
: Left-sided (Anticausal)

Can always extend with zero-padding


Right-sided Left-sided
2012-09-05 Dan Ellis 18
Operations on sequences

Addition operation:

Adder

Multiplication operation

Multiplier
A
x[n]
y[n]
x[n]
y[n]
w[n]
y[n] = x[n] +w[n]
y[n] = Ax[n]
2012-09-05 Dan Ellis 19
More operations

Product (modulation) operation:

Modulator

E.g. Windowing:
Multiplying an infinite-length sequence
by a finite-length window sequence
to extract a region
x[n]
y[n]
w[n]
y[n] = x[n] w[n]
2012-09-05 Dan Ellis 20
Time shifting

Time-shifting operation:
where N is an integer

If N > 0, it is delaying operation

Unit delay

If N < 0, it is an advance operation

Unit advance
y[n]
x[n]
y[n]
x[n]
2012-09-05 Dan Ellis 21
Combination of basic operations

Example


y[n] =
1
x[n] +
2
x[n 1]
+
3
x[n 2] +
4
x[n 3]
2012-09-05 Dan Ellis 22
Up- and down-sampling

Certain operations change the effective


sampling rate of sequences by adding
or removing samples

Up-sampling = adding more samples


= interpolation

Down-sampling = discarding samples


= decimation
2012-09-05 Dan Ellis 23
Down-sampling

In down-sampling by an integer factor


M > 1, every M-th sample of the input
sequence is kept and M - 1 in-between
samples are removed:

x
d
[n] = x[nM]

x
d
[n] M
2012-09-05 Dan Ellis 24
Down-sampling

An example of down-sampling

y[n] = x[3n]
3
2012-09-05 Dan Ellis 25
Up-sampling

Up-sampling is the converse of down-


sampling: L-1 zero values are inserted
between each pair of original values.
L
x
u
[n] =

x[n/L] n = 0, L, 2L, . . .
0 otherwise
2012-09-05 Dan Ellis 26
Up-sampling

An example of up-sampling
3
not inverse of downsampling!
2012-09-05 Dan Ellis 27
Complex numbers

.. a mathematical convenience that lead


to simple expressions

A second imaginary dimension (j-1)


is added to all values.

Rectangular form: x = x
re
+ jx
im

where magnitude |x| = (x
re
2
+ x
im
2
)
and phase = tan
-1
(x
im
/x
re
)

Polar form: x = |x| e


j

= |x|cos + j |x|sin
! (! ! ! )

e
j
= cos + j sin
2012-09-05 Dan Ellis 28
Complex math

When adding, real


and imaginary parts
add: (a+jb) + (c+jd)
= (a+c) + j(b+d)

When multiplying,
magnitudes multiply
and phases add:
re
j

se
j

= rse
j(

Phases modulo 2
x
2012-09-05 Dan Ellis 29
Complex conjugate

Flips imaginary part / negates phase:


Conjugate x* = x
re
jx
im
= |x| e
j()

Useful in resolving to real quantities:


x + x* = x
re
+ jx
im
+ x
re
jx
im
= 2x
re

xx* = |x| e
j()
|x| e
j()
= |x|
2
2012-09-05 Dan Ellis 30
Classes of sequences

Useful to define broad categories

Finite/infinite (extent in n)

Real/complex:
x[n] = x
re
[n] + jx
im
[n]
2012-09-05 Dan Ellis 31
Classification by symmetry

Conjugate symmetric sequence:



Conjugate antisymmetric:
x
ca
[n] = x
ca
*[-n] = x
re
[-n] + jx
im
[-n]
if x[n] = x
re
[n] + jx
im
[n]
then x
cs
[n] = x
cs
*[-n]
= x
re
[-n] jx
im
[-n]
2012-09-05 Dan Ellis 32
Conjugate symmetric decomposition

Any sequence can be expressed as


conjugate symmetric (CS) /
antisymmetric (CA) parts:
x[n] = x
cs
[n] + x
ca
[n]
where:
x
cs
[n] =
1
/
2
(x[n] + x*[-n]) = x
cs
*[-n]
x
ca
[n] =
1
/
2
(x[n] x*[-n]) = -x
ca
*[-n]

When signals are real,


CS Even (x
re
[n] = x
re
[-n]), CA Odd
2012-09-05 Dan Ellis 33
Basic sequences

Unit sample sequence:

Shift in time:
[n - k]

Can express any sequence with :


{
0
,
1
,
2
..}=
0
[n] +
1
[n-1] +
2
[n-2]..
1
4
3 2 1 0
1 2 3 4 5 6
n
1
k2 k1 k
k+1 k+2 k+3
n
2012-09-05 Dan Ellis 34
More basic sequences

Unit step sequence:

Relate to unit sample:




[n] =
1, n 0
0, n < 0




[n] = [n] [n 1]
[n] = [k]
k=
n

2012-09-05 Dan Ellis 35


Exponential sequences

Exponential sequences are


eigenfunctions of LTI systems

General form: x[n] = A


n

If A and are real (and positive):


|| > 1 || < 1
2012-09-05 Dan Ellis 36
Complex exponentials
x[n] = A
n

Constants A, can be complex :


A = |A|e
j
; = e
( + j!)
x[n] =

|A| e
n
e
j(!n + )
scale
varying
magnitude
varying
phase
2012-09-05 Dan Ellis 37
Complex exponentials

Complex exponential sequence can


project down onto real & imaginary
axes to give sinusoidal sequences
x
re
[n] = e
-n/12
cos(n/6) x
im
[n] = e
-n/12
sin(n/6) M


x[n] = exp (
1
12
+ j

6
)n
{ }
x
re
[n] x
im
[n]

e
j
= cos + j sin
2012-09-05 Dan Ellis 38
Periodic sequences

A sequence satisfying
is called a periodic sequence with a
period N where N is a positive integer and
k is any integer.
Smallest value of N satisfying
is called the fundamental period
2012-09-05 Dan Ellis 39
Periodic exponentials

Sinusoidal sequence and


complex exponential sequence
are periodic sequences of period N only if
with N & r positive integers

Smallest value of N satisfying


is the fundamental period of the
sequence

r = 1 one sinusoid cycle per N samples


r > 1 r cycles per N samples M

o
N = 2r
2012-09-05 Dan Ellis 40
Symmetry of periodic sequences

An N-point finite-length sequence x


f
[n]
defines a periodic sequence:

Symmetry of x
f
[n] is not defined
because x
f
[n] is undefined for n < 0

Define Periodic Conjugate Symmetric:


x[n] = x
f
[n
N
]
x
pcs
[n] =1/2 (x[n] +x

[n
N
])
=1/2

x
f
[n] +x

f
[N n]

1 n < N
n modulo N n
N
= n +rN
s.t. 0 n
N
< N, r Z
2012-09-05 Dan Ellis 41
Sampling sinusoids

Sampling a sinusoid is ambiguous:


x
1
[n] = sin(!
0
n)
x
2
[n] = sin((!
0
+2r)n) = sin(!
0
n) = x
1
[n]
2012-09-05 Dan Ellis 42
Aliasing

E.g. for cos(!n), ! = 2r



!
0

all (integer) r appear the same after
sampling

We say that a larger ! appears


aliased to a lower frequency

Principal value for discrete-time


frequency: 0

!
0

( i.e. less than 1/2 cycle per sample)
2012-09-12 Dan Ellis 1
ELEN E4810: Digital Signal Processing
Topic 2: Time domain
1. Discrete-time systems
2. Convolution
3. Linear Constant-Coefficient Difference
Equations (LCCDEs)
4. Correlation
2012-09-12 Dan Ellis 2
1. Discrete-time systems

A system converts input to output:

E.g. Moving Average (MA):


x[n]
DT System
y[n]
(M = 3)
x[n]
y[n]
+
z
-1
z
-1
1/M
1/M
1/M
x[n-1]
x[n-2]

y[n] =
1
M
x[n k]
k=0
M1

{y[n]} = f ({x[n]})
n
A
2012-09-12 Dan Ellis 3
Moving Average (MA)
x[n]
y[n]
+
z
-1
z
-1
1/M
1/M
1/M
x[n-1]
x[n-2]
n
x[n]
-1 1 2 3 4 5 6 7 8 9
A
n
x[n-1]
-1 1 2 3 4 5 6 7 8 9
n
x[n-2]
-1 1 2 3 4 5 6 7 8 9
n
y[n]
-1 1 2 3 4 5 6 7 8 9

y[n] =
1
M
x[n k]
k=0
M1

2012-09-12 Dan Ellis 4


MA Smoother

MA smoothes out rapid variations


(e.g. 12 month moving average)

e.g.
signal noise

y[n] =
1
5
x[n k]
k=0
4

5-pt
moving
average
x[n] = s[n] + d[n]
2012-09-12 Dan Ellis 5
Accumulator

Output accumulates all past inputs:


x[n] y[n] +
z
-1
y[n-1]


y[n] = x[]
=
n

= x[]
=
n1

+ x[n]
= y[n 1] + x[n]
2012-09-12 Dan Ellis 6
Accumulator
n
x[n]
-1 1 2 3 4 5 6 7 8 9
A
n
y[n-1]
-1 1 2 3 4 5 6 7 8 9
n
y[n]
-1 1 2 3 4 5 6 7 8 9
x[n] y[n] +
z
-1
y[n-1]
M
2012-09-12 Dan Ellis 7
Classes of DT systems

Linear systems obey superposition:

if input x
1
[n] output y
1
[n], x
2
y
2
...

given a linear combination


of inputs:

then output
for all , , x
1
, x
2
i.e. same linear combination of outputs
x[n]
DT system
y[n]
2012-09-12 Dan Ellis 8
Linearity: Example 1

Accumulator:
Linear


y[n] = x[]
=
n



y[n] = x
1
[] + x
2
[]
( )
=
n

= x
1
[]
( )

+ x
2
[]
( )

= x
1
[]

+ x
2
[]

= y
1
[n] + y
2
[n]

x[n] = x
1
[n] + x
2
[n]
2012-09-12 Dan Ellis 9
Linearity Example 2:

Teager Energy operator:


Nonlinear

y[n] = x
2
[n] x[n 1] x[n +1]

x[n] = x
1
[n] + x
2
[n]

y[n] = x
1
[n] +x
2
[n]
( )
2

x
1
[n 1] +x
2
[n 1]
( )

x
1
[n +1] +x
2
[n +1]
( )

y
1
[n] + y
2
[n]
2012-09-12 Dan Ellis 10
Linearity Example 3:

Offset accumulator:
but
Nonlinear
.. unless C = 0


y[n] = C+ x[]
=
n



y
1
[n] = C+ x
1
[]
=
n



y[n] = C+ x
1
[] +x
2
[]
( )
=
n

y
1
[n] +y
2
[n]
2012-09-12 Dan Ellis 11
Property: Shift (time) invariance

Time-shift of input
causes same shift in output

i.e. if x
1
[n] y
1
[n]
then

i.e. process doesnt depend on absolute


value of n

x[n] = x
1
[n n
0
]

y[n] = y
1
[n n
0
]
2012-09-12 Dan Ellis 12
Shift-invariance counterexample

Upsampler: L
Not shift invariant
y[n] =

x[n/L] n = 0, L, 2L, . . .
0 otherwise
y
1
[n] = x
1
[n/L] (n = r L)
x[n] = x
1
[n n
0
]
y[n] = x[n/L] = x
1
[n/L n
0
]
= x
1

n L n
0
L

= y
1
[n L n
0
] = y
1
[n n
0
]
x[n]
y[n]
2012-09-12 Dan Ellis 13
Another counterexample

Hence

If
then

Not shift invariant


- parameters depend on n


y[n] = n x[n]
scaling by time index

y
1
[n n
0
] = n n
0
( )
x
1
[n n
0
]
x[n] = x
1
[n n
0
]
y[n] = n x
1
[n n
0
]
2012-09-12 Dan Ellis 14
Linear Shift Invariant (LSI)

Systems which are both linear and


shift invariant are easily manipulated
mathematically

This is still a wide and useful class of


systems

If discrete index corresponds to time,


called Linear Time Invariant (LTI)
2012-09-12 Dan Ellis 15
Causality

If output depends only on past and


current inputs (not future), system is
called causal

Formally, if x
1
[n] y
1
[n] & x
2
[n] y
2
[n]
Causal

x
1
[n] = x
2
[n] n < N
y
1
[n] = y
2
[n] n < N
2012-09-12 Dan Ellis 16
Causality example

Moving average:
y[n] depends on x[n-k], k 0 causal

Centered moving average

.. looks forward in time noncausal

.. Can make causal by delaying



y[n] =
1
M
x[n k]
k=0
M1



y
c
n [ ] = y n + M 1 ( ) / 2 [ ]
=
1
M
x[n] + x[n k] + x[n + k]
k=1
(M 1) / 2
( )
2012-09-12 Dan Ellis 17
Impulse response (IR)

Impulse
(unit sample sequence)

Given a system:
if x[n] = [n] then y[n] = h[n]

LSI system completely specified by h[n]


n
[n]
-1 1 2 3 4 5 6 7
1
-2 -3
x[n]
DT system
y[n]

impulse response
2012-09-12 Dan Ellis 18
Impulse response example

Simple system:

1
x[n]
y[n]
+
z
-1
z
-1

3
n
x[n]
-1 1 2 3 4 5 6 7
1
-2 -3
n
y[n]
-1 1 2 3 4 5 6 7

1
-2 -3

3
x[n] = [n] impulse
y[n] = h[n] impulse response
2012-09-12 Dan Ellis 19
2. Convolution

Impulse response:

Shift invariance:

+ Linearity:

Can express any sequence with s:


x[n] = x[0][n] + x[1][n-1] + x[2][n-2]..
[n]
LSI
h[n]
[n-n
0
]
LSI
h[n-n
0
]
[n-k]
+ [n-l]
LSI
h[n-k]
+ h[n-l]
2012-09-12 Dan Ellis 20
Convolution sum

Hence, since

For LSI,
written as

Summation is symmetric in x and h


i.e. l = n k
Convolution
sum

x[n] = x[k][n k]
k=


y[n] = x[k]h[n k]
k=

*

y[n] = x[n] h[n]
*

x[n] h[n] = x[n l]h[l]
l=

= h[n] x[n]
*
2012-09-12 Dan Ellis 21
Convolution properties

LSI System output y[n] = input x[n]


convolved with impulse response h[n]
h[n] completely describes system

Commutative:

Associative:

Distributive:
x[n] h[n] = h[n] x[n]
* *
(x[n] h[n]) y[n] = x[n] (h[n] y[n])
* * * *
h[n] (x[n] + y[n]) = h[n] x[n] + h[n] y[n]
* *
*
2012-09-12 Dan Ellis 22
Interpreting convolution

Passing a signal through a (LSI) system


is equivalent to convolving it with the
systems impulse response
x[n]
h[n]
y[n] = x[n] h[n]

x[n]={0 3 1 2 -1}
h[n] = {3 2 1}
n
-1 1 2 3 4 5 6 7 -2 -3
n
-1 1 2 3 5 6 7 -2 -3

y[n] = x[k]h[n k]
k=

= h[k]x[n k]
k=

2012-09-12 Dan Ellis 23


Convolution interpretation 1

Time-reverse h,
shift by n, take inner
product against
fixed x
k
-1 1 2 3 5 6 7 -2 -3
x[k]
k
-1 1 2 3 4 5 6 7 -2 -3
h[0-k]
k
-1 1 2 3 4 5 6 7 -2 -3
h[1-k]
k
-1 1 2 3 4 5 6 7 -2 -3
h[2-k]
n
-1 1 2 3 5 7 -2 -3
y[n]
0
9 9
11
2
-1

y[n] = x[k]h[n k]
k=

=g[k]
=g[k-1]
call h[-n] = g[n]
2012-09-12 Dan Ellis 24
Convolution interpretation 2

Shifted xs
weighted by
points in h

Conversely,
weighted,
delayed
versions of h ...
n
-1 1 2 3 5 7 -2 -3
y[n]
0
9 9
11
2
-1
n
-1 1 2 3 5 6 7 -2 -3
x[n]
n
-1 1 2 3 4 6 7 -2 -3
x[n-1]
n
-1 1 2 3 4 5 7 -2 -3
x[n-2]
k
-
1
1
2
3
4
5
6
7
-
2
-
3
h
[
k
]

y[n] = h[k]x[n k]
k=

2012-09-12 Dan Ellis 25


Matrix interpretation

Diagonals in X matrix are equal



y[n] = x[n k]h[k]
k=



y[0]
y[1]
y[2]
...










=
x[0]
x[1]
x[2]
...
x[1]
x[0]
x[1]
...
x[2]
x[1]
x[0]
...










h[0]
h[1]
h[2]








2012-09-12 Dan Ellis 26
Convolution notes

Total nonzero length of convolving N


and M point sequences is N+M-1

Adding the indices of the terms within


the summation gives n :
i.e. summation indices move in opposite
senses

y[n] = h[k]x[n k]
k=


k + n k
( )
= n
2012-09-12 Dan Ellis 27
Convolution in MATLAB

The M-file conv implements the


convolution sum of two finite-length
sequences

If
then conv(a,b) yields


a = [0 3 1 2 -1]


b = [3 2 1]


[0 9 9 11 2 0 -1]
M
2012-09-12 Dan Ellis 28
Connected systems

Cascade connection:
Impulse response h[n] of the cascade of
two systems with impulse responses
h
1
[n] and h
2
[n] is

By commutativity,
*
*

h[n] = h
1
[n]
2012-09-12 Dan Ellis 29
Inverse systems

[n] is identity for convolution


i.e.

Consider

h
2
[n] is the inverse system of h
1
[n]
x[n] [n]
=
x[n] *
x[n] y[n] z[n]
=
x[n]
z[n]
=
h
2
[n] y[n]
=
h
2
[n] h
1
[n] x[n]
* * *
if
*

h
2
[n] h
1
[n] = [n]
2012-09-12 Dan Ellis 30
Inverse systems

Use inverse system to recover input x[n]


from output y[n] (e.g. to undo effects of
transmission channel)

Only sometimes possible - e.g. cannot


invert h
1
[n] = 0

In general, attempt to solve


*

h
2
[n] h
1
[n] = [n]
2012-09-12 Dan Ellis 31
Inverse system example

Accumulator:
Impulse response h
1
[n] = [n]

Backwards difference
.. has desired property:

Thus, backwards difference is inverse


system of accumulator.

[n] [n 1] = [n]
n
-1 1 2 3 5 6 7 -2 -3 4
n
-1 1 2 3 5 6 7 -2 -3 4
2012-09-12 Dan Ellis 32
Parallel connection

Impulse response of two parallel


systems added together is:
2012-09-12 Dan Ellis 33
3. Linear Constant-Coefficient
Difference Equation (LCCDE)

General spec. of DT, LSI, finite-dim sys:

Rearrange for y[n] in causal form:

WLOG, always have d


0
= 1

defined by {d
k
},{p
k
}
order = max(N,M)

d
k
y[n k]
k=0
N

= p
k
x[n k]
k=0
M


y[n] =
d
k
d
0
y[n k]
k=1
N

+
p
k
d
0
x[n k]
k=0
M

2012-09-12 Dan Ellis 34


Solving LCCDEs

Total solution
Complementary Solution
satisfies
Particular Solution
for given forcing function
x[n]

y[n] = y
c
[n] + y
p
[n]

d
k
y[n k]
k=0
N

= 0
2012-09-12 Dan Ellis 35
Complementary Solution

General form of unforced oscillation


i.e. systems natural modes

Assume y
c
has form
Characteristic polynomial
of system - depends only on {d
k
}

y
c
[n] =
n


d
k

nk
k=0
N

= 0

nN
d
0

N
+ d
1

N1
++ d
N1
+ d
N
( )
= 0
d
k

Nk
k=0
N

= 0
2012-09-12 Dan Ellis 36
Complementary Solution

factors into roots


i
, i.e.

Each/any
i
satisfies eqn.

Thus, complementary solution:


Any linear combination will work

i
s are free to match initial conditions

d
k

Nk
k=0
N

= 0

(
1
)(
2
)... = 0

y
c
[n] =
1

1
n
+
2

2
n
+
3

3
n
+ ...
2012-09-12 Dan Ellis 37
Complementary Solution

Repeated roots in chr. poly:

Complex
i
s sinusoidal

y
c
[n] =
1

1
n
+
2
n
1
n
+
3
n
2

1
n
+... +
L
n
L1

1
n
+ ...

(
1
)
L
(
2
)... = 0

y
c
[n] =
i

i
n
2012-09-12 Dan Ellis 38
Particular Solution

Recall: Total solution

Particular solution reflects input

Modes usually decay away for large n


leaving just y
p
[n]

Assume form of x[n], scaled by :


e.g. x[n] constant y
p
[n] =
x[n] =
0
n
y
p
[n] =
0
n
(
0

i
)
or = n
L

0
n
(
0

i
)

y[n] = y
c
[n] + y
p
[n]
2012-09-12 Dan Ellis 39
LCCDE example

Need input: x[n] = 8[n]

Need initial conditions:


y[-1] = 1, y[-2] = -1
x[n] y[n] +

y[n] + y[n 1] 6y[n 2] = x[n]
2012-09-12 Dan Ellis 40
LCCDE example

Complementary solution:


1
,
2
are unknown at this point
roots
1
= -3,
2
= 2

y[n] + y[n 1] 6y[n 2] = 0; y[n] =
n

n2

2
+ 6
( )
= 0
+ 3
( )
2
( )
= 0

y
c
[n] =
1
3
( )
n
+
2
2
( )
n
2012-09-12 Dan Ellis 41
LCCDE example

Particular solution:

Input x[n] is constant = 8[n]


assume y
p
[n] = , substitute in:
(large n)

y[n] + y[n 1] 6y[n 2] = x[n]
+ 6 = 8[n]
4 = 8 = 2
2012-09-12 Dan Ellis 42

Total solution

Solve for unknown


i
s by substituting
initial conditions into DE at n = 0, 1, ...

n = 0
LCCDE example
from ICs
=
1
3
( )
n
+
2
2
( )
n
+

y[n] = y
c
[n] + y
p
[n]

y[n] + y[n 1] 6y[n 2] = x[n]

y[0] + y[1] 6y[2] = x[0]

1
+
2
+ +1+ 6 = 8

1
+
2
= 3
2012-09-12 Dan Ellis 43
LCCDE example

n = 1

solve:
1
= -1.8,
2
= 4.8

Hence, system output:

Dont find
i
s by solving with ICs at
n = -1,-2
(ICs may not reflect natural modes;
Mitra example 2.37/38 is wrong)

y[1] + y[0] 6y[1] = x[1]

1
(3) +
2
(2) + +
1
+
2
+ 6 = 8
2
1
+ 3
2
=18

y[n] = 1.8(3)
n
+ 4.8(2)
n
2 n 0
M
2012-09-12 Dan Ellis 44
LCCDE solving summary

Difference Equation (DE):


Ay[n] + By[n-1] + ... = Cx[n] + Dx[n-1] + ...
Initial Conditions (ICs): y[-1] = ...

DE RHS = 0 with y[n]=


n
roots {
i
}
gives complementary soln

Particular soln: y
p
[n] ~ x[n]
solve for
0
n
at large n


i
s by substituting DE at n = 0, 1, ...
ICs for y[-1], y[-2]; y
t
=y
c
+y
p
for y[0], y[1]

y
c
[n] =
i

i
n

2012-09-12 Dan Ellis 45


LCCDEs: zero input/zero state

Alternative approach to solving


LCCDEs is to solve two subproblems:

y
zi
[n], response with zero input (just ICs)

y
zs
[n], response with zero state (just x[n])

Because of linearity, y[n] = y


zi
[n]+y
zs
[n]

Both subproblems are real

But, have to solve for


i
s twice
(then sum them)
2012-09-12 Dan Ellis 46
Impulse response of LCCDEs

Impulse response:
i.e. solve with x[n] = [n] y[n] = h[n]
(zero ICs)

With x[n] = [n], form of y


p
[n] = [n]
solve y[n] for n = 0,1, 2... to find
i
s
[n] LCCDE
h[n]
2012-09-12 Dan Ellis 47

y[0] + y[1] 6y[2] = x[0]
LCCDE IR example

e.g.
(from before); x[n] = [n]; y[n] = 0 for n<0

n = 0:

n = 1:

n = 2:

thus
1
n 0
Infinite length

y[n] + y[n 1] 6y[n 2] = x[n]

y
c
[n] =
1
3
( )
n
+
2
2
( )
n

h[n] = 0.6 3
( )
n
+ 0.4 2
( )
n
M
y
p
[n] = [n]

1
+
2
+ = 1

1
= 0.6,
2
= 0.4, = 0

1
(3) +
2
(2) + 1 = 0

1
(9) +
2
(4) 1 6 = 0
2012-09-12 Dan Ellis 48
System property: Stability

Certain systems can be unstable e.g.


Output grows without limit in some
conditions
x[n] y[n] +
z
-1
2
n
y[n]
-1 1 2 3 4
...
2012-09-12 Dan Ellis 49
Stability

Several definitions for stability; we use


Bounded-input, bounded-output
(BIBO) stable

For every bounded input


output is also subject to a finite bound,

x[n] < B
x
n

y[n] < B
y
n
2012-09-12 Dan Ellis 50
Stability example

MA filter:
BIBO Stable

y[n] =
1
M
x[n k]
k=0
M1


y[n] =
1
M
x[n k]
k=0
M1

1
M
x[n k]
k=0
M1

1
M
M B
x
B
y
2012-09-12 Dan Ellis 51
Stability & LCCDEs

LCCDE output is of form:

s and s depend on input & ICs,


but to be bounded for any input
we need || < 1

y[n] =
1

1
n
+
2

2
n
+ ... +
0
n
+ ...
2012-09-12 Dan Ellis 52
4. Correlation

Correlation ~ identifies similarity


between sequences:

Note:
Cross
correlation
of x against y
lag
call m = n
M
r
xy
[] =

n=
x[n]y[n ]
r
yx
[] =

n=
y[n]x[n ]
=

m=
y[m+ ]x[m] = r
xy
[]
2012-09-12 Dan Ellis 53
Correlation and convolution

Correlation:

Convolution:

Hence:
Correlation may be calculated by
convolving with time-reversed sequence
r
xy
[n] =

k=
x[k]y[k n]
x[n] y[n] =

k=
x[k]y[n k]
r
xy
[n] = x[n] y[n]
2012-09-12 Dan Ellis 54
Autocorrelation

Autocorrelation (AC) is correlation of


signal with itself:

Note:
Energy of
sequence x[n]


r
xx
[] = x[n]x[n ]
n=

= r
xx
[]

r
xx
[0] = x
2
[n]
n=

=
x
2012-09-12 Dan Ellis 55
Correlation maxima

Note:

Similarly:

From geometry,

when x//y, cos = 1, else cos < 1


angle
between
x and y


r
xx
[] r
xx
[0]
r
xx
[]
r
xx
[0]
1


r
xy
[]
x

y

r
xy
[]
r
xx
[0]r
yy
[0]
1

xy = x
i
y
i
i

= x y cos

x
i
2

2012-09-12 Dan Ellis 56


AC of a periodic sequence

Sequence of period N:

Calculate AC over a finite window:


if M >> N

x [n] = x [n + N]


r
x x
[] =
1
2M +1
x [n] x [n ]
n=M
M

=
1
N
x [n] x [n ]
n=0
N1

2012-09-12 Dan Ellis 57


AC of a periodic sequence

i.e AC of periodic sequence is periodic


Average energy per
sample or Power of x

r
x x
[0] =
1
N
x
2
[n]
n=0
N1

= P
x


r
x x
[ + N] =
1
N
x [n] x [n N]
n=0
N1

= r
x x
[]
2012-09-12 Dan Ellis 58
What correlations look like

AC of any x[n]

AC of periodic

Cross correlation

r
x x
[

]

r
xx
[

]

r
xy
[

2012-09-12 Dan Ellis 59


What correlation looks like
2012-09-12 Dan Ellis
Correlation in action

Close mic vs.


video camera
mic
60

Short-time
cross-correlation
2012-09-24 Dan Ellis 1
ELEN E4810: Digital Signal Processing
Topic 3: Fourier domain
1. The Fourier domain
2. Discrete-Time Fourier Transform (DTFT)
3. Discrete Fourier Transform (DFT)
4. Convolution with the DFT
2012-09-24 Dan Ellis 2
1. The Fourier Transform

Basic observation (continuous time):


A periodic signal can be decomposed
into sinusoids at integer multiples of the
fundamental frequency

i.e. if
we can approach with
Harmonics
of the
fundamental

x (t) = x (t +T)

x
x(t)
M

k=0
a
k
cos

2k
T
t +
k

2012-09-24 Dan Ellis


1.5 1 0.5 0 0.5 1 1.5
1
0.5
0
0.5
1
3
Fourier Series

For a square wave,


i.e.
M
M

k=0
a
k
cos

2k
T
t +
k

x(t) = cos

2
T
t

1
3
cos

2
T
3t

+
1
5
cos

2
T
5t

. . .

k
= 0; a
k
=

(1)
k1
2
1
k
k = 1, 3, 5, . . .
0 otherwise
2012-09-24 Dan Ellis 4
Fourier domain

x is equivalently described
by its Fourier Series
parameters:

Complex form:
Negative a
k
is
equivalent to phase of
k
1 2 3 5 6 7 4
a
k
1.0
k
1 2 3 5 6 7 4

a
k
= (1)
k1
2
1
k
k = 1, 3, 5, . . .
x(t)
M

k=M
c
k
e
j
2k
T
t
2012-09-24 Dan Ellis 5
Fourier analysis

How to find ?
Inner product with complex sinusoids:
but
e
j

= cos + jsin
{|c
k
|}, {arg{c
k
}}
=
1
T

x(t) cos(
2k
T
t)dt j

x(t) sin(
2k
T
t)dt

x(t)
M

k=M
c
k
e
j
2k
T
t
c
k
=
1
T

T/2
T/2
x(t)e
j
2k
T
t
dt
2012-09-24 Dan Ellis 6
Fourier analysis

Consider
.. so c
k
should = 0 except k = l

Then


c
k
=
1
T
x t ( ) cos
2kt
T
dt j x t ( ) sin
2kt
T
dt

( )
=
1
T
cos
2lt
T
cos
2kt
T
dt j cos
2lt
T
sin
2kt
T
dt

( )
0
evenodd

x(t) = cos

l
2
T
t

x(t)
M

k=M
c
k
e
j
2k
T
t
2012-09-24 Dan Ellis 7
Fourier analysis

Works if k, l are positive integers,

(say
T=2)
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
-1
-0.5
0
0.5
1
cos(1t)
cos(2t)
t /
=
1
4

cos(k + l)t + cos(k l)tdt


=
1
4

sin(k+l)t
k+l
+
sin(kl)t
kl

=
1
2
(sinc(k + l) + sinc(k l))
1
2

cos(kt) cos(lt)dt =

1 k = l
0 otherwise
2012-09-24 Dan Ellis 8
sinc

= 1 when x = 0
= 0 when x = r, r 0, r = 1, 2, 3,...

sincx =
sin x
x

1.0
2 3 4 4 3 2
sin(x)
sin(x)
/x
y=x
x
2012-09-24 Dan Ellis 9
Fourier Analysis

Thus,
because real & imag sinusoids in
pick out the corresponding sinusoidal
components linearly combined
in
c
k
=
1
T

T/2
T/2
x(t)e
j
2k
T
t
dt
e
j
2k
T
t
x(t) =
M

k=M
c
k
e
j
2k
T
t
2012-09-24 Dan Ellis 10
Fourier Transform

Fourier series for periodic signals


extends naturally to Fourier Transform
for any (CT) signal (not just periodic):

Discrete index k continuous freq.


Inverse Fourier
Transform (IFT)
Fourier
Transform (FT)
x(t) =
1
2

X(j)e
jt
d
X(j) =

x(t)e
jt
dt
2012-09-24 Dan Ellis 11
Fourier Transform

Mapping between two continuous


functions:
2 ambiguity
0 0.002 0.004 0.006 0.008
time / sec
level
/ dB
-0.01
0
0.01
0.02
x(t)
0 2000 4000 6000 8000
freq / Hz
-80
-60
-40
-20
0
|
X()
|
0 2000 4000 6000 8000
freq / Hz
0
/2
/2

arg{X()}
2012-09-24 Dan Ellis 12
Fourier Transform of a sine

Assume
Now, since
...we know
...where (x) is the Dirac delta function
(continuous time) i.e.


f(x)
x
x
0
(x-x
0
)

x(t) = e
j
0
t


x x
0
( )

f x ( )dx = f x
0
( )

x(t) = Ae
j
0
t

X() = A(
0
)
x(t) =
1
2p
Z

X(W)e
jWt
dW
X(W) = 2pd(WW
0
)
2012-09-24 Dan Ellis 13
Fourier Transforms
Time Frequency
Fourier Series
(FS)
Continuous
periodic x(t)
Discrete
infinite c
k
Fourier
Transform (FT)
Continuous
infinite x(t)
Continuous
infinite X()
Discrete-Time
FT (DTFT)
Discrete
infinite x[n]
Continuous
periodic X(e
j
)
Discrete FT
(DFT)
Discrete
finite/pdc x[n]
Discrete
finite/pdc X[k]
~
~
2012-09-24 Dan Ellis 14
2. Discrete Time FT (DTFT)

FT defined for discrete sequences:

Summation (not integral)

Discrete (normalized)
frequency variable !

Argument is e
j!
, not j!
DTFT

X(e
j
) = x[n]e
jn
n=

2012-09-24 Dan Ellis 15


DTFT example

e.g. x[n] =
n
[n], || < 1

n
-1 1 2 3 4 5 6 7

X(e
j
) =
n
[n]e
jn
n=

= e
j
( )
n
n=0

=
1
1e
j


S = c
n
n=0

cS = c
n
n=1

S cS = c
0
=1
S =
1
1 c
Im
Re
n
1 - e
-j
0
|X(e
j
)|

2 3 4 5
1
2
3

arg{X(e
j
)}

2 3 4 5
( |c | < 1 )
2012-09-24 Dan Ellis 16
Periodicity of X(e
j!
)

X(e
j!
) has periodicity 2 in ! :

Phase ambiguity of e
j!

makes it implicit

X(e
j (+2 )
) = x[n]

e
j (+2 )n
= x[n]

e
jn
e
j 2n
= X(e
j
)
0

|X(e
j
)|

2 3 4 5
1
2
3

arg{X(e
j
)}

2 3 4 5
2012-09-24 Dan Ellis 17
Inverse DTFT (IDTFT)

Same basic form as other IFTs:

Note: continuous, periodic X(e


j!
)
discrete, infinite x[n] ...

IDTFT is actually forward Fourier Series


(except for sign of !)
IDTFT

x[n] =
1
2
X(e
j
)e
jn
d

2012-09-24 Dan Ellis 18


IDTFT

Verify by substituting in DTFT:


= 0 unless
n = l
i.e. = [n-l]

x[n] =
1
2
X(e
j
)e
jn
d

=
1
2
x[l]e
jl
l

( )
e
jn
d

= x[l]
l

1
2
e
j(nl )
d

= x[l]sinc
l

(n l) = x[n]
2012-09-24 Dan Ellis 19
sinc again

Same as cos imag jsin part cancels

1
2

e
j(nl)
dw =
1
2

e
j(nl)
j(n l)

=
1
2

e
j(nl)
e
j(nl)
j(n l)

=
1
2

2j sin(n l)
j(n l)

= sinc (n l)
2012-09-24 Dan Ellis 20

x[n] = [n]

i.e. x[n] X(e


j!
)
[n] 1
DTFTs of simple sequences
(for all !)
n
-2 2 3 1 -1 -3
!
-
x[n]
X(e
j

)

X(e
j
) = x[n]e
jn
n=

= e
j0
=1
2012-09-24 Dan Ellis 21
DTFTs of simple sequences

:
over - < ! <
but X(e
j!
) must be periodic in !

If !
0
= 0 then x[n] = 1 n
so
IDTFT

x[n] = e
j
0
n

x[n] =
1
2
X(e
j
)e
jn
d


X(e
j
) = 2 (
0
)


e
j
0
n
2 (
0
2k)
k


1 2 ( 2k)
k

X(e
j
!
)
0 24
2012-09-24 Dan Ellis 22


[n]
1
1 e
j
+ ( + 2k)
k

DTFTs of simple sequences

From before:

[n] tricky - not finite


( | | < 1)
DTFT of 1/2

n
[n]
1
1e
j
2012-09-24 Dan Ellis 23
DTFT properties

Linear:

Time shift:

Frequency shift:
delay
in
frequency

g[n] + h[n] G(e
j
) + H(e
j
)

g[n n
0
] e
jn
0
G(e
j
)

e
j
0
n
g[n] G e
j
0
( )
( )
2012-09-24 Dan Ellis 24
DTFT example

x[n] = [n] +
n
[n-1] ?
= [n] + (
n-1
[n-1])

x[n] =
n
[n]

X(e
je
) =1+o e
je1

1
1oe
je
|
\

|
.
|
=1+
oe
je
1oe
je
=
1oe
je
+oe
je
1oe
je
=
1
1oe
je
2012-09-24 Dan Ellis 25
DTFT symmetry

If x[n] X(e
j!
) then...
x[-n] X(e
-j!
)
x
*
[n] X
*
(e
-j!
)
Re{x[n]} X
CS
(e
j!
)
jIm{x[n]} X
CA
(e
j!
)
x
cs
[n] Re{X(e
j!
)}
x
ca
[n] jIm{X(e
j!
)}
conjugate symmetry cancels Im parts on IDTFT
from summation
(e
-j!
)
*
= e
j!

=
1
2
X e
j
( )
+ X
*
e
j
( ) [ ]

=
1
2
X e
j
( )
X
*
e
j
( ) [ ]

X(e
j
) = x[n]e
jn
n=

2012-09-24 Dan Ellis

When x[n] is pure real, X(e


j!
) = X
*
(e
-j!
)
x
cs
[n] x
ev
[n] = x
ev
[-n] X
R
(e
j!
) = X
R
(e
-j!
)
x
ca
[n] x
od
[n] = -x
od
[-n] X
I
(e
j!
) = -X
I
(e
-j!
)
26
DTFT of real x[n]
x[n] real, even
X(e
j!
) even, real
Real
n
I
m
a
g
x
re
[n]
x
im
[n]
X
CS
2012-09-24 Dan Ellis 27
DTFT and convolution

Convolution:


Convolution
becomes
multiplication

x[n] = g[n] h[n]


X(e
j
) = g[n] h[n] ( )e
jn
n=

= g[k]h[n k]
k

( )
n

e
jn
= g[k]e
jk
h[n k]e
j(nk )
n

( )
k

= G(e
j
) H(e
j
)

g[n] h[n] G(e
j
)H(e
j
)
2012-09-24 Dan Ellis 28
Convolution with DTFT

Since
we can calculate a convolution by:

finding DTFTs of g, h G, H

multiply them: GH

IDTFT of product is result,



g[n]
h[n]
y[n]
DTFT
DTFT
IDTFT

G(e
j
)

g[n] h[n] G(e
j
)H(e
j
)

H(e
j
)

Y(e
j
)
M

g[n] h[n]
2012-09-24 Dan Ellis 29

x[n] =
n
[n]

h[n] = [n] - [n-1]


y[n] = x[n] h[n]



y[n] = [n] i.e. ...
DTFT convolution example

X(e
j
) =
1
1e
j

H(e
j
) =1 e
j1
( )
1

Y(e
j
) = H(e
j
)X(e
j
)

=
1
1e
j
1e
j
( )
=1
2012-09-24 Dan Ellis 30
DTFT modulation

Modulation:
Could solve if g[n] was just sinusoids...

Dual of convolution in time



x[n] = g[n] h[n]

X(e
je
) =
1
2t
G(e
ju
)e
jun
du
t
t
}
|
\

|
.
|
n

h[n]e
jen
=
1
2t
G(e
ju
) h[n]e
j (eu )n
n

[ ]
du
t
t
}

g[n] h[n]
1
2
G(e
j
)H(e
j ( )
)d

2012-09-24 Dan Ellis 31


Parsevals relation

Energy in time and frequency domains


are equal:

If g = h, then gg
*
= |g|
2
= energy...

g[n]h
*
[n]
n

=
1
2
G(e
j
)H
*
(e
j
)d

2012-09-24 Dan Ellis 32


Energy density spectrum

Energy of sequence

By Parseval

Define Energy Density Spectrum (EDS)


g
= g[n]
n

g
=
1
2
G(e
j
)
2
d


S
gg
(e
j
) = G(e
j
)
2
2012-09-24 Dan Ellis 33
EDS and autocorrelation

Autocorrelation of g[n]:

If g[n] is real, G(e


-j!
) = G
*
(e
j!
), so

Mag-sq of spectrum is DTFT of autoco


no phase
info.


r
gg
[] = g[n]g[n ]
n=

= g[n] g[n]


DTFT r
gg
[]
{ }
= G(e
j
)G(e
j
)


DTFT r
gg
[]
{ }
= G(e
j
)
2
= S
gg
(e
j
)
2012-09-24 Dan Ellis 34
3. Discrete FT (DFT)

A finite or periodic sequence has only


N unique values, x[n] for 0 n < N

Spectrum is completely defined by N


distinct frequency samples

Divide 0..2 into N equal steps,


{!
k
} = 2k
/
N
Discrete FT
(DFT)
Discrete
finite/pdc x[n]
Discrete
finite/pdc X[k]
2012-09-24 Dan Ellis 35

Uniform sampling of DTFT spectrum:

DFT:
where i.e. 1/N
th
of a revolution
DFT and IDFT

X[k] = X(e
j
)
=
2k
N
= x[n]e
j
2k
N
n
n=0
N1


X[k] = x[n]W
N
kn
n=0
N1


W
N
= e
j
2
N
1
2/N
W
N
2012-09-24 Dan Ellis 36
IDFT

Inverse DFT IDFT

Check:
Sum of complete set
of rotated vectors
= 0 if l n; = N if l = n
re
im
W
N
W
N
2

x[n] =
1
N
X[k]W
N
nk
k=0
N1


x[n] =
1
N
x[l]W
N
kl
l

( )
W
N
nk
k

=
1
N
x[l] W
N
k (ln)
k=0
N1

l=0
N1

= x[n]
0 n < N
or finite
geometric series
= (1-W
N
lN
)/(1-W
N
l
)
2012-09-24 Dan Ellis 37
DFT examples

Finite impulse

Periodic sinusoid:



x[n] =
1 n = 0
0 n =1..N 1




X[k] = x[n]W
N
kn
n=0
N1

= W
N
0
=1 k

X[k] =
1
2
W
N
rn
+W
N
rn
( )
n=0
N1

W
N
kn
=
N / 2 k = r,k = N r
0 o.w.



(0 k < N)
x[n] = cos

2rn
N

=
1
2

W
rn
N
+W
rn
N

(r Z)
2012-09-24 Dan Ellis 38
DFT: Matrix form

as a matrix multiply:

i.e.

X[k] = x[n] W
N
kn
n=0
N1


X= D
N
x

X[0]
X[1]
X[2]
.
.
.
X[N 1]

1 1 1 1
1 W
1
N
W
2
N
W
(N1)
N
1 W
2
N
W
4
N
W
2(N1)
N
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1 W
(N1)
N
W
2(N1)
N
W
(N1)
2
N

x[0]
x[1]
x[2]
.
.
.
x[N 1]

2012-09-24 Dan Ellis 39


Matrix IDFT

If
then

i.e. inverse DFT is also just a matrix,


=
1
/
N
D
N
*

X= D
N
x

x = D
N
1
X
D
1
N
=
1
N

1 1 1 1
1 W
1
N
W
2
N
W
(N1)
N
1 W
2
N
W
4
N
W
2(N1)
N
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1 W
(N1)
N
W
2(N1)
N
W
(N1)
2
N

2012-09-24 Dan Ellis 40


DFT and MATLAB

MATLAB is concerned with sequences


not continuous functions like X(e
j!
)

Instead, we use the DFT to sample


X(e
j!
) on an (arbitrarily-fine) grid:

X = freqz(x,1,w); samples the DTFT


of sequence x at angular frequencies in w

X = fft(x); calculates the N-point DFT


of an N-point sequence x
M
2012-09-24 Dan Ellis 41
DFT and DTFT

DFT samples DTFT at discrete freqs:


DTFT
DFT
continuous freq !
infinite x[n], -<n<
discrete freq k=N!/2
finite x[n], 0n<N
!
X(e
j!
)
X[k]
k=1...

X(e
j
) = x[n]e
jn
n=


X[k] = x[n]W
N
kn
n=0
N1


X[k] = X(e
j
)
=
2k
N
2012-09-24 Dan Ellis 42
DTFT from DFT

N-point DFT completely specifies the


continuous DTFT of the finite sequence
periodic
sinc
interpolation

X(e
je
) =
1
N
X[k]W
N
kn
k=0
N1

|
\

|
.
|
e
jen
n=0
N1

=
1
N
X[k]
k=0
N1

e
j e
2tk
N
( )
n
n=0
N1

=
1
N
X[k]
k=0
N1


sin N
Ae
k
2
sin
Ae
k
2
e
j
( N1)
2
Ae
k

k
=
2k
N
2012-09-24 Dan Ellis 43
Periodic sinc

= N when !
k
= 0; = (-)N when !
k
/2 =
= 0 when !
k
/2 = r/N, r = 1, 2, ...
other values in-between...


e
j
k
n
n=0
N1

=
1 e
jN
k
1 e
j
k
=
e
jN
k
/ 2
e
j
k
/ 2

e
jN
k
/ 2
e
jN
k
/ 2
e
j
k
/ 2
e
j
k
/ 2
= e
j
( N1)
2

k
sin N

k
2
sin

k
2
pure phase
pure real
2012-09-24 Dan Ellis 44
Periodic sinc
DFT DTFT
= interpolation
by periodic
sinc
X[k]X(e
j!
)

sin Nx
sin x
(N = 8)
2
sinx
sinNx
sinNx
/
sinx
x

0
N
-N
-1 0 k = 1
= 2/N
k = 3
= 6/N
k = 4
= 8/N
-0.5
0
0.5
1
1.5
X(e
j
0) = X[k]
sin N
k
/2
N sin
k
/2
X[3]
sin N
3
/2
N sin
3
/2
X[k] = X(e
j2k/N
)

0
freq
2012-09-24 Dan Ellis 45
DFT from overlength DTFT

If x[n] has more than N points, can still


form

IDFT of X[k] will give N point

How does relate to ?



X[k] = X(e
j
)
=
2k
N
x[n]
x[n] x[n]
2012-09-24 Dan Ellis 46
DFT from overlength DTFT
=1 for n-l = rN, rI
= 0 otherwise
all values shifted by
exact multiples of N pts
to lie in 0 n < N
DTFT
x[n]
sample
X(e
j

)
IDFT
X[k]
0 n < N -A n < B

x [n]
0 n < N
x[n] =
1
N
N1

k=0

=
x[]W
k
N

W
nk
N
=

=
x[]

1
N
N1

k=0
W
k(n)
N

x[n] =

r=
x[n rN]
2012-09-24 Dan Ellis 47
DFT from DTFT example

If x[n] = { 8, 5, 4, 3, 2, 2, 1, 1} (8 point)

We form X[k] for k = 0, 1, 2, 3


by sampling X(e
j!
) at ! = 0, /2, , 3/2

IDFT of X[k] gives 4 pt

Overlap only for r = -1:


(N = 4)
x[n] =

8 5 4 3
+ + + +
2 2 2 1

= {10 7 5 4}
x[n] =

r=
x[n rN]
2012-09-24 Dan Ellis 48
DFT from DTFT example

x[n]

x[n+N]
(r = -1)

is the time aliased or folded down


version of x[n].
n
-1 1 2 3 4 5 6 7 8
n
-1 1 2 3 4 5 -2 -3 -4 -5
n
1 2 3

x [n]

x [n]
2012-09-24 Dan Ellis 49
Properties: Circular time shift

DFT properties mirror DTFT, with twists:

Time shift must stay within N-pt window

Modulo-N indexing keeps index between


0 and N-1:
0 n
0
< N
g[n n
0

N
] W
kn
0
N
G[k]
g[n n
0

N
] =

g[n n
0
] n n
0
g[N +n n
0
] n < n
0
2012-09-24 Dan Ellis 50
Circular time shift

Points shifted out to the right dont


disappear they come in from the left

Like a barrel shifter:


5-pt sequence
delay by 2
g[n]
1 2 3 4
n
g[<n-2>
5
]
1 2 3 4
n
origin pointer
2012-09-24 Dan Ellis 51
Circular time reversal

Time reversal is tricky in modulo-N


indexing - not reversing the sequence:

Zero point stays fixed; remainder flips


1 2 3 4
n
5 6 7 8 9 10 11 -7 -6 -5 -4 -3 -2 -1

x n
[ ]
5-pt sequence
made periodic
Time-reversed
periodic sequence
1 2 3 4
n
5 6 7 8 9 10 11 -7 -6 -5 -4 -3 -2 -1

x n
N
[ ]
2012-09-24 Dan Ellis 52
Duality

DFT and IDFT are very similar

both map an N-pt vector to an N-pt vector

Duality:
if
then

i.e. if you treat DFT sequence as a time


sequence, result is almost symmetric
Circular
time reversal

g n
[ ]
G k
[ ]

G n
[ ]
N g k
N
[ ]
2012-09-24 Dan Ellis 53
4. Convolution with the DFT

IDTFT of product of DTFTs of two N-pt


sequences is their 2N-1 pt convolution

IDFT of the product of two N-pt DFTs


can only give N points!

Equivalent of 2N-1 pt result time aliased:

i.e.

must be, because G[k]H[k] are exact


samples of G(e
j!
)H(e
j!
)

This is known as circular convolution



y
c
n
[ ]
= y
l
[n + rN]
r=

(0 n < N)
2012-09-24 Dan Ellis 54
Circular convolution

Can also do entire convolution with


modulo-N indexing

Hence, Circular Convolution:

Written as

g[n] h[n]
N

g m
[ ]
h n m
N
[ ]
m=0
N1

G[k]H[k]
2012-09-24 Dan Ellis 55
Circular convolution example

4 pt sequences: g[n]={1 2 0 1}
n
1 2 3
h[n]={2 2 1 0}
n
1 2 3
1
2
0
1
n
h[<n - 0>
4
]
1 2 3
n
h[<n - 1>
4
]
1 2 3
n
h[<n - 2>
4
]
1 2 3
n
h[<n - 3>
4
]
1 2 3
n
g[n] h[n]={4 7 5 4}
1 2 3
4
check: g[n] h[n]
={2 6 5 4 2 1 0}
*

g m
[ ]
h n m
N
[ ]
m=0
N1

2012-09-24 Dan Ellis 56


DFT properties summary

Circular convolution

Modulation

Duality

Parseval

G n
[ ]
N g k
N
[ ]

g m
[ ]
h n m
N
[ ]
m=0
N1

G[k]H[k]

g n
[ ]
h n
[ ]

1
N
G[m]H[ k m
N
]
m=0
N1


x n
[ ]
2
n=0
N1

=
1
N
X k
[ ]
2
k=0
N1

2012-09-24 Dan Ellis 57


Linear convolution w/ the DFT

DFT fast circular convolution

.. but we need linear convolution

Circular conv. is time-aliased linear


conv.; can aliasing be avoided?

e.g. convolving L-pt g[n] with M-pt h[n]:


y[n] = g[n] h[n] has L+M-1 nonzero pts

Set DFT size N L+M-1 no aliasing


*
2012-09-24 Dan Ellis 58
Linear convolution w/ the DFT

Procedure (N = L + M - 1):

pad L-pt g[n] with (at least)


M-1 zeros
N-pt DFT G[k], k = 0..N-1

pad M-pt h[n] with (at least)


L-1 zeros
N-pt DFT H[k], k = 0..N-1

Y[k] = G[k]H[k], k = 0..N-1

IDFT{Y[k]}
n
g[n]
L
n
h[n]
M
n
y
c
[n]
N

= y
L
n + rN
[ ]
r=

= y
L
n
[ ]
(0 n < N)
2012-09-24 Dan Ellis 59
Overlap-Add convolution

Very long g[n] break up into


segments, convolve piecewise, overlap
bound size of DFT, processing delay

Make

Called Overlap-Add (OLA) convolution...


g
i
[n] =

g[n] i N n < (i + 1) N
0 otherwise
g[n] =

i
g
i
[n]
h[n] g[n] =

i
h[n] g
i
[n]
2012-09-24 Dan Ellis 60
Overlap-Add convolution
n
g[n]
n
g
0
[n]
n
g
1
[n]
n
g
2
[n]
N 2N 3N
n
g
0
[n] h[n]
n
n
h[n]
n
g
1
[n] h[n]
g
2
[n] h[n]
n
N 2N 3N
h[n] g[n]
valid OLA sum
*
L
L
*
*
*
2012-10-03 Dan Ellis 1
ELEN E4810: Digital Signal Processing
Topic 4: The Z Transform
1. The Z Transform
2. Inverse Z Transform
2012-10-03 Dan Ellis 2
1. The Z Transform

Powerful tool for analyzing & designing


DT systems

Generalization of the DTFT:

z is complex...

z = e
j!
DTFT

z = re
j!



G(z) = Z g n
[ ] { }
= g[n]z
n
n=

Z Transform

g n
[ ]
r
n
e
jn
n

DTFT of
r
-n
g[n]
2012-10-03 Dan Ellis 3
Region of Convergence (ROC)

Critical question:
Does summation
converge (to a finite value)?

In general, depends on the value of z

Region of Convergence:
Portion of complex z-plane
for which a particular G(z)
will converge

G(z) = x[n]z
n
n=

z-plane
Re{z}
Im{z}
ROC
|z| >

2012-10-03 Dan Ellis 4


ROC Example

e.g. x[n] =
n
[n]

converges only for |z


-1
| < 1
i.e. ROC is |z| > ||

|| < 1 (e.g. 0.8) - finite energy sequence

|| > 1 (e.g. 1.2) - divergent sequence,


infinite energy, DTFT does not exist
but still has ZT when |z| > 1.2 (in ROC)

X(z) =
n
z
n
n=0

=
1
1 z
1
(see previous slide)
n
-1 1 2 3 4 -2
2012-10-03 Dan Ellis 5
About ROCs

ROCs always defined in terms of |z|


circular regions on z-plane
(inside circles/outside circles/rings)

If ROC includes
unit circle (|z| = 1),
g[n] has a DTFT
(finite energy
sequence)
z-plane
Re{z}
Im{z}
Unit circle
lies in ROC
DTFT OK
1
2012-10-03 Dan Ellis 6
Another ROC example

Anticausal (left-sided) sequence:

Same ZT as
n
[n], different sequence?

x n
[ ]
=
n
n 1
[ ]
n
-1
1 2 3 4
-2 -3 -4 -5


X z ( ) =
n
n 1 [ ]
( )
z
n
n

=
n
z
n
n=
1

=
m
z
m
m=1

=
1
z
1
1
1
z
=
1
1 z
1
ROC:
|| > |z|
2012-10-03 Dan Ellis 7
ROC is necessary!

To completely define a ZT, you must


specify the ROC:
x[n] =
n
[n]
ROC |z| > ||
x[n] = -
n
[-n-1]
ROC |z| < ||

A single G(z) can describe several


sequences with different ROCs

X(z) =
1
1 z
1

X(z) =
1
1 z
1
n
-1
1 2 3 4
-2 -3 -4
z-plane
Re
Im
||
n
-1 1 2 3 4 -2 -3 -4
DTFTs?
2012-10-03 Dan Ellis 8
Rational Z-transforms

G(z) can be any function;


rational polynomials are important
class:

By convention, expressed in terms of z


-1

matches ZT definition

(Reminiscent of LCCDE expression...)




G z
( )
=
P z
( )
D z
( )
=
p
0
+ p
1
z
1
++ p
M1
z
(M1)
+ p
M
z
M
d
0
+ d
1
z
1
++ d
N1
z
(N1)
+ d
N
z
N
2012-10-03 Dan Ellis 9
Factored rational ZTs

Numerator, denominator can be


factored:

} are roots of numerator


G(z) = 0 {

} are the zeros of G(z)

} are roots of denominator


G(z) = {

} are the poles of G(z)




G z
( )
=
p
0

=1
M
1

z
1
( )
d
0

=1
N
1

z
1
( )
=
z
M
p
0

=1
M
z

( )
z
N
d
0

=1
N
z

( )
2012-10-03 Dan Ellis 10
Pole-zero diagram

Can plot poles and zeros on


complex z-plane:
z-plane
Re{z}
Im{z}
1
o
o
o
o

poles


(cpx conj for real g[n])
zeros

2012-10-03 Dan Ellis 11


Z-plane surface

G(z): cpx function of a cpx variable

Can calculate value over entire z-plane

Slice between surface and unit cylinder


(|z| = 1 z = e
j
!
) is G(e
j
!
), the DTFT
M
ROC
not
shown!!
2012-10-03 Dan Ellis 12
Z-plane and DTFT

Unwrapping the cylindrical slice gives


the DTFT:
|G(e
j!
)|
z = e
j!
0
! / rad/samp
2012-10-03 Dan Ellis 13
ZT is Linear

Thus, if
then


G(z) = Z g n
[ ] { }
= g[n]z
n
n

Z Transform

y[n] =
1

1
n
[n] +
2

2
n
[n]
Y(z) =

1
1
1
z
1
+

2
1
2
z
1
y[n] = g[n] + h[n]
Y(z) = (g[n]+h[n])z
-n
= g[n]z
-n
+ h[n]z
-n
= G(z)+H(z)
ROC:
|z|>|
1
|,|
2
|
Linear
2012-10-03 Dan Ellis 14
ZT of LCCDEs

LCCDEs have solutions of form:

Hence ZT

Each term
i
n
in g[n] corresponds to a
pole
i
of G(z) ... and vice versa

LCCDE solns are right-sided


ROCs are |z| > |
i
|

y
c
[n] =
i

i
n
n
[ ]
+ ...


Y
c
z ( ) =

i
1
i
z
1
+
(same
s)
outside
circles
2012-10-03 Dan Ellis 15
ROCs and sidedness

Two sequences have:

Each ZT pole region in ROC outside


or inside || for R/L sided term in g[n]

Overall ROC is intersection of each terms



G(z) =
1
1 z
1
z-plane
Re
Im
||
ROC |z| > || g[n] =
n
[n]
n
ROC |z| < || g[n] = -
n
[-n-1] n
LEFT-SIDED
RIGHT-SIDED
( || < 1 )
2012-10-03 Dan Ellis 16

G(z) =
1
1
1
z
1
+
1
1
2
z
1
ROC intersections

Consider
with |
1
| < 1 , |
2
| > 1 ...

Two possible sequences for


1
term...

Similarly for
2
...
4 possible g[n] seqs and ROCs ...
no ROC specified

1
n
[n]
n
-
1
n
[-n-1]
n
or
n
-
2
n
[-n-1]
n

2
n
[n]
or
2012-10-03 Dan Ellis 17
ROC intersections: Case 1
ROC: |z| > |
1
| and |z| > |
2
|
Im
Re
|
1
| |
2
|
Im

G(z) =
1
1
1
z
1
+
1
1
2
z
1
n
g[n] =
1
n
[n] +
2
n
[n]
both right-sided:
2012-10-03 Dan Ellis 18
ROC intersections: Case 2

Im
Re
|
1
| |
2
|
Im

G(z) =
1
1
1
z
1
+
1
1
2
z
1
n
g[n] = -
1
n
[-n-1] -
2
n
[-n-1]
both left-sided:
ROC: |z| < |
1
| and |z| < |
2
|
2012-10-03 Dan Ellis 19
ROC intersections: Case 3
ROC: |z| > |
1
| and |z| < |
2
|
Im
Re
|
1
| |
2
|
Im

G(z) =
1
1
1
z
1
+
1
1
2
z
1
n
g[n] =
1
n
[n] -
2
n
[-n-1]
two-sided:
2012-10-03 Dan Ellis 20
ROC intersections: Case 4

ROC: |z| < |
1
| and |z| > |
2
| ?
Re
|
1
| |
2
|
Im

G(z) =
1
1
1
z
1
+
1
1
2
z
1
n
g[n] = -
1
n
[-n-1] +
2
n
[n]
two-sided:
no ROC
...
2012-10-03 Dan Ellis 21
ROC intersections

Note: Two-sided exponential

No overlap in ROCs
ZT does not exist
(does not converge for any z)

g n
[ ]
=
n
=
n
n
[ ]
+
n
n 1
[ ]

< n <
ROC
|z| > ||
ROC
|z| < ||
Re
||
Im
n
2012-10-03 Dan Ellis 22
Some common Z transforms
g[n] G(z) ROC
[n] 1 z
[n] |z| > 1

n
[n] |z| > ||
r
n
cos(!
0
n)[n] |z| > r
r
n
sin(!
0
n)[n] |z| > r

1
1z
1

1
1z
1

1r cos
0
( )
z
1
12r cos
0
( )
z
1
+r
2
z
2

r sin
0
( )
z
1
12r cos
0
( )
z
1
+r
2
z
2
sum of
re
j!
0
n
+ re
-j!
0
n
poles at z = re
j!
0

conjugate pole
pair
2012-10-03 Dan Ellis 23
Z Transform properties
g[n] G(z) w/ROC Rg
Conjugation g
*
[n] G
*
(z
*
) Rg
Time reversal g[-n] G(1/z) 1/Rg
Time shift g[n-n
0
] z
-n
0G(z) Rg (0/?)
Exp. scaling
n
g[n] G(z/) ||Rg
Diff. wrt z ng[n] Rg (0/?)

z
dG(z)
dz
2012-10-03 Dan Ellis 24
Z Transform properties
g[n] G(z) ROC
Convolution g[n] h[n] G(z)H(z)
Modulation g[n]h[n]
Parseval:

at least
RgRh

1
2j
G v
( )
H
z
v
( )
v
1
dv
C


g n
[ ]
h
*
n
[ ]
n=

=
1
2j
G v
( )
H
*
1
v
( )
v
1
dv
C

at least
RgRh
2012-10-03 Dan Ellis 25
ZT Example

; can express as

Hence,

x n
[ ]
= r
n
cos
0
n
( )
n
[ ]

1
2
n
[ ]
re
j
0
( )
n
+ re
j
0
( )
n






= v n
[ ]
+ v
*
n
[ ]
v[n] =
1
/
2
[n]
n
; = re
j!
0
V(z) = 1/(2(1- re
j!
0

z
-1
))
ROC: |z| > r

X z
( )
= V z
( )
+V
*
z
*
( )

=
1
2
1
1re
j
0
z
1
+
1
1re
j
0
z
1 ( )

=
1r cos
0
( )
z
1
12r cos
0
( )
z
1
+r
2
z
2
2012-10-03 Dan Ellis 26
Another ZT example

y n
[ ]
= n +1
( )

n
n
[ ]
= x[n] + nx[n]
where x[n] =
n
[n]

X z
( )
=
1
1z
1

z
dX z
( )
dz
= z
d
dz
1
1oz
1
|
\

|
.
|
=
oz
1
(1oz
1
)
2

Y z
( )
=
1
1z
1
+
z
1
(1z
1
)
2
=
1
(1z
1
)
2
repeated
root - IZT
( |z| > || )
ROC |z| > ||
2012-10-03 Dan Ellis 27
2. Inverse Z Transform (IZT)

Forward z transform was defined as:

3 approaches to inverting G(z) to g[n]:

Generalization of inverse DTFT

Power series in z (long division)

Manipulate into recognizable


pieces (partial fractions)


G(z) = Z g n
[ ] { }
= g[n]z
n
n=

the useful
one
2012-10-03 Dan Ellis 28
IZT #1: Generalize IDTFT

If then

so

Any closed contour around origin will do

Cauchy: g[n] = [residues of G(z)z


n-1
]

z = re
j
IDTFT
z = re
j!
d! = dz/jz
Counterclockwise
closed contour at |z| = r
within ROC
Re
Im
g[n]r
n
=
1
2

re
j

e
jn
d
=
1
2j

C
G(z) z
n1
r
n
dz
G(z) = G(re
j
) =

g[n]r
n
e
jn
= DTFT

g[n]r
n

2012-10-03 Dan Ellis 29


IZT #2: Long division

Since
if we could express G(z) as a simple
power series G(z) = a + bz
-1
+ cz
-2
...
then can just read off g[n] = {a, b, c, ...}

Typically G(z) is right-sided (causal)


and a rational polynomial

Can expand as power series through


long division of polynomials

G(z) = g[n]z
n
n=


G z
( )
=
P(z)
D(z)
2012-10-03 Dan Ellis 30
IZT #2: Long division

Procedure:

Express numerator, denominator in


descending powers of z (for a causal fn)

Find constant to cancel highest term


first term in result

Subtract & repeat lower terms in result

Just like long division for base-10


numbers
2012-10-03 Dan Ellis 31
IZT #2: Long division

e.g.

H z
( )
=
1+ 2z
1
1+ 0.4z
1
0.12z
2

1+ 0.4z
1
0.12z
2

1+ 2z
1
)

1+1.6z
1
0.52z
2
+ 0.4z
3
...

1+ 0.4z
1
0.12z
2

1.6z
1
+ 0.12z
2

1.6z
1
+ 0.64z
2
0.192z
3

0.52z
2
+ 0.192z
3
...
Result
2012-10-03 Dan Ellis 32
IZT#3: Partial Fractions

Basic idea: Rearrange G(z) as sum of


terms recognized as simple ZTs

especially
or sin/cos forms

i.e. given products



rearrange to sums

1
1z
1

n
n
[ ]


P(z)
1z
1
( )
1 z
1
( )



A
1z
1
+
B
1 z
1
+
2012-10-03 Dan Ellis 33
Partial Fractions

Note that:

Can do the reverse i.e.


go from to

if order of P(z) is less than D(z)



A
1z
1
+
B
1 z
1
+
C
1z
1
=
A 1 z
1
( )
1z
1
( )
+ B 1z
1
( )
1z
1
( )
+C 1z
1
( )
1 z
1
( )
1z
1
( )
1 z
1
( )
1z
1
( )
order 3 polynomial
order 2 polynomial
u + vz
-1
+ wz
-2


P(z)

=1
N
(1

z
1
)

z
1
=1
N

else cancel
w/ long div.
2012-10-03 Dan Ellis 34
Partial Fractions

Procedure:

where
i.e. evaluate F(z) at the pole
but multiplied by the pole term
dominates = residue of pole


F(z) =
P(z)

=1
N
(1

z
1
)
=

z
1
=1
N

= 1

z
1
( )
F z
( )
z=



f n
[ ]
=

( )
n
n
[ ]
=1
N

order N-1
(cancels term in
denominator)
no repeated
poles!
2012-10-03 Dan Ellis 35
Partial Fractions Example

Given (again)
factor:

where:

H z
( )
=
1+ 2z
1
1+ 0.4z
1
0.12z
2

=
1+ 2z
1
1+ 0.6z
1
( )
1 0.2z
1
( )
=

1
1+ 0.6z
1
+

2
1 0.2z
1

1
= 1+ 0.6z
1
( )
H z
( )
z=0.6
=
1+ 2z
1
1 0.2z
1
z=0.6
= 1.75

2
=
1+ 2z
1
1+ 0.6z
1
z=0.2
= 2.75
2012-10-03 Dan Ellis 36
Partial Fractions Example

Hence

If we know ROC |z| > || i.e. h[n] causal:


= 1.75{ 1 -0.6 0.36 -0.216 ...}
+2.75{ 1 0.2 0.04 0.008 ...}
= {1 1.6 -0.52 0.4 ...}

H z
( )
=
1.75
1+ 0.6z
1
+
2.75
1 0.2z
1

h n
[ ]
= 1.75
( )
0.6
( )
n
n
[ ]
+ 2.75
( )
0.2
( )
n
n
[ ]
same as
long division!
2012-10-10 Dan Ellis 1
ELEN E4810: Digital Signal Processing
Topic 5:
Transform-Domain Systems
1. Frequency Response (FR)
2. Transfer Function (TF)
3. Phase Delay and Group Delay
2012-10-10 Dan Ellis 2
1. Frequency Response (FR)

Fourier analysis expresses any


signal as the sum of sinusoids
e.g. IDTFT:

Sinusoids are the eigenfunctions of LSI


systems (only scaled, not changed)

Knowing the scaling for every sinusoid


fully describes system behavior
frequency response
describes how a
system affects each
pure frequency
x[n] =
1
2

X(e
j
)e
jn
d
2012-10-10 Dan Ellis 3
Sinusoids as Eigenfunctions

IR h[n] completely describes LSI system:

Complex sinusoid input i.e.

Output is sinusoid scaled by FT at


y[n] = x[n] h[n] =

m
h[m]x[n m] x[n]
h[n]
x[n] = e
j
0
n
y[n] =

m
h[m]e
j
0
(nm)
=

m
h[m]e
j
0
m
e
j
0
n
y[n] = H(e
j
0
) x[n] = |H(e
j
0
)| e
j(
0
n+(
0
))
H(e
j
)
= |H(e
j
)|e
j()

0
2012-10-10 Dan Ellis 4
System Response from H(e
j
!
)

If x[n] is a complex sinusoid at !


0
then the output of a system with IR h[n]
is the same sinusoid scaled by |H(e
j!0
)|
and phase-shifted by arg{H(e
j!0
)} = (!
0
)
where H(e
j!
) = DTFT{h[n]}
(Any signal can be expressed as sines...)

|H(e
j!
)| magnitude response gain

arg{H(e
j!
)} phase resp. phase shift
2012-10-10 Dan Ellis 5

In practice signals are real e.g.

Real
Real Sinusoids

-
0

0
|X(e
j

)|
A/2
x[n] =Acos(
0
n +)
=
A
2

e
j(
0
n+)
+e
j(
0
n+)

=
A
2
e
j
e
j
0
n
+
A
2
e
j
e
j
0
n
y[n] =
A
2
e
j
H(e
j
0
)e
j
0
n
+
A
2
e
j
H(e
j
0
)e
j
0
n
h[n] H(e
j
) = H

(e
j
) = |H(e
j
)|e
j()
y[n] = A|H(e
j
0
)| cos (
0
n + +(
0
))
2012-10-10 Dan Ellis 6
Real Sinusoids

A real sinusoid of frequency !


0
passed through an LSI system
with a real impulse response h[n]
has its gain modified by |H(e
j!
0)|
and its phase shifted by (!
0
).
Acos(
0
n +) A|H(e
j
0
)| cos (
0
n + +(
0
))
h[n]
2012-10-10 Dan Ellis 7
Transient / Steady State

Most signals start at a finite time, e.g.


What is the effect?
Steady state
- same as with pure sine input
Transient response
- consequence of gating
y[n] = h[n] x[n] =

n
m=
h[m]e
j
0
(nm)
=

m=
h[m]e
j
0
(nm)

m=n+1
h[m]e
j
0
(nm)
= H(e
j
0
)e
j
0
n
(

m=n+1
h[m]e
j
0
m
)e
j
0
n
x[n] = e
j
0
n
[n]
2012-10-10 Dan Ellis
-40 -30 -20 -10 0 10 20 30 40 time / n
Total output
Steady State
Transient
8
Transient / Steady State

FT of IR h[n]s tail from time n onwards

zero for FIR h[n] for n N

tends to zero with large n for any stable IR


transient
y[n] = H(e
j
0
)e
j
0
n
(

m=n+1
h[m]e
j
0
m
)e
j
0
n
x[n] = e
j
0
n
[n]
2012-10-10 Dan Ellis 9
FR example

MA filter
n
-1 1 2 3 4 5 6
1/M
y[n] =
1
M
M1

=0
x[n ]
=x[n] h[n]
H(e
j
) = DTFT{h[n]}
=

n=
h[n]e
jn
=
1
M
M1

n=0
e
jn
=
1
M
1 e
jM
1 e
j
=
1
M
e
j
(M1)
2
sin(M/2)
sin(/2)
2012-10-10 Dan Ellis
0
2
4
6
0
2
4
6
0 2 2
-
0

10
FR example

MA filter:

Response to ...
(jumps at sign changes:
r=M!/2)
(M = 5)
H(e
j
) =
1
M
e
j
(M1)
2
sin(M/2)
sin(/2)

H(e
j
)

1
M
sin(M/2)
sin(/2)

() =
(M 1)
2
+ r
x[n] = e
j
0
n
+e
j
1
n
2012-10-10 Dan Ellis
-20 0 20 40 60 80
-2
-1
0
1
2
n
x[n]
y[n]
11
FR example

MA filter

input

output
M
y[n] = H(e
j
0
)e
j
0
n
+ H(e
j
1
)e
j
1
n
x[n] = e
j
0
n
+e
j
1
n

0
= 0.1 H(e
j
0
) 0.8e
j
0

1
= 0.5 H(e
j
1
) ()0.2e
j
1
0
2
4
6
0
2
4
6
0 2 2
-
0

2012-10-10 Dan Ellis 12


2. Transfer Function (TF)
Linking LCCDE, ZT & Freq. Resp...

LCCDE:

Take ZT:

Hence:

or:
Transfer
function
H(z)
N

k=0
d
k
y[n k] =
N

k=0
p
k
x[n k]

k
d
k
z
k
Y (z) =

k
p
k
z
k
X(z)
Y (z) =
P
k
p
k
z
k
P
k
d
k
z
k
X(z)
Y (z) = H(z)X(z)
2012-10-10 Dan Ellis 13
Transfer Function (TF)

Alternatively,
ZT

Note: same H(z)

e.g. FIR filter, h[n] = {h


0
, h
1
,... h
M-1
}
p
k
=h
k
, d
0
=1, DE is
... if system
has DE form
... from IR
y[n] = h[n] x[n]
Y (z) = H(z)X(z)
1 y[n] =
M1

k=0
h
k
x[n k]
=
P
p
k
z
k
P
d
k
z
k

n
h[n]z
n
2012-10-10 Dan Ellis 14
Transfer Function (TF)

Hence, MA filter:
z-plane
Re{z}
Im{z}
1

z
M
=1 i.e.
M roots of 1
@ z=e
j2r/M
pole @ z=1
cancels
ROC?
|H(e
j!
)|
h[n] =

1
M
0 n M
0 otherwise
(ignore poles
at z=0)
y[n] =
1
M
M1

=0
x[n ]
H(z) =
1
M

M1
=0
z
n
=
1z
M
M(1z
1
)
=
z
M
1
Mz
M1
(z1)
2012-10-10 Dan Ellis 15
TF example

factorize:

0
= 0.6+j0.8

0
= 0.3

1
= 0.5+j0.7
...
y[n] = x[n 1] 1.2x[n 2] + x[n 3]
+ 1.3y[n 1] 1.04y[n 2] + 0.222y[n 3]
H(z) =
z
1
(1
0
z
1
)(1

0
z
1
)
(1
0
z
1
)(1
1
z
1
)(1

1
z
1
)
H(z) =
Y (z)
X(z)
=
z
1
1.2z
2
+z
3
1 1.3z
1
+ 1.04z
2
0.222z
3
2012-10-10 Dan Ellis 16
TF example

Poles
i
ROC

causal ROC is |z| > max|


i
|

includes u.circle stable


Re{z}
Im{z}
1

0
*
H(z) =
z
1
(1
0
z
1
)(1

0
z
1
)
(1
0
z
1
)(1
1
z
1
)(1

1
z
1
)

0
= 0.6+j0.8

0
= 0.3

1
= 0.5+j0.7
2012-10-10 Dan Ellis 17
TF FR

DTFT H(e
j

) = ZT H(z)|
z = e
j
i.e. Frequency Response is
Transfer Function evald on Unit Circle

H z
( )
=
p
0
1
k
z
1
( )
k=1
M

d
0
1
k
z
1
( )
k=1
N

=
p
0
z
M
z
k
( )
k=1
M

d
0
z
N
z
k
( )
k=1
N


H e
j
( )
=
p
0
d
0
e
j NM
( )
e
j

k
( )
k=1
M

e
j

k
( )
k=1
N

factor:
2012-10-10 Dan Ellis 18
TF FR

Magnitude
response

k
,
k
are
TF roots
on z-plane
Phase
response
() =arg

p
0
d
0

+ (N M)
+
M

k=1
arg

e
j

k=1
arg

e
j

H(e
j
) =
p
0
d
0
e
j(NM)

M
k=1

e
j

N
k=1
(e
j

k
)

H(e
j
)

p
0
d
0

M
k=1

e
j

N
k=1
|e
j

k
|
2012-10-10 Dan Ellis 19
FR: Geometric Interpretation

Have

On z-plane:

H e
j
( )
=
p
0
d
0
e
j NM
( )
e
j

k
( )
k=1
M

e
j

k
( )
k=1
N

Constant/
linear part
Product/ratio of terms
related to poles/zeros
Each (e
j!
- ) term corresponds
to a vector from pole/zero to
point e
j!
on the unit circle
Overall FR is product/ratio of
all these vectors
Im{z}

e
j!

i
e
j!

i
Re{z}
e
j!
2012-10-10 Dan Ellis 20
FR: Geometric Interpretation

Magnitude |H(e
j!
)| is product of
lengths of vectors from zeros
divided by product of lengths
of vectors from poles

Phase (!) is sum of angles of


vectors from zeros
minus sum of angles of
vectors from poles
Im{z}

e
j!

i
e
j!

i
Re{z}
e
j!

i
2012-10-10 Dan Ellis
-1 -0.5 0 0.5 1
-1
-0.5
0
0.5
1
Real Part
I
m
a
g
i
n
a
r
y

P
a
r
t
-0.8
-0.6 -0.4
-0.2
0.0
0.2
0.4 0.6
0.8
0
1
2
magnitude
phase
0 0.5 1.5 2
-
-/2
0
/2

21
FR: Geometric Interpretation

Magnitude and phase of a single zero:

Pole is reciprocal mag. & negated phase


2012-10-10 Dan Ellis
-1 -0.5 0 0.5 1
-1
-0.5
0
0.5
1
Real Part
I
m
a
g
i
n
a
r
y

P
a
r
t
-0.8
-0.6 -0.4
-0.2
0.0
0.2
0.4 0.6
0.8
0
1
2
magnitude
0 0.5 1.5 2
-
-/2
0
/2

phase

22
FR: Geometric Interpretation

Multiple
poles,
zeros:

H z
( )
=
z 0.8e
j0.3
( )
z 0.8e
j0.3
( )
z 0.9e
j0.3
( )
z 0.9e
j0.3
( )
V
2012-10-10 Dan Ellis 23
Geom. Interp. vs. 3D surface

3D magnitude surface for same system


Full surface Showing ROC
2012-10-10 Dan Ellis 24
Geom. Interp: Observations

Roots near unit circle


rapid changes in magnitude & phase

zeros cause mag. minima (= 0 on u.c.)

poles cause mag. peaks ( 10= at u.c.)

rapid change in relative angle phase

Pole and zero near each other


cancel out when seen from afar;
affect behavior when z = e
j!
gets close
M
PEZdemo
2012-10-10 Dan Ellis 25
Filtering

Idea: Separate information in frequency


with constructed H(e
j!
)

e.g.

Construct a filter:
|H(e
j!
1)| 1
|H(e
j!
2)| 0

Then

x n
[ ]
= Acos
1
n
( )
+ Bcos
2
n
( )
interested
in this part
dont care about
this part

y n
[ ]
= h n
[ ]
x n
[ ]
Acos
1
n +
1
( ) ( )
filtered
out
!
!
2
!
1
!
1
!
2
H(e
j!
)
X(e
j!
)
B
/
2
B
/
2
A
/
2
A
/
2
2012-10-10 Dan Ellis 26
Filtering example

Consider
filter family:
3 pt FIR filters
with h[n] = { }

Frequency Response:

n
-1 1 2 3 4 -2


H e
j
( )
= h n
[ ]
e
jn
n

= + e
j
+e
2 j
= e
j
+ e
j
+ e
j
( ) ( )
= e
j
+ 2cos
( )

H e
j
( )
= + 2cos
x[n]
+
z
-1
z
-1
y[n]

can set and


to obtain desired
|H(e
j!
)| ...
2012-10-10 Dan Ellis 27
Filtering example (contd)

h[n] = { }

Consider input as mix of sinusoids


at !
1
= 0.1 rad/samp
and !
2
= 0.4 rad/samp

Solve
= -12.46, = 6.76 ...
want to remove
i.e. make H(e
j!
2) = 0

H e
j
( )
= + 2cos

H(e
j
)

= | + 2cos |
=

1 =
1
= 0.1
0 =
2
= 0.4
2012-10-10 Dan Ellis 28
Filtering example (contd)

Filter
IR

Freq.
resp

input/
output
0 5 10 15 20 25 30 35 40 45
n
n
-15
-10
-5
0
5
10
15
0 0.2 0.4 0.6 0.8 1
/ rad
-20
-10
0
10
20
0 20 40 60 80 100 120 140 160 180
-2
-1
0
1
2
dB
2012-10-10 Dan Ellis 29
3. Phase- and group-delay

For sinusoidal input x[n] = cos!


0
n,
we saw

i.e.
or

where is phase delay



y n
[ ]
= H e
j
0
( )
cos
0
n +
0
( ) ( )
gain phase shift
or time shift

cos e
0
n +
u e
0
( )
e
0
|
\

|
.
|
|
\

|
.
|
cos e
0
n t
p
e
0
( )
( ) ( )

p

( )
=

( )

subtraction so
positive
p
means
delay (causal)
2012-10-10 Dan Ellis 30
Phase delay example

For our 3pt filter:

i.e. 1 sample delay (at all frequencies)


(as observed)
n
-1 1 2 3 4 -2


H e
j
( )
= e
j
+ 2cos
( )

u e
( )
= e
t
p
e
( )
=
e
e
|
\

|
.
|
= +1
2012-10-10 Dan Ellis 31
Group Delay

Consider a modulated carrier


e.g. x[n] = A[n]cos(!
c
n)
with A[n] = Acos(!
m
n) and !
m
<< !
c
0 50 100 150 200 250 300 350 400
1
0.5
0
0.5
1
2012-10-10 Dan Ellis 32
Group Delay

So: x[n] = Acos(!


m
n)cos(!
c
n)

Now:

Assume |H(e
j!
)| 1 around !
c
!
m
but (!
c
-!
m
) =
l
; (!
c
+!
m
) =
u
...
!
!
c
X(e
j!
)
!
c
+!
m
!
c
-!
m

=
A
2
cos
c

m
( )
n + cos
c
+
m
( )
n
[ ]

y n
[ ]
= h n
[ ]
- x n
[ ]
=
A
2
H e
j e
c
e
m
( )
( )
cos e
c
e
m
( )
n
+H e
j e
c
+e
m
( )
( )
cos e
c
+e
m
( )
n
|
\


|
.
|
|
2012-10-10 Dan Ellis 33
Group Delay

y n
[ ]
=
A
2
cos e
c
e
m
( )
n +u
l
[ ]
+cos e
c
+e
m
( )
n +u
u
[ ]
|
\

|
.
|
= Acos e
c
n +
u
u
+u
l
2
|
\

|
.
|
cos e
m
n +
u
u
u
l
2
|
\

|
.
|
phase shift
of carrier
phase shift
of envelope
2012-10-10 Dan Ellis 34
Group Delay

If (!
c
) is locally linear i.e.
" (!
c
+!) = (!
c
) + S!,

Then carrier phase shift


so carrier delay , phase delay

Envelope phase shift


delay group delay

S =
d
( )
d
=
c

l
+
u
2
=
c
( )


c
( )

c
=
p

u

l
2
=
m
S

g

c
( )
=
d
( )
d
=
c
!
c
(!)
S
!
!
c
!
m
!
c
+!
m

u
2012-10-10 Dan Ellis 35
Group Delay

If (!) is not linear around !


c
, A[n] suffers
phase distortion correction...
!
!
c

p
, phase
delay

g
, group
delay
(!)
n
0 50 100 150 200 250 300 350 400
-1
-0.5
0
0.5
1
Envelope
(group) delay
Carrier
(phase) delay
M
2012-10-17 Dan Ellis 1
ELEN E4810: Digital Signal Processing
Topic 6:
Filters - Introduction
1. Simple Filters
2. Ideal Filters
3. Linear Phase and FIR filter types
2012-10-17 Dan Ellis 2
1. Simple Filters

Filter = system for altering signal in


some useful way

LSI systems:

are characterized by H(z) (or h[n])

have different gains (& phase shifts)


at different frequencies

can be designed systematically


for specific filtering tasks
2012-10-17 Dan Ellis 3
FIR & IIR

FIR = finite impulse response


no feedback in block diagram
no poles (only zeros)

IIR = infinite impulse response


feedback in block diagram
poles (and often zeros)
2012-10-17 Dan Ellis 4
Simple FIR Lowpass

h
L
[n] = {
1
/
2

1
/
2
}
(2 pt moving avg.)

H
L
z
( )
=
1
2
1+ z
1
( )
=
z +1
2z
1
ZP
zero at
z = -1

H
L
e
j
( )
= e
j 2
cos 2
( )
n
-1 1 2 3 4 -2
1
/
2
h
L
[n]
!

|H
L
(e
j!
)|
1
/
2
sample delay
e
j!/2
+e
-j!/2
2012-10-17 Dan Ellis 5

Filters are often


characterized by their
cutoff frequency !
c
:

Cutoff frequency is most often defined


as the half-power point, i.e.

If
then

H e
j
c
( )
2
=
1
2
max H e
j
( )
2
{ }
H =
1
2
H
max
Simple FIR Lowpass
!
!
c
|H(e
j!
)|

H e
j
( )
= cos 2
( )

c
= 2cos
1
1
2
=

2

1
1/2
0.707
/2
2012-10-17 Dan Ellis 6
deciBels

Filter magnitude responses are


often described in deciBels (dB)

dB is simply a scaled log value:

Half-power also known as 3dB point:



dB = 20log
10
level
( )
=10log
10
power
( )

H
cutoff
=
1
2
H
max

dB H
cutoff
{ }
= dB H
max
{ }
+ 20log
10
1
2
( )
= dB H
max
{ }
3.01
power =
level
2
2012-10-17 Dan Ellis 7

We usually plot magnitudes in dB:

A gain of 0 corresponds to - dB
deciBels
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1/2
1
/
|
H
(
e
j

)
|
0 0.2 0.4 0.6 0.8 1
-20
-15
-10
-5
-3
0
/
|
H
(
e
j

)
|

/

d
B
2012-10-17 Dan Ellis 8
Simple FIR Highpass

h
H
[n] = {
1
/
2
-
1
/
2
}

3dB point !
c
=

/
2
(again)

H
H
z
( )
=
1
2
1 z
1
( )
=
z 1
2z
1
ZP
zero at
z = 1

H
H
e
j
( )
= je
j 2
sin 2
( )
!

|H
H
(e
j!
)|
n
-1 2 3 4 -2
1
/
2
h
H
[n]
-
1
/
2
!
c
/2
2012-10-17 Dan Ellis 9
FIR Lowpass and Highpass

Note:
h
L
[n] = {
1
/
2

1
/
2
} h
H
[n] = {
1
/
2
-
1
/
2
}

i.e.

i.e. 180 rotation of


the z-plane,
shift of frequency
response

h
H
n
[ ]
= (1)
n
h
L
n
[ ]
H
H
z
( )
= H
L
z
( )
1
z
1
j
-j
1
-z
1
-j
j
!
|H
L
(e
j!
)|
2
!
|H
H
(e
j!
)|

2012-10-17 Dan Ellis


-1 0 1
-1
0
1
z-plane
freq /
|
H
(
e
j

)
|
|
H
(
e
j

)
|
freq /
0 0.5 1
0
0.5
1
10
-2
10
-1
10
0
10
-2
10
-1
10
0

1 1
10
Simple IIR Lowpass
IIR feedback, zeros and poles,
conditional stability, h[n] less useful
scale to make
gain = 1 at ! = 0
K = (1 - )/2
pole-zero
diagram
frequency
response
FR on
log-log axes
0dB
-20dB
-40dB
x[n]
z
-1
+
z
-1
+
y[n]
K
H
LP
(z) = K
1 +z
1
1 z
1
2012-10-17 Dan Ellis 11
Simple IIR Lowpass

Cutoff freq. !
c
from

H
LP
z
( )
= K
1+ z
1
1z
1

H
LP
e
j
c
( )
2
=
max
2
max = 1
using K=(1-)/2

1
( )
2
4
1+ e
j
c
( )
1+ e
j
c
( )
1e
j
c
( )
1e
j
c
( )
=
1
2

cos
c
=
2
1+
2
=
1 sin
c
cos
c
Design Equation
2012-10-17 Dan Ellis

-1 0 1
-1
-0.5
0
0.5
1
z-plane
Freq /
Freq /
|
H
(
e
j

)
|

/

d
B
|
H
(
e
j

)
|
10
-2
10
-1
10
0
-40
-30
-20
-10
0
0 0.5 1
0
0.5
1
1
1
12
Simple IIR Highpass
Pass ! = H
HP
(-1) = 1
K = (1+)/2
Design Equation:
(again)

=
1 sin
c
cos
c
H
HP
(z) = K
1z
1
1 z
1
2012-10-17 Dan Ellis 13
Highpass and Lowpass

Consider lowpass filter:

Then:

However,
(unless H(e
j!
) is pure real - not for IIR)
Highpass

c/w (-1)
n
h[n]
just another z poly
H
LP
(e
j
) =

1 0
0 large
1 H
LP
(e
j
) =

0 0
1 large
|1 H
LP
(z)| = 1 |H
LP
(z)|
2012-10-17 Dan Ellis
-1 0 1
1
0.5
0
0.5
1
z-plane
/
/
|
H
(
e
j
w
)
|
|
H
(
e
j
w
)
|

/

d
B
0 0.5 1
0
0.5
1
10
-2
10
-1
10
0
-40
-30
-20
-10
0
r
B

14
Simple IIR Bandpass

H
BP
z
( )
=
1
2
1 z
2
1 1+
( )
z
1
+z
2
= K
1+ z
1
( )
1 z
1
( )
1 2r cos z
1
+ r
2
z
2
where
Center freq
3dB bandwidth

r = cos =
1+
( )
2

B = cos
1
2
1+
2






c
= cos
1

D
e
s
i
g
n
B 1/2
2012-10-17 Dan Ellis 15
Simple Filter Example

Design a second-order IIR bandpass


filter with !
c
= 0.4, 3dB b/w of 0.1

c
= 0.4 = cos
c
= 0.3090

B = 0.1
2
1+
2
= cos 0.1
( )
= 0.7265

H
BP
z
( )
=
1
2
1 z
2
1 1+
( )
z
1
+z
2
=
0.1367 1 z
2
( )
1 0.5335z
1
+ 0.7265z
2
M
sensitive..
2012-10-17 Dan Ellis
/
/
|
H
(
e
j
w
)
|
|
H
(
e
j
w
)
|

/

d
B
-1 0 1
-1
-0.5
0
0.5
1
0 0.5 1
0
0.5
1
10
-2
10
-1
10
0
-40
-30
-20
-10
0
z-plane
16
Simple IIR Bandstop

Design eqns:

H
BS
z
( )
=
1+
2
1 2z
1
+ z
2
1 1+
( )
z
1
+z
2
zeros at !
c
(per 1 - 2r cos z
-1
+ r
2
z
-2
)
same poles as H
BP

B = cos
1
2o
1+o
2
|
\

|
.
|
o =
1
cos B

1
cos
2
B
1

c
= cos
1
= cos
c
B
2012-10-17 Dan Ellis 17
Cascading Filters

Repeating a filter (cascade connection)


makes its characteristics more abrupt:

Repeated roots in z-plane:


H(e
j!
)
H(e
j!
) H(e
j!
) H(e
j!
)
!
|H(e
j!
)|
!
|H(e
j!
)|
3
1
ZP
h
h
3
2012-10-17 Dan Ellis 18
1
/
8
1
/
4
Cascading Filters

Cascade systems are higher order


e.g. longer (finite) impulse response:

In general, cascade filters will not be


optimal (...) for a given order
n
-1 2 3 4 -2
1
/
2
h[n]
-
1
/
2
n
-1 2 3 4 -2
-
1
/
2
n
-1 2 4 -2
-
3
/
8
h[n]h[n] h[n]h[n]h[n]
2012-10-17 Dan Ellis
10
-2
10
-1
10
0
-60
-50
-40
-30
-20
-10
0
2nd order
-6dB/oct
-12dB/oct
4th order
/
|
H
(
e
j

)
|
19
Cascading Filters

Cascading filters improves rolloff slope:

But: 3dB cutoff frequency will change


(gain at !
c
3N dB)
!
c
2012-10-17 Dan Ellis
Interlude: The Big Picture
20
IR
LCCDE
DTFT
ZT
y[n] = h[n] x[n]
Y (e
j
) = H(e
j
)X(e
j
)
Y (z) = G

M
j=1
(1
j
z
1
)

N
k=1
(1
k
z
1
)
X(z)
y[n] =
M

j=0
p
j
x[n j]
N

k=1
d
k
y[n k]
h
[
k
]
x[n k]
0
1
2
3
0
0 W
W
\He
jW
\
[He
jW
]
DTFT
IDTFT
Z
T
I
Z
T
X(e
j
) =

n
x[n]e
jn
x[n] =
1
2

X(e
j
)e
jn
d

j
p
j
x[n j]

j
p
j
z
j
X(z)
X
(
z
)
=

n
x
[n
]z

n
X(e
j
) = X(z)|
z=e
j
y
c
[n] + y
p
[n] =

n
i
+
n
0
n 0
x[n] y[n] +
z
p
0
p
1
p
2
-d
1
-d
2
-1
z
-1
z
-1
z
-1
Re{z}
Im{z}

0
1
1
0
0
*
*

[n
]

1
1

1 |z
|
>
|
|
2012-10-17 Dan Ellis 21
2. Ideal filters

Typical filter requirements:

gain = 1 for wanted parts (pass band)

gain = 0 for unwanted parts (stop band)

Ideal characteristics
would be like:

no phase distortion etc.

What is this filter?

can calculate IR h[n] as IDTFT of ideal


response...
!
|H(e
j!
)|
brickwall
LP filter
2012-10-17 Dan Ellis 22
Ideal Lowpass Filter

Given ideal H(e


j!
):
(assume (!) = 0)

!
!
c
!
c

h n
[ ]
= IDTFT H e
j
( ) { }
=
1
2
H e
j
( )
e
jn
d

=
1
2
e
jn
d


h n
[ ]
=
sin
c
n
n
Ideal lowpass filter
0
2012-10-17 Dan Ellis 23
Ideal Lowpass Filter

Problems!

doubly infinite (n = -..)

no rational polynomial very long FIR

excellent frequency-domain characteristics


poor time-domain characteristics
(blurring, ringing a general problem)

h n
[ ]
=
sin
c
n
n
-20 -15 -10 -5 0 5 10 15
-0.05
0
0.05
0.1
0.15
0.2
n
h[n]
2012-10-17 Dan Ellis 24

lower-order realization (less computation)

better time-domain properties (less ringing)

easier to design...
Practical filter specifications
|
H
(
e
j
!
)
|

/

d
B !
-40
-1
1
Pass band Stop band
T
r
a
n
s
i
t
i
o
n
allow PB ripples
allow SB ripples
allow transition band
2012-10-17 Dan Ellis 25

|H(e
j!
)| alone can hide phase distortion

differing delays for adjacent frequencies


can mangle the signal

Prefer filters with a flat phase response


e.g. (!) = 0 zero phase filter

A filter with constant delay


p
= D at all
freqs has (!) = D! linear phase

Linear phase can shift to zero phase




H e
j
( )
= e
jD

H ( )
3. Linear-phase Filters
pure-real (zero-phase)
portion
2012-10-17 Dan Ellis 26
Time reversal filtering

v[n] = x[n]h[n] V(e


j!
) = H(e
j!
)X(e
j!
)

u[n] = v[-n] U(e


j!
) = V(e
-j!
) = V
*
(e
j!
)

w[n] = u[n]h[n] W(e


j!
) = H(e
j!
)U(e
j!
)

y[n] = w[-n] Y(e


j!
) = W
*
(e
j!
)
= (H(e
j!
)(H(e
j!
)X(e
j!
))
*
)
*

Y(e
j!
) = X(e
j!
)|H(e
j!
)|
2

Achieves zero-phase result

Not causal! Need whole signal first


H(z) x[n]
Time
reversal
H(z)
Time
reversal
y[n]
v[n] u[n] w[n]
if v real
2012-10-17 Dan Ellis 27
Linear Phase FIR filters

(Anti)Symmetric FIR filters are almost


the only way to get zero/linear phase

4 types:
Odd length Even length
Symmetric
Anti-
symmetric
Type 1 Type 2
Type 3 Type 4
n n
n
n
0 N N/2
2012-10-17 Dan Ellis
0.4 0.2 0 0.2 0.4 0.6 0.8 1
-1
-0.5
0
0.5
1
/
n=1
n=2
28
Linear Phase FIR: Type 1

Length L odd order N = L - 1 even

Symmetric h[n] = h[N - n]


(h[N/2] unique)



H e
j
( )
= h n
[ ]
e
jn
n=0
N

= e
j
N
2
h
N
2
[ ]
+ 2 h
N
2
n
[ ]
cosn
n=1
N / 2

( )
linear phase
D = -(!)/! = N/2
pure-real H(!) from cosine basis
~
!
H(!)
~
2012-10-17 Dan Ellis
0 2 4 6 8 1 3 5 7 n
0
1
2
3
4
5
-10
0
10
20
- -0. 5 0 0.5 - -0. 5 0 0.5
-
-0.5
0
0.5

Impulse response Magnitude response Phase response


dB
29
Linear Phase FIR: Type 1

Where are the N zeros?


thus for a zero

h n
[ ]
= h N n
[ ]
H z
( )
= z
N
H
1
z
( )

H
( )
= 0 H
1

( )
= 0
Reciprocal zeros
(as well as cpx conj)
Reciprocal
pair
Conjugate
reciprocal
constellation
No reciprocal
on u.circle
= -D!
-2
2012-10-17 Dan Ellis
0.4 0.2 0 0.2 0.4 0.6 0.8 1
1
0.5
0
0.5
1
/
n -
1
/
2
=
1
/
2
n -
1
/
2
= 1
1
/
2
n -
1
/
2
= 2
1
/
2
30
Linear Phase FIR: Type 2

Length L even order N = L - 1 odd

Symmetric h[n] = h[N - n]


(no unique point)



H e
j
( )
= e
j
N
2
h
N+1
2
n
[ ]
cos n
1
2
( )
n=1
N+1
( )
/ 2

Non-integer delay
of N/2 samples
H(!) from double-length cosine basis
~
always
zero
at !=
2012-10-17 Dan Ellis
0 2 4 6 8 10
0
1
2
3
4
5
- -0.5 0 0.5
- -0.5 0 0.5
-20
0
20
-0.5
0
0.5

-
-2 -1 0 1 2
-1
-0.5
0
0.5
1
Impulse Response Magnitude Response
Phase Response
Pole-zero diagram
31
Linear Phase FIR: Type 2

Zeros:
at z = -1,

H z
( )
= z
N
H
1
z
( )

H 1
( )
= 1
( )
N
H 1
( )
H e
j
( )
= 0
odd
LPF-like
dB
2012-10-17 Dan Ellis
-0. 4 -0. 2 0 0.2 0.4 0.6 0.8 1
-1
-0. 5
0
0.5
1
/
n

= 2
n = 1
odd
functions
zero at
= 0,
32
Linear Phase FIR: Type 3

Length L odd order N = L - 1 even

Antisymmetric h[n] = h[N - n]


h[N/2]= -h[N/2] = 0



H e
j
( )
= h
N
2
n
[ ]
e
j
N
2
n
( )
e
j
N
2
+n
( )
( )
n=1
N / 2

= je
j
N
2
2 h
N
2
n
[ ]
n=1
N / 2

sinn
( )
(!) = /2 - !N/2
Antisymmetric
/2 phase shift in
addition to linear phase
2012-10-17 Dan Ellis
- -0.5 0 0.5
- -0.5 0 0.5 -2 -1 0 1 2
Impulse Response Magnitude Response
Phase Response
Pole-zero diagram
0 2 4 6 8 10
-2
0
2
-20
0
20
-0.5
0
0.5

-
33
Linear Phase FIR: Type 3

Zeros:

H z
( )
= z
N
H
1
z
( )

H 1
( )
= H 1
( )
= 0 ; H 1
( )
= H 1
( )
= 0
2012-10-17 Dan Ellis 34
Linear Phase FIR: Type 4

Length L even order N = L - 1 odd

Antisymmetric h[n] = -h[N - n]


(no center point)



H e
j
( )
= je
j
N
2
2 h
N+1
2
n
[ ]
n=1
N / 2

sin n
1
2
( )
fractional-sample
delay
/2 offset offset sine basis
0.4 0.2 0 0.2 0.4 0.6 0.8 1
1
0.5
0
0.5
1
/
odd
functions
n -
1
/
2
=
1
/
2
n -
1
/
2
= 1
1
/
2
2012-10-17 Dan Ellis 35
Linear Phase FIR: Type 4

Zeros:
(H(-1) OK because N is odd)

H 1
( )
= H 1
( )
= 0
- -0.5 0 0.5
Impulse Response Magnitude Response
Phase Response
Pole-zero diagram
-20
0
20
-0.5
0
0.5

-
0 2 4 6 8 10
-4
-2
0
2
4
-2 0 2
-1
0
1
- -0.5 0 0.5

dB

2012-10-17 Dan Ellis 36


4 Linear Phase FIR Types
Odd length Even length
A
n
t
i
s
y
m
m
e
t
r
i
c
S
y
m
m
e
t
r
i
c
1 2
3 4
h[n]
n
!
H(!)
~
ZP
h[n]
n
!
H(!)
~
ZP
D D

h[n]
n
!
H(!)
~
ZP
D

h[n]
n
!
H(!)
~
ZP
D

2012-10-31 Dan Ellis 1


ELEN E4810: Digital Signal Processing
Topic 7:
Filter types and structures
1. More filter types
2. Minimum and maximum phase
3. Filter implementation structures
2012-10-31 Dan Ellis 2
1. More Filter Types

We have seen the basics of filters


and a range of simple examples

Now look at a couple of other classes:

Comb filters - multiple pass/stop bands

Allpass filters - only modify signal phase


2012-10-31 Dan Ellis 3
Comb Filters

Replace all system delays z


-1
with
longer delays z
-L

System that behaves the same at a
longer timescale
x[n] y[n] +
z
-L
z
-L
z
-L
z
-L
2012-10-31 Dan Ellis 4
Comb Filters

Parent filter impulse response h[n]


becomes comb filter output as:
g[n] = {h[0] 0 0 0 0 h[1] 0 0 0 0 h[2]..}

Thus,

G z
( )
= g n
[ ]
z
n
n

= h n
[ ]
z
nL
n

= H z
L
( )
L-1 zeros
2012-10-31 Dan Ellis 5
Comb Filters

Hence frequency response:

Low-pass response

pass ! = 0, 2 /L, 4 /L...

cut ! = /L, 3 /L, 5 /L...



G e
j
( )
= H e
jL
( )
parent frequency response
compressed
& repeated L times
H(e
j!
) G(e
j!
)
L copies
of H(e
j!
)
useful to enhance
a harmonic series
2012-10-31 Dan Ellis 6
Allpass Filters

Allpass filter has |A(e


j!
)|
2
= K for all !
i.e. spectral energy is not changed

Phase response is not zero (else trivial)

phase correction special effects

e.g.
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
400
300
200
100
0
Normalized Frequency ( rad/sample)
P
h
a
s
e

(
d
e
g
r
e
e
s
)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
20
15
10
5
0
5
Normalized Frequency ( rad/sample)
M
a
g
n
i
t
u
d
e

(
d
B
)
2012-10-31 Dan Ellis 7
Allpass Filters

Allpass has special form of system fn:

A
M
(z) has poles where D
M
() = 0
A
M
(z) has zeros = 1/ =
-1


A
M
z
( )
=
d
M
+ d
M1
z
1
+ ... + d
1
z
M1
( )
+ z
M
1+ d
1
z
1
+ ... + d
M1
z
M1
( )
+ d
M
z
M
= z
M
D
M
z
1
( )
D
M
z
( )
mirror-image
polynomials
=
2012-10-31 Dan Ellis 8
Allpass Filters

Any (stable) D
M
can be used:

Phase is always
decreasing:
-M at ! =

A
M
z
( )
= z
M
D
M
z
1
( )
D
M
z
( )
poles
from 1/D
M
(z)
reciprocal
zeros
from D
M
(z
-1
)
M
peak
group
delay
0 0.2 0.4 0.6 0.8 1
3
2

0
/
a
r
g
{
H
(
z
)
}

3 2 1 0 1 2
1
0
1
Re{z}
I
m
{
z
}
2012-10-31 Dan Ellis 9
Why do mirror-img polys give const gain?

Conj-sym system fn can be factored as:

z = e
j!
z
-1
= e
-j!

also on u.circle...
Allpass Filters

A
M
z
( )
=
K z
i
*
1
( )
i

z
i
( )
i

=
K
i
*
1
z
i
*
z
1
( )
i

z
i
( )
i

e
j!

e
-j!

ZP

i
o
+ complex
conjugate p/z

i
*
1

i
*
2012-10-31 Dan Ellis 10
2. Minimum/Maximum Phase

In AP filters, reciprocal roots have..

same effect on magnitude (modulo const.)

different effect on phase

In normal filters, can try


substituting reciprocal roots

reciprocal of stable pole will be unstable

reciprocals of zeros?
Variants of filters with same magnitude
response, different phase
2012-10-31 Dan Ellis 11
Minimum/Maximum Phase

Hence:
reciprocal
zero..
.. added
phase
lag
.. same
mag..
0
1
2
3
0 0.2 0.4 0.6 0.8 /
-
-0.5
0
0.5
1
2
3
0 0.2 0.4 0.6 0.8 /
-
-0.5
0
0.5
a a
b 1/b
0
H
1
(z) =
z - b
z - a
H
2
(z) =
b(z - 1/b)
z - a
|H(e
j
)|
(H(e
j
))
2012-10-31 Dan Ellis 12
Minimum/Maximum Phase

For a given magnitude response

All zeros inside u.circle minimum phase

All zeros outside u.c. maximum phase


(greatest phase dispersion for that order)

Otherwise, mixed phase

i.e. for a given magnitude response


several filters & phase fns are possible;
minimum phase is canonical, best
2012-10-31 Dan Ellis 13
Minimum/Maximum Phase

Note:
Min. phase + Allpass = Max. phase
o
o
o
o
o
o

z
( )
z
*
( )
z

z
1

( )
z
1

*
( )
z
( )
z
*
( )

z
1

( )
z
1

*
( )
z
x =
pole-
zero
cancln
2012-10-31 Dan Ellis 14
Inverse Systems

h
i
[n] is called the inverse of h
f
[n] iff

Z-transform:

i.e. H
i
(z) recovers x[n] from o/p of H
f
(z)

h
i
n
[ ]
h
f
n
[ ]
= n
[ ]

H
f
e
j
( )
H
i
e
j
( )
=1
H
f
(z) H
i
(z)
x[n] y[n] w[n]

W z
( )
= H
i
z
( )
Y z
( )
= H
i
z
( )
H
f
z
( )
X z
( )
= X z
( )

w n
[ ]
= x n
[ ]
2012-10-31 Dan Ellis 15

What is H
i
(z)?

H
i
(z) is reciprocal polynomial of H
f
(z)

Just swap
poles and
zeros:
Inverse Systems

H
i
z
( )
H
f
z
( )
=1
H
i
z
( )
=1/ H
f
z
( )

H
f
z
( )
=
P z
( )
D z
( )
H
i
z
( )
=
D z
( )
P z
( )
poles of fwd
zeros of bwd
zeros of fwd
poles of bwd
o
o
o
H
f
(z) H
i
(z)
2012-10-31 Dan Ellis 16
Inverse Systems
When does H
i
(z) exist?

Causal+stable all H
i
(z) poles inside u.c.
all zeros of H
f
(z) must be inside u.c.
H
f
(z) must be minimum phase

H
f
(z) zeros outside u.c. unstable H
i
(z)

H
f
(z) zeros on u.c. unstable H
i
(z)
only invert if min.phase, H
f
(e
j!
) 0

H
i
e
j
( )
=1/ H
f
e
j
( )
=1/ 0
=
!
lose...
2012-10-31 Dan Ellis 17
System Identification

Inverse filtering = given y and H, find x

System ID = given y (and ~x), find H

Just run convolution backwards?

H(z)
x[n] y[n]

y n
[ ]
= h k
[ ]
x n k
[ ]
k=0


y 0
[ ]
= h 0
[ ]
x 0
[ ]
h 0
[ ]
y 1
[ ]
= h 0
[ ]
x 1
[ ]
+ h 1
[ ]
x 0
[ ]
h 1
[ ]
...
but: errors
accumulate
deconvolution
2012-10-31 Dan Ellis 18
System Identification

Better approach uses correlations;


Cross-correlate input and output:

If r
xx
is simple, can recover h
?
[n]...

e.g. (pseudo-) white noise:


H
?
(z)
x[n] y[n] + noise


r
xy

[ ]
= y
[ ]
x
[ ]
= h
?

[ ]
x
[ ]
x
[ ]
= h
?

[ ]
r
xx

[ ]


r
xx

[ ]

[ ]
h
?
n
[ ]
r
xy

[ ]
2012-10-31 Dan Ellis 19
System Identification

Can also work in frequency domain:

x[n] is not observable S


xy
unavailable,
but S
xx
(e
j!
) may still be known, so:

Use e.g. min.phase to rebuild H(e


j!
)...

S
xy
z
( )
= H
?
z
( )
S
xx
z
( )
make a const.

S
yy
e
j
( )
=Y e
j
( )
Y
*
e
j
( )
= H e
j
( )
X e
j
( )
H
*
e
j
( )
X
*
e
j
( )
= H e
j
( )
2
S
xx
e
j
( )
2012-10-31 Dan Ellis 20
3. Filter Structures

Many different implementations,


representations of same filter

Different costs, speeds, layouts, noise


performance, ...
2012-10-31 Dan Ellis 21
Block Diagrams

Useful way to illustrate implementations

Z-transform helps analysis:

Approach

Output of summers as dummy variables

Everything else is just multiplicative



Y z
( )
= G
1
z
( )
X z
( )
+G
2
z
( )
Y z
( ) [ ]
Y z
( )
1G
1
z
( )
G
2
z
( ) [ ]
= G
1
z
( )
X z
( )
H z
( )
=
Y z
( )
X z
( )
=
G
1
z
( )
1G
1
z
( )
G
2
z
( )
G
1
(z)
+
G
2
(z)
x[n] y[n]
2012-10-31 Dan Ellis 22
Block Diagrams

More complex example:


x[n]
+
y[n] + z
-1
-

+ z
-1
-
"
w
1
w
2
w
3
+

W
1
= X z
1
W
3
W
2
= W
1
z
1
W
2
W
3
= z
1
W
2
+W
2

Y = z
1
W
3
+ W
1

W
2
=
W
1
1+z
1

W
3
=
z
1
+
( )
W
1
1+z
1

Y
X
=
+ z
1
+
( )
+ z
2

( )
1+ z
1
+
( )
+ z
2

( )
stackable
2nd order section
2012-10-31 Dan Ellis 23
Delay-Free Loops

Cant have them!

At time n = 0, setup inputs x and v ;


need u for y, also y for u cant calculate

Algebra:
x
+
y

+

u
v
x
y
+
u
v
+

1
1 BA

B
1 BA

B
1 BA

BA
1 BA
can simplify...
y = B(v +Au) u = x +y
y = B(v +A(x +y))
y(1 BA) = Bv +BAx
y =
Bv +BAx
1 BA
2012-10-31 Dan Ellis 24
Equivalent Structures

Modifications to block diagrams that do


not change the filter

e.g. Commutation H = AB = BA

Factoring AB+CB = (A+C)B


A B B A

A(z)B(z)
+
C(z)B(z)
x
1
x
2
y
A(z)
+
x
1
x
2
y
C(z)
B(z)
fewer blocks less computation
2012-10-31 Dan Ellis 25
Equivalent Structures

Transpose

reverse paths

addersnodes

inputoutput
+
z
-1
z
-1
y x b
1
b
2
b
3
+
z
-1
z
-1
x y b
1
b
2
b
3
+


Y = b
1
X + b
2
z
1
X + b
3
z
2
X
= b
1
X + z
1
b
2
X + z
1
b
3
X
( )
2012-10-31 Dan Ellis 26
FIR Filter Structures

Direct form Tapped Delay Line

Transpose

Re-use delay line if several inputs x


i

for single output y ?
z
-1
+
z
-1
+
z
-1
+
z
-1
+
x
h
0
h
1
h
2
h
3
h
4

y n
[ ]
= h
0
x n
[ ]
+ h
1
x n 1
[ ]
+...
= h
k
x n k
[ ]
k=0
4

y
z
-1
+
z
-1
+
z
-1
+
z
-1
+
x
h
4
h
3
h
2
h
1
h
0
y
2012-10-31 Dan Ellis 27
FIR Filter Structures

Cascade

factored into e.g. 2nd order sections



H z
( )
= h
0
+ h
1
z
1
+ h
2
z
2
+ h
3
z
3
= h
0
1
0
z
1
( )
1
1
z
1
( )
1
1
*
z
1
( )
= h
0
1
0
z
1
( )
1 2 Re
1
{ }
z
1
+
1
2
z
2
( )
+
z
-1
z
-1
y
-2Re{
1
}
|
1
|
2
z
-1
x h
0

0
+
2012-10-31 Dan Ellis 28
FIR Filter Structures

Linear Phase:
Symmetric filters with h[n] = (-)h[N - n]

Also Transpose form:


gains first, feeding folded delay/sum line
n

y n
[ ]
= b
0
x n
[ ]
+ x n 4
[ ] ( )
+b
1
x n 1
[ ]
+ x n 3
[ ] ( )
+b
2
x n 2
[ ]
z
-1
+
z
-1
x
b
0
b
1
b
2
+
z
-1
z
-1
+ +
y
...
half as many
multiplies
2012-10-31 Dan Ellis 29
IIR Filter Structures

IIR: numerator + denominator



H z
( )
=
p
0
+ p
1
z
1
+ p
2
z
2
+...
1+ d
1
z
1
+ d
2
z
2
+...
= P z
( )

1
D z
( )
+
z
-1
z
-1
p
0
p
1
p
2
+
z
-1
z
-1
-d
1
-d
2
FIR
all-pole
IIR
2012-10-31 Dan Ellis 30
IIR Filter Structures

Hence, Direct form I

Commutation Direct form II (DF2)


+
z
-1
z
-1
p
0
p
1
p
2
z
-1
z
-1
-d
1
-d
2
+
p
0
p
1
p
2
+
z
-1
z
-1
-d
1
-d
2
same signal
delay lines merge
canonical
= min. memory usage
2012-10-31 Dan Ellis 31
IIR Filter Structures

Use Transpose on FIR/IIR/DF2

Direct Form II Transpose


-d
1
-d
2
+
z
-1
z
-1
x y
p
0
p
1
p
2
+
+
2012-10-31 Dan Ellis 32
Factored IIR Structures

Real-output filters have


conjugate-symm roots:

Can always group into 2nd order terms


with real coefficients:

H z
( )
=
1
1 ( + j)z
1
( )
1 ( j)z
1
( )

H z
( )
=
p
0
1
1
z
1
( )
1 2
2
z
1
+ (
2
2
+
2
2
)z
2
( )
...
1
1
z
1
( )
1 2
2
z
1
+ (
2
2
+
2
2
)z
2
( )
...

real root
2012-10-31 Dan Ellis 33
Cascade IIR Structure

Implement as cascade of
second order sections (in DFII)

Second order sections (SOS):

modular - any order from optimized block

well-behaved, real coefficients (sensitive?)


+
-2
2
+
z
-1
2
2
+ +
z
-1
+
p
0
-
1
+
z
-1

1
x y

2
2
+
2
2
(
2
2
+
2
2
)
fwd gain
factored
out
z
-1
z
-1
2012-10-31 Dan Ellis 34
Second-Order Sections

Free choices:

grouping of pole pairs with zero pairs

order of sections

Optimize numerical properties:

avoid very large values (overflow)

avoid very small values (quantization)

e.g. Matlabs zp2sos

attempt to put close roots in same section

intersperse gain & attenuation?


2012-10-31 Dan Ellis 35
Second Order Sections

Factorization affects intermediate values


Original System
(2 pair poles, zeros)
Factorization 1 Factorization 2

2012-10-31 Dan Ellis 36
Parallel IIR Structures

Can express H(z) as sum of terms (IZT)

Or, second-order terms:

Suggests parallel realization...




H(z) = consts +

z
1
=1
N

= 1

z
1
( )
F z
( )
z=


H(z) =
0
+

0k
+
1k
z
1
1+
1k
z
1
+
2k
z
2
k

2012-10-31 Dan Ellis 37


Parallel IIR Structures

Sum terms become


parallel paths

Poles of each SOS


are from full TF

System zeros arise


from output sum

Why do this?

stability/sensitivity

reuse common terms


+

01

11
+
z
-1
z
-1
-
11
-
21
+

0
+

02

12
+
z
-1
z
-1
-
12
-
22
x
y
2012-11-12 Dan Ellis 1
ELEN E4810: Digital Signal Processing
Topic 8:
Filter Design: IIR
1. Filter Design Specifications
2. Analog Filter Design
3. Digital Filters from Analog Prototypes
2012-11-12 Dan Ellis 2
1. Filter Design Specifications

The filter design process:


Design Implement Analysis
P
r
o
b
l
e
m
S
o
l
u
t
i
o
n
G(z)
transfer
function
performance
constraints
magnitude response
phase response
cost/complexity
FIR/IIR
subtype
order
platform
structure
...
2012-11-12 Dan Ellis 3
Performance Constraints

.. in terms of magnitude response:


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 P
-40
-30
-20
-10
0
frequencyW
g
a
i
n

/

d
B
passband
edge
frequency
Passband
Stopband
passband
ripple
minimum
stopband
attenuation
optimal
filter
will touch
here
stopband
edge
frequency
t
r
a
n
s
i
t
i
o
n

b
a
n
d
\G(e
jW
)\
2012-11-12 Dan Ellis 4

Best filter:

improving one usually worsens others

But: increasing filter order (i.e. cost)


can improve all three measures
Performance Constraints
smallest
Passband Ripple
greatest
Minimum SB Attenuation
narrowest
Transition Band
2012-11-12 Dan Ellis 5
Passband Ripple

Assume peak passband gain = 1


then minimum passband gain =

Or, ripple

1
1+
2

max
= 20log
10
1+
2
dB
PB ripple
parameter
1
\G(e
jW
)\
Passband
detail

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 P


-40
-30
-20
-10
0
\G(e
jW
)\
2012-11-12 Dan Ellis 6
Stopband Ripple

Peak passband gain is A larger than


peak stopband gain

Hence, minimum stopband attenuation


SB ripple
parameter

s
= 20log
10
1
A
= 20log
10
A dB
0
freqW
Stopband
detail
1
A
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 P
-40
-30
-20
-10
0
\G(e
jW
)\
2012-11-12 Dan Ellis 7
Filter Type Choice: FIR vs. IIR
FIR

No feedback
(just zeros)

Always stable

Can be
linear phase

High order
(20-2000)

Unrelated to
continuous-
time filtering
IIR

Feedback
(poles & zeros)

May be unstable

Difficult to control
phase

Typ. < 1/10th


order of FIR (4-20)

Derive from
analog prototype
BUT
2012-11-12 Dan Ellis 8
FIR vs. IIR

If you care about computational cost


use low-complexity IIR
(computation no object linear phase FIR)

If you care about phase response


use linear-phase FIR
(phase unimportant go with simple IIR)
2012-11-12 Dan Ellis 9
IIR Filter Design

IIR filters are directly related to


analog filters (continuous time)

via a mapping of H(s) (CT) to H(z) (DT) that


preserves many properties

Analog filter design is sophisticated

signal processing research since 1940s


Design IIR filters via analog prototype

need to learn some CT filter design


2012-11-12 Dan Ellis 10
2. Analog Filter Design

Decades of analysis of transistor-based


filters sophisticated, well understood

Basic choices:

ripples vs. flatness in stop and/or passband

more ripples narrower transition band


Family PassBd StopBd
Butterworth flat flat
Chebyshev I ripples flat
Chebyshev II flat ripples
Elliptical ripples ripples
2012-11-12 Dan Ellis 11
CT Transfer Functions

Analog systems: s-transform (Laplace)

frequency response still from a polynomial


Continuous-time Discrete-time

H
a
s
( )
= h
a
t
( )
e
st
dt


H
d
z
( )
= h
d
n
[ ]
z
n

Transform
Frequency
response
Pole/zero
diagram

H
a
j
( )

H
d
e
j
( )
s-plane
Re{s}
Im{s}
j
stable
poles
stable
poles
z-plane
Re{z}
Im{z}
1
e
j!
s scales freq
2012-11-12 Dan Ellis 12
Maximally flat in pass and stop bands

Magnitude
response (LP):


c
,
|H
a
(j)|
2
1

=
c
,
|H
a
(j)|
2
=
1
/
2
Butterworth Filters

H
a
j
( )
2
=
1
1+

c
( )
2N
filter
order
N
3dB point
0 1 2 3
0
0.2
0.4
0.6
0.8
1
\
H
a
(
j
7
)
\
7/7
c
N
N
2012-11-12 Dan Ellis 13
Butterworth Filters


c
, |H
a
(j)|
2
(
c
/)
2
N

flat
@ = 0 for n = 1 .. 2N-1

d
n
d
n
H
a
j
( )
2
= 0
Log-log
magnitude
response
6N dB/oct
rolloff
10
-1
10
0
10
1
-70
-60
-50
-40
-30
-20
-10
0
\
H
a
(
j
7
)
\

/

d
B
7/7
c
N
N
slope
-6N dB/oct
2012-11-12 Dan Ellis
\H
a
(j7)\
7
7
p
7
s

A
14
Butterworth Filters

How to meet design specifications?




1
1 +

p

c
( )
2N

1
1 +
2


1
1 +

s

c
( )
2N

1
A
2

k
1
=

A
2
1
=discrimination, 1

k =

s
=selectivity, < 1

N
1
2
log
10
A
2
1

2 ( )
log
10

s

p
( )
Design
Equation
2012-11-12 Dan Ellis 15
Butterworth Filters

... but what is H


a
(s)?

Traditionally, look it up in a table

calculate N normalized filter with


c
= 1

scale all coefficients for desired


c

In fact,
where

H
a
j
( )
2
=
1
1+ (

c
)
2N

H
a
s
( )
=
1
s p
i
( )
i


p
i
=
c
e
j
N+2i1
2N
i =1..N
s-plane
Re{s}
Im{s}

c






2N
= 1
odd-indexed uniform divisions of
c
-radius circle
2012-11-12 Dan Ellis
7
1kHz
7
p
5kHz
7
s

A
0 dB
= -1 dB
= -40 dB
16
Butterworth Example

1dB= 20log
10
1
1+
2

2
= 0.259

40dB= 20log
10
1
A

A =100
Design a Butterworth
filter with 1 dB cutoff
at 1kHz and a
minimum attenuation
of 40 dB at 5 kHz

p
= 5

N
1
2
log
10
9999
0.259
log
10
5
N = 4 3.28
2012-11-12 Dan Ellis
0 2000 4000 6000
-60
-50
-40
-30
-20
-10
0
freq / Hz
g
a
i
n

/

d
B
17
Butterworth Example

Order N = 4 will satisfy constraints;


What are
c
and filter coefficients?

from a table,
-1dB
= 0.845 when
c
= 1

c
= 1000/0.845 = 1.184 kHz

from a table, get normalized coefficients for


N = 4, scale by 11842

Or, use Matlab:


[B,A] =
butter(N,Wc,s);
M
2012-11-12 Dan Ellis 18
Chebyshev I Filter

Equiripple in passband (flat in stopband)


minimize maximum error
Chebyshev
polynomial
of order N
0 0.5 1 1.5 2
-40
-30
-20
-10
0
7
g
a
i
n

/

d
B
N
N
ripple
depth
|H(j)|
2
=
1
1 +
2
T
2
N
(

p
)
T
N
() =

cos(N cos
1
) || 1
cosh(N cosh
1
) || > 1
2012-11-12 Dan Ellis 19
Chebyshev I Filter
-2
0
-1
1
2
T
5
(7)
-1
0
1
2
3
T
5
2
(7)
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2
0
0.5
1
1/(1+0.1 T
5
2
(7
7
))
T
N
() =

cos(N cos
1
) || 1
cosh(N cosh
1
) || > 1
|H(j)|
2
=
1
1 +
2
T
2
N
(

p
)
2012-11-12 Dan Ellis 20
Chebyshev I Filter

Design procedure:

desired passband ripple "

min. stopband atten.,


p
,
s
N :

1
A
2
=
1
1+
2
T
N
2
(

p
)
=
1
1+
2
cosh N cosh
1
s

p
( )
[ ]
2

N
cosh
1
A
2
1

( )
cosh
1
s

p
( )
1/k
1
, discrimination
1/k, selectivity
2012-11-12 Dan Ellis 21
Chebyshev I Filter

What is H
a
(s)?

complicated, get from a table

.. or from Matlab cheby1(N,r,Wp,s)

all-pole; can inspect them:


..like squashed-in Butterworth (why?)
-1 -0.5 0 0.5 1
-1
-0.5
0
0.5
1
Re{s}
I
m
{
s
}
2012-11-12 Dan Ellis 22
Chebyshev II Filter

Flat in passband, equiripple in stopband

Filter has poles and zeros (some )

Complicated pole/zero pattern



H
a
j
( )
2
=
1
1+
2
T
N
(

p
)
T
N
(

)








2
zeros on imaginary axis
constant
~1/T
N
(1/)
0 0.5 1 1.5 2
-60
-50
-40
-30
-20
-10
0
7
g
a
i
n

/

d
B
N
N
peak of
stopband
ripples
2012-11-12 Dan Ellis
0 0.5 1 1.5 2
-60
-50
-40
-30
-20
-10
0
7
g
a
i
n

/

d
B
N
N
peak
SB
ripples
PB
ripple
depth
23
Elliptical (Cauer) Filters

Ripples in both passband and stopband

Complicated; not even closed form for


very narrow
transition band

H
a
j
( )
2
=
1
1+
2
R
N
2
(

p
)
function; satisfies
R
N
(
-1
) = R
N
()
-1
zeros for <1 poles for >1
2012-11-12 Dan Ellis 24
Analog Filter Types Summary
N = 6
r = 3 dB
A = 40 dB
0 0.5 1 1.5 2
-60
-50
-40
-30
-20
-10
0
0 0.5 1 1.5 2
-60
-50
-40
-30
-20
-10
0
0 0.5 1 1.5 2
-60
-50
-40
-30
-20
-10
0
0 0.5 1 1.5 2
-60
-50
-40
-30
-20
-10
0

c
g
a
i
n

/

d
B

g
a
i
n

/

d
B

g
a
i
n

/

d
B

g
a
i
n

/

d
B
Butterworth Chebyshev I
Chebyshev II Elliptical

p
r
r
A A

p
2012-11-12 Dan Ellis 25
Analog Filter Transformations

All filters types shown as lowpass;


other types (highpass, bandpass..)
derived via transformations

i.e.

General mapping of s-plane


BUT keep LHHP & j j;
poles OK, frequency response shuffled

H
LP
s
( )
s = F
1
s
( )
H
D
s
( )
Desired alternate
response; still a
rational polynomial
lowpass
prototype
^
2012-11-12 Dan Ellis 26
Lowpass-to-Highpass

Example transformation:

take prototype H
LP
(s) polynomial

replace s with

simplify and rearrange


new polynomial H
HP
(s)

H
HP
s
( )
= H
LP
s
( )
s=

p


p
s

p


p
s
^
2012-11-12 Dan Ellis 27
Lowpass-to-Highpass

What happens to frequency response?

Frequency axes inverted



s = j s =

p


p
j
= j

p


p
( )


=

p


p


=
p


=


p

<
p


<


p
LP passband HP passband

>
p


>


p
LP stopband HP stopband
imaginary axis
stays on self...
...freq.freq.
7
7p
7
^
7p
^
LPF
HPF
2012-11-12 Dan Ellis 28
Transformation Example
Design a Butterworth highpass filter
with PB edge -0.1dB @ 4 kHz (
p
)
and SB edge -40 dB @ 1 kHz (
s
)

Lowpass prototype: make


p
= 1

Butterworth -0.1dB @
p
=1, -40dB @
s
=4
^
^

s
=
( )

p


p


s
=
( )
4

N
1
2
log
10
A
2
1

2 ( )
log
10

s

p
( )

p
@ 0.1dB
1
1+ (

c
)
10
=10
0.1
10

c
=
p
/ 0.6866 =1.4564
N=5, 2N=10
2012-11-12 Dan Ellis 29
Transformation Example

LPF proto has

Map to HPF:


p

=
c
e
j
N+21
2N
Re{s}
Im{s}



H
LP
s ( ) =

c
N
s p

( )
=1
N


H
HP
s
( )
= H
LP
s
( )
s=

p


p
s


H
HP
s
( )
=

c
N

p


p
s
p

( )
=1
N

=

c
N
s
N

p


p
p

s
( )
=1
N

N zeros
@ s = 0
^
new poles @ s =
p

p
/p
l
^
^
2012-11-12 Dan Ellis 30
Transformation Example

In Matlab:
[N,Wc]=buttord(1,4,0.1,40,'s');
[B,A] = butter(N, Wc, 's');
[U,D] = lp2hp(B,A,2*pi*4000);

p

s
R
p
R
s
-2 -1 0 1
x 10
4
-1.5
-1
-0.5
0
0.5
1
1.5
x 10
4
Re{s}
I
m
{
s
}
0 1000 2000 3000 4000 5000
-60
-50
-40
-30
-20
-10
0
7 / Hz
g
a
i
n

/

d
B
2012-11-12 Dan Ellis 31
3. Analog Protos IIR Filters

Can we map high-performance CT


filters to DT domain?

Approach: transformation H
a
(s)G(z)
i.e.
where s = F(z) maps s-plane z-plane:

G z
( )
= H
a
s
( )
s=F z
( )
s-plane
Re{s}
Im{s}
z-plane
Re{z}
Im{z}
1
H
a
(s
0
)
G(z
0
)
Every value of G(z)
is a value of H
a
(s)
somewhere on the
s-plane & vice-versa
s = F(z)
z = F
-1
(s)
2012-11-12 Dan Ellis 32
CT to DT Transformation

Desired properties for s = F(z):

s-plane j axis z-plane unit circle


preserves frequency response values

s-plane LHHP z-plane unit circle interior


preserves stability of poles
s-plane
Re{s}
Im{s}
j
z-plane
Re{z}
Im{z}
1
e
j!
LHHPUCI
Imu.c.
2012-11-12 Dan Ellis 33
Bilinear Transformation

Solution:

Hence inverse:

Freq. axis?

Poles?
Bilinear
Transform

z =
1+ s
1 s
unique,
1:1 mapping

s = j z =
1+ j
1 j
|z| = 1 i.e.
on unit circle

s = + j z =
1+
( )
+ j
1
( )
j

z
2
=
1+ 2 +
2
+
2
1 2 +
2
+
2
< 0
|z| < 1
s =
1 z
1
1 +z
1
=
z 1
z +1
2012-11-12 Dan Ellis
-4 -2 0 2 4
-4
-3
-2
-1
0
1
2
3
4
-4 -2 0 2 4
-4
-3
-2
-1
0
1
2
3
4
34
Bilinear Transformation

How can entire half-plane fit inside u.c.?

Highly nonuniform warping!


s-plane
z-plane
2012-11-12 Dan Ellis 35
Bilinear Transformation

What is CTDT freq. relation ! ?

i.e.

infinite range of CT frequency


maps to finite DT freq. range

nonlinear; as !

z = e
j
s =
1e
j
1+e
j
=
2 j sin / 2
2 cos / 2
= j tan

2
u.circle
im.axis


= tan

2
( )
= 2 tan
1


< <

< <

d
d

pack it all in!


(CT)
(DT)
-5 0 5
-P
-0.5P
0
0.5P
P
7
W
2012-11-12 Dan Ellis 36
Frequency Warping

Bilinear transform makes


for all !,

G e
j
( )
= H
a
j
( )
=2 tan
1

Same gain &


phase (", A...),
in same order,
but with
warped
frequency axis
0 1 2 3 4 5 6
-60
-40
-20
0
-60 -40 -20 0
0
0.2
0.4
0.6
0.8

|H
a
(j)|
|G(e
j
)|
= 2tan
1
()

2012-11-12 Dan Ellis 37
Design Procedure

Obtain DT filter specs:

general form (LP, HP...),

Warp frequencies to CT:

Design analog filter for

H
a
(s), CT filter polynomial

Convert to DT domain:

G(z), rational polynomial in z

Implement digital filter!


p
,
s
,
1
1+
2
,
1
A

p
= tan

p
2

s
= tan

s
2

p
,
s
,
1
1+
2
,
1
A

G z
( )
= H
a
s
( )
s=
1z
1
1+z
1
Old-
style
2012-11-12 Dan Ellis 38
Bilinear Transform Example

DT domain requirements:
Lowpass, 1 dB ripple in PB, !
p
= 0.4,
SB attenuation 40 dB @ !
s
= 0.5,
attenuation increases with frequency
PB ripples,
SB monotonic
Chebyshev I
0.4 0. 1 5
WP
-40
-1
0
|
G
(
e
j
W
)
|

/

d
B
2012-11-12 Dan Ellis 39
Bilinear Transform Example

Warp to CT domain:

Magnitude specs:
1 dB PB ripple
40 dB SB atten.

p
= tan

p
2
= tan 0.2 = 0.7265 rad/sec

s
= tan

s
2
= tan 0.25 =1.0 rad/sec

1
1+
2
=10
1/ 20
= 0.8913 = 0.5087

1
A
=10
40 / 20
= 0.01 A =100
2012-11-12 Dan Ellis
0 0.5 1 1.5 2
-60
-50
-40
-30
-20
-10
0
0 0.2 0.4 0.6 0.8 1
-60
-50
-40
-30
-20
-10
0
\
H
a
(
j
7
)
\

/

d
B
7 W/P
\
G
(
e
j
W
)
\

/

d
B
CT DT
40
Bilinear Transform Example

Chebyshev I design criteria:

Design analog filter, map to DT, check:



N
cosh
1
A
2
1

( )
cosh
1
s

p
( )
= 7.09 i.e. need N = 8
>> N=8;
>> wp=0.7265;
>> pbripple = 1.0;
>> [B,A] = cheby1(N,pbripple,wp,'s');
>> [b,a] = bilinear(B,A,.5);
M
freqs(B,A) freqz(b,a)
2012-11-12 Dan Ellis 41
Other Filter Shapes

Example was IIR LPF from LP prototype

For other shapes (HPF, bandpass,...):

Transform LPX in CT or DT domain...


DT
specs
CT
specs
H
LP
(s)
H
D
(s)
G
LP
(z)
G
D
(z)
Bilinear
warp
Analog
design
CT
trans
DT
trans
Bilinear
transform
Bilinear
transform
2012-11-12 Dan Ellis 42
DT Spectral Transformations

Same idea as CT LPFHPF mapping,


but in z-domain:

To behave well, should:

map u.c. u.c. (preserve G(e


j!
) values)

map u.c. interior u.c. interior (stability)

i.e.

in fact, matches the definition of an


allpass filter ... replace delays with

z = F z
( )

G
D
z
( )
= G
L
z
( )
z=F z
( )
= G
L
F z
( ) ( )

F z
( )
=1 z =1

F z
( )
<1 z <1

F z
( )

F z
( )
1
2012-11-12 Dan Ellis 43
DT Frequency Warping

Simplest mapping
has effect of warping frequency axis:

z = F z
( )
=
z
1 z

tan

2
( )
=
1+
1
tan

2
( )
> 0 :
expand HF
< 0 :
expand LF
z e
j
z e
j

e
j
1 e
j
0 0.2 0.4 0.6 0.8 P
W
W
^
0
0.2
0.4
0.6
0.8
P
A = 0.6
A = -0.6
2012-11-12 Dan Ellis 44
Another Design Example

Spec:

Bandpass, from 800-1600 Hz (SR = 8kHz)

Ripple = 1dB, min. stopband atten. = 60 dB

8th order, best transition band


Use elliptical for best performance

Full design path:

prewarp spec frequencies

design analog LPF prototype

analog LPF BPF

CT BPF DT BPF (Bilinear)


2012-11-12 Dan Ellis
Another Design Example

Frequency prewarping

800 Hz @ SR = 8kHz
= 800/8000 x 2 rad/sample

L
=0.2 ,
U
=1600/8000 x 2 =0.4

L
= tan

L
/2 = 0.3249,
U
= 0.7265

Note distinction between:

applications CT domain (800-1600 Hz)


and

CT prototype domain (0.3249-0.7265 rad/s)


45
2012-11-12 Dan Ellis 46
Another Design Example

Or, do it all in one step in Matlab:


[b,a] = ellip(8,1,60,
[800 1600]/(8000/2));
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
-1000
-500
0
500
1000
Normalized Frequency (P rad/sample)
P
h
a
s
e

(
d
e
g
r
e
e
s
)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
-80
-60
-40
-20
0
Normalized Frequency (P rad/sample)
M
a
g
n
i
t
u
d
e

(
d
B
)
2012-11-19 Dan Ellis 1
ELEN E4810: Digital Signal Processing
Topic 9:
Filter Design: FIR
1. Windowed Impulse Response
2. Window Shapes
3. Design by Iterative Optimization
2012-11-19 Dan Ellis 2
1. FIR Filter Design

FIR filters

no poles (just zeros)

no precedent in analog filter design

Approaches

windowing ideal impulse response

iterative (computer-aided) design


2012-11-19 Dan Ellis 3
Least Integral-Squared Error

Given desired FR H
d
(e
j!
), what is the
best finite h
t
[n] to approximate it?

Can try to minimize


Integral Squared Error (ISE) of
frequency responses:
best in what sense?

=
1
2
H
d
e
j
( )
H
t
e
j
( )
2
d

= DTFT{h
t
[n]}
2012-11-19 Dan Ellis 4
Least Integral-Squared Error

Ideal IR is h
d
[n] = IDTFT{H
d
(e
j!
)},
(usually infinite-extent)

By Parseval, ISE

But: h
t
[n] only exists for n = -M..M ,

= h
d
n
[ ]
h
t
n
[ ]
2
n=



= h
d
n [ ] h
t
n [ ]
2
n=M
M

+ h
d
n [ ]
2
n<M, n>M

minimized by making
h
t
[n] = h
d
[n], -M n M
not altered by h
t
[n]
2012-11-19 Dan Ellis 5
Least Integral-Squared Error

Thus, minimum mean-squared error


approximation in 2M+1 point FIR is
truncated IDTFT:

Make causal by delaying by M points


h
t
[n] = 0 for n < 0
2M 0 M M M
n n
h
t
[n] h'
t
[n]
zero
h
t
[n] =

1
2

H
d
(e
j
)e
jn
d M n M
0 otherwise
2012-11-19 Dan Ellis
-20 -15 -10 -5 0 5 10 15
-0.05
0
0.05
0.1
0.15
0.2
n
h[n]
6
Approximating Ideal Filters

From topic 6,
ideal lowpass
has:
and:
!
!
c
!
c
(doubly infinite)
H
LP
(e
j
) =

1 || <
c
0
c
< || <
h
LP
[n] =
sin
c
n
n
2012-11-19 Dan Ellis 7
Approximating Ideal Filters

Thus, minimum ISE causal


approximation to an ideal lowpass
2M+1 points
Causal shift
0 5 10 15 20 25
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
h
L
P
[
n
]
n

h
LP
[n] =

sin
c
(nM)
(nM)
0 n 2M
0 otherwise
2012-11-19 Dan Ellis
0 0.2 0.4 0.6 0.8
WP
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
H
t
(
e
j
W
)
129
point
25
point
Gibbs ears
8
Gibbs Phenomenon

Truncated ideal filters have Gibbs Ears:


Increasing filter length
narrower ears
(reduces ISE)
but height the same
not optimal by
minimax criterion
(~9% overshoot)
2012-11-19 Dan Ellis
-20 -15 -10 -5 0 5 10 15
-0.05
0
0.05
0.1
0.15
0.2
n
h[n]
9
Where Gibbs comes from

Truncation of h
d
[n] to 2M+1 points is
multiplication by a rectangular window:

Multiplication in time domain is


convolution in frequency domain:
w
R
[n]
h
t
[n] = h
d
[n] w
R
[n]
w
R
[n] =

1 M n M
0 otherwise
g[n] h[n]
1
2

G(e
j
)H(e
j()
)d
2012-11-19 Dan Ellis
W
P P
W
P P
W
P P

10
Where Gibbs comes from

Thus, FR of truncated response


is convolution of ideal FR
and FR of rectangular window (pd.sinc):
H
d
(e
j!
)
DTFT{w
R
[n]}
periodic sinc...
H
t
(e
j!
)
sin Nx
sin x
convanim.mp4
2012-11-19 Dan Ellis 11
Where Gibbs comes from

Rectangular window:

Mainlobe width
( 1/L) determines
transition band

Sidelobe height
determines ripples
periodic sinc

invariant
with length
-P -0.5P 0 0.5P P
0
W
R
(e
jW
)
W
pP
2M + 1
Mainlobe
Sidelobes
/
w
R
[n] =

1 M n M
0 otherwise
W
R
(e
j
) =

M
n=M
e
jn
=
sin(2M+1)

2
sin

2
2012-11-19 Dan Ellis 12
2. Window Shapes for Filters

Windowing (infinite) ideal response


FIR filter:

Rectangular window has best ISE error

Other tapered windows vary in:

mainlobe transition band width

sidelobes size of ripples near transition

Variety of classic windows...



h
t
n
[ ]
= h
d
n
[ ]
w n
[ ]
2012-11-19 Dan Ellis
-10 -5 0 5 10
0
0.5
1
0 0.2 0.4 0.6 0.8
0
5
10
15
20
-10 -5 0 5 10
0
0.5
1
0 0.2 0.4 0.6 0.8
0
2
4
6
8
-10 -5 0 5 10
0
0.5
1
0 0.2 0.4 0.6 0.8
0
2
4
6
8
-10 -5 0 5 10
0
0.5
1
0 0.2 0.4 0.6 0.8
0
2
4
6
8
W
P
n
W
P
n
W
P
n
W
P
n
13


0.42 + 0.5cos(2
n
2M+1
)
+0.08cos(2
2n
2M+1
)
Window Shapes for FIR Filters

Rectangular:

Hann:

Hamming:

Blackman:

w n
[ ]
=1 M n M


0.5 + 0.5cos(2
n
2M+1
)


0.54 + 0.46cos(2
n
2M+1
)
double width
mainlobe
reduced 1st
sidelobe
triple width
mainlobe
big
sidelobes!
2012-11-19 Dan Ellis
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 P
W
-80
-70
-60
-50
-40
-30
-20
-10
0
Blackman
Hamming
Hann
Rectangular \
W
(
e
j
W
)
\

/

d
B
14
Window Shapes for FIR Filters

Comparison on dB scale:
2
2M+1
2012-11-19 Dan Ellis 15
Adjustable Windows

So far, discrete main-sidelobe tradeoffs..

Kaiser window = parametric, continuous


tradeoff:

Empirically, for min. SB atten. of dB:



w n
[ ]
=
I
0
1 (
n
M
)
2
( )
I
0
()
M n M
modified zero-order
Bessel function
required
order
transition
width
=

0.11( 8.7) > 50


0.58( 21)
0.4
+ 0.08( 21) 21 50
0 < 21
N =
8
2.3
2012-11-19 Dan Ellis 16
Windowed Filter Example

Design a 25 point FIR low-pass filter


with a cutoff of 600 Hz (SR = 8 kHz)

No specific transition/ripple reqs


compromise: use Hamming window

Convert the frequency to radians/


sample:
!
2
8 kHz

4 kHz
0.15
600 Hz
H(e
j!
)

c
=
600
8000
2 = 0.15
(f
s
)
2012-11-19 Dan Ellis 17
Windowed Filter Example
1. Get ideal filter impulse response:
2. Get window:
Hamming @ N = 25 M = 12 (N = 2M+1)
3. Apply window:

c
= 0.15

h
d
n
[ ]
=
sin0.15n
n

w n
[ ]
= 0.54 + 0.46cos 2
n
25
( )
12 n 12

h n
[ ]
= h
d
n
[ ]
w n
[ ]
=
sin0.15n
n
0.54 + 0.46cos
2n
25
( )
12 n 12
M
-20 -10 0 10
n
h[n]
0
0.05
0.1
0.15
2012-11-19 Dan Ellis 18
Freq. Resp. (FR) Arithmetic

Ideal LPF has pure-real FR i.e.


(!) = 0, H(e
j!
) = |H(e
j!
)|
Can build piecewise-constant FRs by
combining ideal responses, e.g. HPF:
!
!
c
!
c
!
!
1
-1
1
[n]
+
-h
LP
[n]
=
h
HP
[n]
i.e. H(e
j!
) = 1
H
LP
(e
j!
) = 1
for |!| < !
c
= [n] - (sin!
c
n)/n
w
o
u
l
d
n

t

w
o
r
k

i
f
p
h
a
s
e
s

w
e
r
e

n
o
n
z
e
r
o
!
+
=
0
2012-11-19 Dan Ellis 19
3. Iterative FIR Filter Design

Can derive filter coefficients by


iterative optimization:

Gradient descent / nonlinear optimizn


Filter coefs
h[n]
Goodness of fit
criterion
error "
Estimate
derivatives
"/h[n]
Update filter
to reduce "
Desired
response
H(e
j!
)
2012-11-19 Dan Ellis 20
Error Criteria

= W
( )
D e
j
( )
H e
j
( ) [ ]
R

p
d
error
measurement
region
error
weighting
desired
response
actual
response
exponent:
2 least sq
minimax
= W(!)[D(e
j!
) H(e
j!
)]
W
1
W
2
W
3
W
4
W
W
H(e
jW
)
W(W)
R
D(e
jW
)
W
E(W)
2012-11-19 Dan Ellis 21
Minimax FIR Filters

Iterative design of FIR filters with:

equiripple (minimax criterion)

linear-phase
symmetric IR h[n] = ()h[-n]

Recall: symmetric FIR filters have FR


with pure-real

i.e. combo of cosines of multiples of !

H e
j
( )
= e
jM

H
( )


H
( )
= a k
[ ]
cos k
( )
k=0
M

a[0] = h[M]
a[k] = 2h[M - k]
n
M
(type I
order 2M)
2012-11-19 Dan Ellis 22
Minimax FIR Filters

Now, cos(k!) can be expressed as a


polynomial in cos(!)
k
and lower powers

e.g. cos(2!) = 2(cos!)


2
- 1

Thus, we can find s such that

[k]s easily lead to a[k]s


M
th
order
polynomial in cos!

H() =
M

k=0
a[k]cos(k) =
M

k=0
[k](cos)
k


H
( )
= a k
[ ]
cos k
( )
k=0
M

2012-11-19 Dan Ellis 23


Minimax FIR Filters

An M
th
order polynomial has at most
M - 1 maxima and minima:
has at most M-1 min/max (ripples)


H
( )
= k
[ ]
cos
( )
k
k=0
M

M
th
order
polynomial in cos!


H
( )
5
th
order
polynomial
in cos!
1
2
3
4
-1 -0.5 0 0.5 1
-0.2
-0.1
0
0.1
0.2
0 0.2 0.4 0.6 0.8 P
W WP
-0.2
-0.1
0
0.1
0.2
cosW
cosW
W
1
2
3
4
M

k=0
[k](cos )
k

H()
2012-11-19 Dan Ellis 24

Key ingredient to Parks-McClellan:


is the unique, best, weighted-
minimax order 2M approx. to D(e
j!
)

has at least M+2 extremal freqs


over ! subset R

error magnitude is equal at each extremal:

peak error alternates in sign:



Alternation Theorem


H
( )


H
( )

0
<
1
< ... <
M
<
M+1


i
( )
= i


i
( )
=
i+1
( )

2012-11-19 Dan Ellis 25


Alternation Theorem

Hence, for a frequency response:

If "(!) reaches a peak


error magnitude " at
some set of extremal
frequences !
i

And the sign of the peak


error alternates

And we have at least


M+2 of them

Then optimal minimax


(10th order filter, M = 5)
0 W
p
W
c
P
W
2
W
3
0
0.2
0.4
0.6
0.8
1
W
1
W
0
W
4
W
5
W
6
-E
0
E
E
W
D(e
jW
) desired frq.resp.
H(e
jW
) response mag.
E(W)
error
W(W)
scaled error
bound
band-edge extrema
local min/max extrema
2012-11-19 Dan Ellis 26
Alternation Theorem

By Alternation Theorem,
M+2 extrema of alternating signs
optimal minimax filter

But has at most M-1 extrema


need at least 3 more from band edges

2 bands give 4 band edges


can afford to miss only one

Alternation rules out transition band


edges, thus have 1 or 2 outer edges


H
( )
2012-11-19 Dan Ellis 27
Alternation Theorem

For M = 5 (10
th
order):

8 extrema (M+3,
4 band edges)
- great!

7 extrema (M+2,
3 band edges)
- OK!

6 extrema (M+1,
only 2 transition
band edges)
NOT OPTIMAL
0 0.2 0.4 0.6 0.8 P
W
W
H
1
W
~
H
2
W
~
H
3
W
~
W
P
P
0
0.5
1
0 0.2 0.4 0.6 0.8
0
0.5
1
0 0.2 0.4 0.6 0.8
0
0.5
1
2012-11-19 Dan Ellis 28
Parks-McClellan Algorithm

To recap:

FIR CAD constraints


D(e
j!
), W(!) "(!)

Zero-phase FIR
H(!) =
k

k
cos
k
! M-1 min/max

Alternation theorem
optimal M+2 pk errs, alterng sign

Hence, can spot best filter when we


see it but how to find it?
~
2012-11-19 Dan Ellis 29
Parks-McClellan Algorithm

Alternation [H(!)-D(!)]/W(!) must =


" at M+2 (unknown) frequencies {!
i
}...

Iteratively update h[n] with


Remez exchange algorithm:

estimate/guess M+2 extremals {!


i
}

solve for [n], " ( h[n] )

find actual min/max in "(!) new {!


i
}

repeat until |"(!


i
)| is constant

Converges rapidly!
~ ~
M
2012-11-19 Dan Ellis 30
Parks-McClellan Algorithm

In Matlab,
>> h=firpm(10,[0 0.4 0.6 1],
[1 1 0 0],
[1 2]);
filter order (2M) band edges
desired magnitude
at band edges
error weights
per band
-5 0 5
-0.2
0
0.2
0.4
0.6
0 0.5 1
0
0.5
1
-1 0 1 2
-1
-0.5
0
0.5
1
h[n]
H(W)
n
W
~
z-plane
2012-11-28 Dan Ellis 1
ELEN E4810: Digital Signal Processing
Topic 10:
The Fast Fourier Transform
1. Calculation of the DFT
2. The Fast Fourier Transform algorithm
3. Short-Time Fourier Transform
2012-11-28 Dan Ellis 2
1. Calculation of the DFT

Filter design so far has been oriented to


time-domain processing - cheaper!

But: frequency-domain processing


makes some problems very simple:

use all of x[n], or use short-time windows

Need an efficient way to calculate DFT


Fourier domain
processing
x[n] y[n] DFT IDFT
X[k] Y[k]
2012-11-28 Dan Ellis 3
The DFT

Recall the DFT:

discrete transform of discrete sequence

Matrix form:

X[k] = x[n]W
N
kn
n=0
N1


W
N
= e
j
2
N
( )
W
N
r
has only
N distinct values
W
N
@2/N
Structure
opportunities
for
efficiency

X[0]
X[1]
X[2]
.
.
.
X[N 1]

1 1 1 1
1 W
1
N
W
2
N
W
(N1)
N
1 W
2
N
W
4
N
W
2(N1)
N
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1 W
(N1)
N
W
2(N1)
N
W
(N1)
2
N

x[0]
x[1]
x[2]
.
.
.
x[N 1]

2012-11-28 Dan Ellis 4


Computational Complexity

N complex multiplies
+ N-1 complex adds per point (k)
N points (k = 0.. N-1)

cpx mult: (a+jb)(c+jd) = ac - bd + j(ad + bc)


= 4 real mults + 2 real adds

cpx add = 2 real adds

N points: 4N
2
real mults, 4N
2
-2N real adds

X[k] = x[n]W
N
kn
n=0
N1

2012-11-28 Dan Ellis 5


Goertzels Algorithm

Now:

i.e.
where


X k
[ ]
= x
[ ]
W
N
k
=0
N1

= W
N
kN
x
[ ]

W
N
k N
( )
looks like a
convolution

X k
[ ]
= y
k
N
[ ]
y
k
n
[ ]
= x
e
n
[ ]
h
k
[n]
x
e
[n] = {
x[n] 0 n < N
0 n = N
h
k
[n] = {
W
N
-kn
n 0
0 n < 0
+
z
-1
W
N
-k
x
e
[n]
x
e
[N] = 0
y
k
[n]
y
k
[-1] = 0
y
k
[N] = X[k]
2012-11-28 Dan Ellis 6
Goertzels Algorithm

Separate filters for each X[k]

can calculate for just a few values of k

No large buffer, no coefficient table

Same complexity for full X[k]


(4N
2
mults, 4N
2
- 2N adds)

but: can halve multiplies by making the


denominator real:

H z
( )
=
1
1W
N
k
z
1
=
1W
N
k
z
1
1 2cos
2k
N
z
1
+ z
2
evaluate only
for last step
2 real mults
per step
2012-11-28 Dan Ellis 7
2. Fast Fourier Transform FFT

Reduce complexity of DFT


from O(N
2
) to O(NlogN)

grows more slowly with larger N

Works by decomposing large DFT into


several stages of smaller DFTs

Often provided as a highly optimized


library
2012-11-28 Dan Ellis 8
Decimation in Time (DIT) FFT

Can rearrange DFT formula in 2 halves:



X k
[ ]
= x n
[ ]
W
N
nk
n=0
N1

= x 2m
[ ]
W
N
2mk
+ x 2m+1
[ ]
W
N
2m+1
( )
k
( )
m=0
N
2
1

= x 2m
[ ]
W
N
2
mk
m=0
N
2
1

+W
N
k
x 2m+1
[ ]
W
N
2
mk
m=0
N
2
1

N/2 pt DFT of x for even n N/2 pt DFT of x for odd n


X
0
[<k>
N/2
] X
1
[<k>
N/2
]
Arrange
terms
in pairs...
Group terms
from each
pair
k = 0.. N-1
2012-11-28 Dan Ellis 9
Decimation in Time (DIT) FFT

We can evaluate an N-pt DFT as two


N/2-pt DFTs (plus a few mults/adds)

But if DFT
N
{} ~ O(N
2
)
then DFT
N/2
{} ~ O((N/2)
2
) = 1/4 O(N
2
)
Total computation ~ 2 1/4 O(N
2
)
= 1/2 the computation (+") of direct DFT

DFT
N
x n
[ ] { }
= DFT
N
2
x
0
n
[ ] { }
+W
N
k
DFT
N
2
x
1
n
[ ] { }
x[n] for even n x[n] for odd n
2012-11-28 Dan Ellis 10
One-Stage DIT Flowgraph
Classic FFT structure
Even
points
from
x[n]
Odd
points
from
x[n]
Same as
X[0..3]
except for
factors on
X
1
[]
terms
twiddle factors:
always apply to
odd-terms output
NOT mirror-image

X k
[ ]
= X
0
k
N
2
[ ]
+W
N
k
X
1
k
N
2
[ ]
x[0]
x[2]
x[4]
x[6]
x[1]
x[3]
x[5]
x[7]
X[0]
X[1]
X[2]
X[3]
X[4]
X[5]
X[6]
X[7]
X
0
[0]
X
0
[1]
X
0
[2]
X
0
[3]
X
1
[0]
X
1
[1]
X
1
[2]
X
1
[3]
DFT
N
2
DFT
N
2
W
N
0
W
N
1
W
N
2
W
N
3
W
N
4
W
N
5
W
N
6
W
N
7
2012-11-28 Dan Ellis 11

If decomposing one DFT


N
into two
smaller DFT
N/2
s speeds things up ...
Why not further divide into DFT
N/4
s ?

i.e.

make:

Similarly,
Multiple DIT Stages

X k
[ ]
= X
0
k
N
2
[ ]
+W
N
k
X
1
k
N
2
[ ]
0 k < N

X
0
k
[ ]
= X
00
k
N
4
[ ]
+W
N
2
k
X
01
k
N
4
[ ]
0 k < N/2
N/4-pt DFT of even points
in even subset of x[n]
N/4-pt DFT of odd points
from even subset

X
1
k
[ ]
= X
10
k
N
4
[ ]
+W
N
2
k
X
11
k
N
4
[ ]
2012-11-28 Dan Ellis 12
Two-Stage DIT Flowgraph
x[0]
x[4]
x[2]
x[6]
x[1]
x[5]
x[3]
x[7]
X[0]
X[1]
X[2]
X[3]
X[4]
X[5]
X[6]
X[7]
X
0
[0]
X
00
X
01
X
10
X
11
X
0
[1]
X
0
[2]
X
0
[3]
X
1
[0]
X
1
[1]
X
1
[2]
X
1
[3]
DFT
N
4
DFT
N
4
DFT
N
4
DFT
N
4
W
N
0
W
N
1
W
N
2
W
N
3
W
N
4
W
N
5
W
N
6
W
N
7
W
N/2
3
W
N/2
0
W
N/2
3
W
N/2
0
different from before same as before
2012-11-28 Dan Ellis 13
Multi-stage DIT FFT

Can keep doing this until we get down


to 2-pt DFTs:
N = 2
M
-pt DFT reduces to M stages of
twiddle factors & summation
(O(N
2
) part vanishes)
real mults < M4N , real adds < 2M2N
complexity ~ O(NM) = O(Nlog
2
N)
DFT
2
X[0] = x[0] + x[1]
X[1] = x[0] - x[1]
-1 = W
2
1
1 = W
2
0
butterfly element

2012-11-28 Dan Ellis 14



W
N
r+
N
2
= e
j
2 r+
N
2
( )
N
= e
j
2r
N
e
j
2N / 2
N
= W
N
r
FFT Implementation Details

Basic butterfly (at any stage):

Can simplify:
W
N
r
W
N
r+N/2
X
X
[r]
X
X
[r+N/2]
X
X0
[r]
X
X1
[r]

2 cpx mults
X
X
[r]
X
X
[r+N/2]
X
X0
[r]
X
X1
[r]
W
N
r
-1
just one cpx mult!
i.e. SUB rather than ADD
2012-11-28 Dan Ellis 15
8-pt DIT FFT Flowgraph

-1s absorbed into summation nodes

W
N
0
disappears

in-place algorithm: sequential stages


b
i
t
-
r
e
v
e
r
s
e
d

i
n
d
e
x
i
n
g
W
4
W
4
W
8
W
8
W
8
x[0]
x[4]
x[2]
x[6]
x[1]
x[5]
x[3]
x[7]
000
100
010
110
001
101
011
111
X[0]
X[1]
X[2]
X[3]
X[4]
X[5]
X[6]
X[7]
-
-
-
-
-
-
-
-
-
-
-
-
2
3
2012-11-28 Dan Ellis 16
FFT for Other Values of N

Having N = 2
M
meant we could divide
each stage into 2 halves = radix-2 FFT

Same approach works for:

N = 3
M
radix-3

N = 4
M
radix-4 - more optimized radix-2

etc...

Composite N = abcd mixed radix


(different N/r point FFTs at each stage)

.. or just zero-pad to make N = 2


M

M
2012-11-28 Dan Ellis 17
Inverse FFT

Recall IDFT:

Thus:

Hence, use FFT to calculate IFFT:



x[n] =
1
N
X[k]W
N
nk
k=0
N1

only differences
from forward DFT

Nx
*
[n] = X[k]W
N
nk
( )
*
k=0
N1

= X
*
[k]W
N
nk
k=0
N1

Forward DFT of x[n] = X


*
[k]|
k=n
i.e. time sequence made from spectrum
DFT
Re{x[n]}
Im{x[n]}
Re
Im
Re
Im
Re{X[k]}
Im{X[k]}
1/N
-1/N -1
pure real flowgraph
x n


1
N
X
*
k

W
N
nk
k0
N 1

*
2012-11-28 Dan Ellis 18

If x[n] is pure-real, DFT wastes mults

Real x[n] Conj. symm. X[k] = X


*
[-k]

Given two real sequences, x[n] and w[n]


call y[n] = jw[n] , v[n] = x[n] + y[n]

N-pt DFT V[k] = X[k] + Y[k]


but: V[k]+V
*
[-k] = X[k]+X
*
[-k]+Y[k]+Y
*
[-k]
X[k]=
1
/
2
(V[k]+V
*
[-k]) , W[k]=
-j
/
2
(V[k]-V
*
[-k])

i.e. compute DFTs of two N-pt real


sequences with a single N-pt DFT
DFT of Real Sequences
X[k]
-Y[k]
2012-11-28 Dan Ellis 19
3. Short-Time
Fourier Transform (STFT)

Fourier Transform (e.g. DTFT) gives


spectrum of an entire sequence:

How to see a time-varying spectrum?

e.g. slow AM of a sinusoid carrier:



x n
[ ]
= 1 cos
2tn
N
|
\

|
.
|
cose
0
n
0 200 400 600 800 1000
-2
-1
0
1
2
n
x[n]
2012-11-28 Dan Ellis 20
Fourier Transform of AM Sine

Spectrum of
whole sequence
indicates
modulation
indirectly...

... as
cancellation
between
closely-
tuned
sines
2cAcB
= cA+B
+cA-B
Nsin2kn
N
-Nsin2(k-1)n
N 2
-Nsin2(k+1)n
N 2
0 0.02 0.04 0.06 0.08
0
200
400
600
N
N/2
\X[k]\
WP
k/(N/2)
-1
-0.5
0
0.5
1
-1
-0.5
0
0.5
1
0 128 256 384 512 640 768 896
-1
-0.5
0
0.5
1
M
2012-11-28 Dan Ellis 21
Fourier Transform of AM Sine

Sometimes wed rather separate


modulation and carrier:
x[n] = A[n]cos!
0
n

A[n] varies on a
different (slower) timescale

One approach:

chop x[n] into short sub-sequences ..

.. where slow modulator is ~ constant

DFT spectrum of pieces show variation


!
A[n]
!
0
2012-11-28 Dan Ellis 22
FT of Short Segments

Break up x[n] into successive, shorter


chunks of length N
FT
, then DFT each:
Shows amplitude modulation
of !
0
energy
0 128 256 384 512 640 768 896 1024 = N
-2
-1
0
1
2
0 64
0
50
100
n
k
k
x
0
[n]
x[n]
X
0
[k]
x
1
[n]
X
1
[k]
x
2
[n]
X
2
[k]
x
3
[n]
X
3
[k]
x
4
[n]
X
4
[k]
x
5
[n]
X
5
[k]
x
6
[n]
X
6
[k]
x
7
[n]
X
7
[k]
k
0
= W
0
N
FT
2P
N
FT
= N/8
2012-11-28 Dan Ellis 23
The Spectrogram

Plot successive DFTs in time-frequency:

This image is called the Spectrogram


time hopsize (between successive frames)
= 128 points
128 256 384 512 640 768 896 1024
k
n
k k
\X
i
[k]\
\X[k,n]\
0
0
5
10
15
20
0
40
60
80
100
120
X
0
[k] X
1
[k] X
2
[k] X
3
[k] X
4
[k] X
5
[k] X
6
[k] X
7
[k]
2012-11-28 Dan Ellis 24
Short-Time Fourier Transform

Spectrogram = STFT magnitude


plotted on time-frequency plane

STFT is (DFT form):

intensity as a function of time & frequency



X k,n
0
[ ]
= x n
0
+ n
[ ]
w[n] e
j
2kn
N
FT
n=0
N
FT
1

frequency
index
time
index
N
FT
points of x
starting at n
window
DFT
kernel
2012-11-28 Dan Ellis 25
STFT Window Shape

w[n] provides time localization of STFT

e.g. rectangular
selects x[n], n
0
n < n
0
+N
W

But: resulting spectrum has same


problems as windowing for FIR design:
n
w[n]

X e
j
,n
0
( )
= DTFT x n
0
+ n
[ ]
w n
[ ] { }
= e
jn
0
X e
j
( )
W e
j
( )
( )
d

spectrum of short-time window


is convolved with (twisted) parent spectrum
DTFT
form of
STFT
2012-11-28 Dan Ellis 26
STFT Window Shape

e.g. if x[n] is a pure sinusoid,

Hence, use tapered window for w[n]



blurring (mainlobe)
+ ghosting (sidelobes)
e.g. Hamming

w n
[ ]
=
0.54 + 0.46cos(2
n
2M+1
)
sidelobes
< -40 dB
W
P
W

X(e
jW
) W(e
jW
)
W
-10 -5 0 5 10
W n
w[n] W(e
jW
)
2012-11-28 Dan Ellis 27
STFT Window Length

Length of w[n] sets temporal resolution

Window length 1/(Mainlobe width)

more time detail less frequency detail


short window measures
only local properties
longer window averages
spectral character
shorter window
more blurred
spectrum
0 200 400 600 800 1000
-0.1
0
0.1
0.2
0 200 400 600 800 1000
-0.1
0
0.1
0.2
x[n] x[n]
w [n]
L S
w [n]
-100 -50 0 50 100
0
0.5
1
-P -0.5P 0 0.5P P
0
10
20
-100 -50 0 50 100
0
0.5
1
-P -0.5P 0 0.5P P
0
10
20
n
n
w
L
[n]
w
S
[n]
W
W
W
S
(e
jW
)
W
L
(e
jW
)
N
1
pts
N
2
pts
N
1
zero at 4
N
2
zero at 4
2012-11-28 Dan Ellis 28
STFT Window Length

Can illustrate time-frequency tradeoff


on the time-frequency plane:

Alternate tilings
of time-freq:
disks show blurring
due to window length;
area of disk is constant
Uncertainty principle:
ft k
half-length window half as many DFT samples
0 100 200 300
0
0.5
1
0
50
100
150
200
250
n
k
2012-11-28 Dan Ellis 29
Spectrograms of Real Sounds
individual t-f
cells merge
into continuous
image
time-domain
successive
short
DFTs
time / s
time / s
f
r
e
q

/

H
z
i
n
t
e
n
s
i
t
y

/

d
B
2.35 2.4 2.45 2.5 2.55 2.6
0
1000
2000
3000
4000
f
r
e
q

/

H
z
0
1000
2000
3000
4000
0
0.1
-50
-40
-30
-20
-10
0
10
0 0.5 1 1.5 2 2.5
2012-11-28 Dan Ellis 30
Narrowband vs. Wideband

Effect of varying window length:


1.4 1.6 1.8 2 2.2 2.4 2.6
f
r
e
q

/

H
z
time / s
level
/ dB
0
1000
2000
3000
4000
f
r
e
q

/

H
z
0
1000
2000
3000
4000
0
0.2
W
i
n
d
o
w

=

2
5
6

p
t

N
a
r
r
o
w
b
a
n
d

W
i
n
d
o
w

=

4
8

p
t

W
i
d
e
b
a
n
d

-50
-40
-30
-20
-10
0
10
M
2012-11-28 Dan Ellis 31
Spectrogram in Matlab
>> [d,sr]=wavread(mpgr1_sx419.wav');
>> Nw=256;
>> specgram(d,Nw,sr)
>> caxis([-80 0])
>> colorbar
(hann) window length
actual sampling rate
(to label time axis)
dB
Time
F
r
e
q
u
e
n
c
y


0.5 1 1.5 2 2.5 3
0
2000
4000
6000
8000
-80
-60
-40
-20
0
2012-11-28 Dan Ellis
-60
-40
-20
0 -1
0
1
-1
0
1
n
Re{x[n]}
I
m
{
x
[
n
]
}
32
STFT as a Filterbank

Consider one row of STFT:


where

Each STFT row is output of a filter


(subsampled by the STFT hop size)

X
k
n
0
[ ]
= x n
0
+ n
[ ]
w n
[ ]
e
j
2kn
N
n=0
N1

= h
k
m
[ ]
x n
0
m
[ ]
m=0
N1
( )


h
k
n
[ ]
= w n
[ ]
e
j
2kn
N
just one freq.
convolution
with
complex IR
2012-11-28 Dan Ellis 33
STFT as a Filterbank

If
then

Each STFT row is the same bandpass


response defined by W(e
j!
),
frequency-shifted to a given DFT bin:

h
k
n
[ ]
= w
( )
n
[ ]
e
j
2kn
N

H
k
e
j
( )
= W e

( )
j
2k
N
( )
( )
shift-in-!
A bank of identical,
frequency-shifted
bandpass filters:
filterbank
\W(e
jW
)\
\H
1
(e
jW
)\ \H
2
(e
jW
)\

W
P
2012-11-28 Dan Ellis 34
STFT Analysis-Synthesis

IDFT of STFT frames can reconstruct


(part of) original waveform

e.g. if
then

Can shift by n
0
, combine, to get x[n]:

Could divide by w[n-n


0
] to recover x[n]...

X k,n
0
[ ]
= DFT x n
0
+ n
[ ]
w n
[ ] { }
IDFT X k,n
0
[ ] { }
= x n
0
+ n
[ ]
w n
[ ]
^
n
0
n
^
x[n]
x[n]w[n-n
0
]
2012-11-28 Dan Ellis 35
STFT Analysis-Synthesis

Dividing by small values of w[n] is bad

Prefer to
overlap windows:
i.e. sample X[k,n
0
]
at n
0
= rH where H = N/2 (for example)

Then
hopsize window length

x n
[ ]
= x n
[ ]
w n rH
[ ]
r

= x n
[ ]

w n rH
[ ]
r

=1
if
n
^
x[n]
x[n]w[n-rH]
2012-11-28 Dan Ellis 36
STFT Analysis-Synthesis

Hann or Hamming windows


with 50% overlap
sum to constant

Can modify individual frames of X[k,n]


and then reconstruct

complex, time-varying modifications

tapered overlap makes things OK



0.54 + 0.46cos(2
n
N
)
( )
+ 0.54 + 0.46cos(2
n
N
2
N
)
( )
=1.08
0 20 40 60 80
0
0.2
0.4
0.6
0.8
1
n
w[n] w[n-N/2]
w[n] + w[n-N/2]
2012-11-28 Dan Ellis 37
STFT Analysis-Synthesis

e.g. Noise reduction:


STFT of
original speech
Speech corrupted
by white noise
Energy threshold
mask
M
100 200 300
20
40
60
80
100
120
r
k
2012-12-05 Dan Ellis 1
ELEN E4810: Digital Signal Processing
Topic 11:
Continuous Signals
1. Sampling and Reconstruction
2. Quantization
2012-12-05 Dan Ellis 2
1. Sampling & Reconstruction

DSP must interact with an analog world:


DSP
Anti-
alias
filter
Sample
and
hold
A to D
Reconstruction
filter
D to A
Sensor Actuator
WORLD
x(t) x[n] y[n] y(t)
ADC DAC
2012-12-05 Dan Ellis 3
Sampling: Frequency Domain

Sampling: CT signal DT signal by


recording values at sampling instants:

What is the relationship of the spectra?

i.e. relate
and

g n
[ ]
= g
a
nT
( )
Discrete Continuous
Sampling period T
samp.freq.
samp
= 2/T rad/sec

G
a
j
( )
= g
a
t
( )
e
jt
dt

G(e
j
) = g n
[ ]
e
jn

CTFT
DTFT
in rad/second
! in rad/sample
2012-12-05 Dan Ellis 4
Sampling

DT signals have same content


as CT signals gated by an impulse train:

g
p
(t) = g
a
(t)p(t) is a CT signal with the
same information as DT sequence g[n]
t
t
t
p(t)
g
a
(t)
g
p
(t)
sampled signal:
still continuous
- but discrete
values
T 3T

= t nT
( )
n=

CT delta fn

2012-12-05 Dan Ellis 5


Spectra of sampled signals

Given CT

CTFT Spectrum

Compare to DTFT

i.e.
by linearity
! ~ T
~ !/T

/
T
=
samp
/2
g
p
(t) =

n=
g
a
(nT) (t nT)
G
p
(j) = F{g
p
(t)} =

n
g
a
(nT)F{(t nT)}
G
p
(j) =

n
g
a
(nT)e
jnT
G(e
j
) =

n
g[n]e
jn
G(e
j
) = G
p
(j)|
T=
2012-12-05 Dan Ellis 6
Spectra of sampled signals

Also, note that


is periodic, thus has Fourier Series:

But
so

p t
( )
= t nT
( )
n


p t
( )
=
1
T
e
j
2
T
( )
kt
k=



c
k
=
1
T
p t
( )
e
j 2kt / T
dt
T / 2
T / 2

=
1
T

F e
j
0
t
x t
( )
{ }
= X j
0
( ) ( )
shift in
frequency


G
p
j ( ) =
1
T
G
a
j k
samp
( ) ( )
k

- scaled sum of replicas of G


a
(j)
shifted by multiples of sampling frequency
samp
2012-12-05 Dan Ellis 7
CT and DT Spectra

G e
j
( )
= G
p
j
( )
T=
=
1
T
G
a
j

T
k
2
T
( ) ( )
k



G e
jT
( )
=
1
T
G
a
j k
samp
( ) ( )
k

DTFT CTFT

g
a
t
( )
G
a
j
( )

p t
( )
P j
( )

g
p
t
( )
G
p
j
( )

g n
[ ]
G e
j
( )

So:

M
g
a
(t) is bandlimited
@

samp

samp

!
M
2
samp
2
samp
M/T
shifted/scaled copies
2 4 2 4
at
have

samp

2
T

T
2
2012-12-05 Dan Ellis 8
Avoiding aliasing

Sampled analog signal has spectrum:

g
a
(t) is bandlimited to
M
rad/sec

When sampling frequency


samp
is large...
no overlap between aliases
can recover g
a
(t) from g
p
(t)
by low-pass filtering

samp

samp
M

samp

M

samp
+
M
G
p
(j)
G
a
(j)
alias of baseband
signal
2012-12-05 Dan Ellis 9
Aliasing & The Nyquist Limit

If bandlimit
M
is too large, or sampling
rate
samp
is too small, aliases will overlap:

Spectral effect cannot be filtered out


cannot recover g
a
(t)

Avoid by:

i.e. bandlimit g
a
(t) at

M
samp

samp

samp

M
G
p
(j)

samp

M

M

samp
2
M
Sampling theorum
Nyquist
frequency

samp
2
2012-12-05 Dan Ellis 10
Anti-Alias Filter

To understand speech, need ~ 3.4 kHz


8 kHz sampling rate (i.e. up to 4 kHz)

Limit of hearing ~20 kHz


44.1 kHz sampling rate for CDs

Must remove energy above Nyquist with


LPF before sampling: Anti-alias filter
M
ADC
Anti-alias
filter
Sample
& hold
A to D
space for
filter rolloff
2012-12-05 Dan Ellis 11
Sampling Bandpass Signals

Signal is not always in baseband


around = 0 ... may be at higher :

If aliases from sampling dont overlap,


no aliasing distortion; can still recover:

Basic limit:
samp
/2 bandwidth

H

L

L

H
A
Bandwidth =
H

L

samp

samp
2
samp
A/T

samp
/2

2012-12-05 Dan Ellis 12

make a continuous
impulse train g
p
(t)

lowpass filter to
extract baseband
g
a
(t)
Reconstruction

To turn g[n]
back to g
a
(t):

Ideal reconstruction filter is brickwall

i.e. sinc - not realizable (especially analog!)

use something with finite transition band...


^
t
g
p
(t)
n
g[n]
t
g
a
(t)

!
G(e
j!
)
G
p
(j)
G
a
(j)

samp
/2
^
^
^
^
^
^
2012-12-05 Dan Ellis 13
2. Quantization

Course so far has been about


discrete-time i.e. quantization of time

Computer representation of signals also


quantizes level (e.g. 16 bit integer word)

Level quantization introduces an error


between ideal & actual signal noise

Resolution (# bits) affects data size


quantization critical for compression

smallest data coarsest quantization


2012-12-05 Dan Ellis 14
Quantization

Quantization is performed in A-to-D:

Quantization has simple transfer curve:


Quantized signal

x = Q x
{ }
Quantization error

e = x x
t
x
n
x
T

A/D
-5 -4 -3 -2 -1 0 1 2 3 4 5
-5
-4
-3
-2
-1
0
1
2
3
4
5
x
Q{x}
e = x - Q{x}
2012-12-05 Dan Ellis 15
Quantization noise

Can usually model quantization as


additive white noise:
i.e. uncorrelated
with self or signal x
+ x[n] x[n]
^
e[n]
-
bits cut off by
quantization;
hard amplitude limit
2012-12-05 Dan Ellis 16
Quantization SNR

Common measure of noise is


Signal-to-Noise ratio (SNR) in dB:

When |x| >> 1 LSB, quantization noise


has ~ uniform distribution:

SNR =10 log
10

x
2

e
2
dB
signal power
noise power
(quantizer step = ")

e
2
=

2
12
-
/
2
+
/
2
P(e[n])
2012-12-05 Dan Ellis 17
Quantization SNR

Now,
x
2
is limited by dynamic range of
converter (to avoid clipping)

e.g. b+1 bit resolution (including sign)


output levels vary -2
b
" .. (2
b
-1)"
! ! ! ! where full-scale range
i.e. ~ 6 dB
SNR per bit
depends
on signal

R
FS
2
...
R
FS
2
R
FS
2
b1
SNR = 10 log
10

2
x

R
2
FS
2
2(b+1)
12

6b + 16.8 20 log
10
R
FS

x
2012-12-05 Dan Ellis 18
Coefficient Quantization

Quantization affects not just signal


but filter constants too

.. depending on implementation

.. may have different resolution

Some coefficients
are very sensitive
to small changes

e.g. poles near


unit circle
high-Q pole
becomes
unstable
z-plane
2012-12-05 Dan Ellis 19
Project Presentations
10:10 Will Silberman & Anton Mayer
10:20 Yin Cui
10:30 Joe Ellis, Miles Sherman, Amit Sarkar
10:40 Andrea Zampieri & Francesco Vicario
10:50 Binyan Chen & Chen-Hsin Wang
11:00 Michael Carapezza & Ugne Klibaite
11:10 Kapil Wattamwar, Ansh Johri, Nida Dangra
11:20 Nayeon Kim & Ban Wang
2012-12-12 Dan Ellis 1
ELEN E4810: Digital Signal Processing
Review Session
1. Filter design
2. Allpass & Minimum phase
3. IIR filter design
4. FIR filter design
5. Implementations
6. FFT
2012-12-12 Dan Ellis
Filter Design

Filters select frequency regions

Performance Margins
2
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 P
-40
-30
-20
-10
0
frequencyW
g
a
i
n

/

d
B
passband
edge
frequency
Passband
Stopband
passband
ripple
minimum
stopband
attenuation
optimal
filter
will touch
here
stopband
edge
frequency
t
r
a
n
s
i
t
i
o
n

b
a
n
d
\G(e
jW
)\
2012-12-12 Dan Ellis
Allpass Filters

Constant gain, variable phase

Mirror-image polynomial

Reciprocal
poles/zeros
3

A
M
z
( )
=
d
M
+ d
M1
z
1
+ ... + d
1
z
M1
( )
+ z
M
1+ d
1
z
1
+ ... + d
M1
z
M1
( )
+ d
M
z
M
= z
M
D
M
z
1
( )
D
M
z
( )
3 2 1 0 1 2
1
0
1
Re{z}
I
m
{
z
}
2012-12-12 Dan Ellis
Minimum & Maximum Phase

Min. phase + Allpass = Max. phase


4
o
o
o
o
o
o

z
( )
z
*
( )
z

z
1

( )
z
1

*
( )
z
( )
z
*
( )

z
1

( )
z
1

*
( )
z
x =
pole-
zero
cancln
2012-12-12 Dan Ellis
Analog Filter Types

{flat | ripple} x {passband | stopband}


5
0 0.5 1 1.5 2
-60
-50
-40
-30
-20
-10
0
0 0.5 1 1.5 2
-60
-50
-40
-30
-20
-10
0
0 0.5 1 1.5 2
-60
-50
-40
-30
-20
-10
0
0 0.5 1 1.5 2
-60
-50
-40
-30
-20
-10
0

c
g
a
i
n

/

d
B

g
a
i
n

/

d
B

g
a
i
n

/

d
B

g
a
i
n

/

d
B
Butterworth Chebyshev I
Chebyshev II Elliptical

p
r
r
A A

p
s-plane
Re{s}
Im{s}

c






2N
= 1
-1 -0.5 0 0.5 1
-1
-0.5
0
0.5
1
Re{s}
I
m
{
s
}
2012-12-12 Dan Ellis
Bilinear Transform

Analog IIR to Discrete-time IIR

Pre-warp to design
6
s-plane
Re{s}
Im{s}
j
z-plane
Re{z}
Im{z}
1
e
j!
LHHPUCI
Imu.c.
s =
1 z
1
1 +z
1
=
z 1
z +1
0 1 2 3 4 5 6
-60
-40
-20
0
-60 -40 -20 0
0
0.2
0.4
0.6
0.8

|H
a
(j)|
|G(e
j
)|
= 2tan
1
()


G e
j
( )
= H
a
j
( )
=2 tan
1

2012-12-12 Dan Ellis


FIR Filters

Linear Phase FIR filters, e.g. h[n] = h[N-n]

Windowed ideal responses


7
W
P P
W
P P
W
P P

H
d
(e
j!
) DTFT{w
R
[n]}
H
t
(e
j!
)
1
h[n]
n
!
H(!)
~
ZP
D
(!)

H e
j
( )
= h n
[ ]
e
jn
n=0
N

= e
j
N
2
h
N
2
[ ]
+ 2 h
N
2
n
[ ]
cosn
n=1
N / 2

( )
linear phase
D = -(!)/! = N/2
2012-12-12 Dan Ellis
Parks-McClellan FIR design

Gradient descent to find best FIR filter

At least M+2 alternating extrema


(order = 2M, length = order + 1)
>> h=firpm(10,[0 0.4 0.6 1],
[1 1 0 0], [1 2]);
8
-5 0 5
-0.2
0
0.2
0.4
0.6
0 0.5 1
0
0.5
1
-1 0 1 2
-1
-0.5
0
0.5
1
h[n]
H(W)
n
W
~
z-plane
2012-12-12 Dan Ellis
Implementations

Polynomial indicates implementation

Decompose into common blocks


(second order sections)
9
+
-2
2
+
z
-1
2
2
+ +
z
-1
+
p
0
-
1
+
z
-1

1
x y

2
2
+
2
2
(
2
2
+
2
2
)
z
-1
z
-1
2012-12-12 Dan Ellis
FFT
10
x[0]
x[4]
x[2]
x[6]
x[1]
x[5]
x[3]
x[7]
X[0]
X[1]
X[2]
X[3]
X[4]
X[5]
X[6]
X[7]
X
0
[0]
X
00
X
01
X
10
X
11
X
0
[1]
X
0
[2]
X
0
[3]
X
1
[0]
X
1
[1]
X
1
[2]
X
1
[3]
DFT
N
4
DFT
N
4
DFT
N
4
DFT
N
4
W
N
0
W
N
1
W
N
2
W
N
3
W
N
4
W
N
5
W
N
6
W
N
7
W
N/2
3
W
N/2
0
W
N/2
3
W
N/2
0
different from before same as before
N/2 pt DFT of x for even n N/2 pt DFT of x for odd n
X
0
[<k>
N/2
] X
1
[<k>
N/2
]
X[k] =

N1
n=0
x[n] W
nk
N
=

N
2
1
m=0
x[2m] W
mk
N
2
+W
k
N

N
2
1
m=0
x[2m+ 1] W
mk
N
2

You might also like