Professional Documents
Culture Documents
Analysis
Andrew Tangborn
Global Modeling and Assimilation Oce, Goddard Space Flight Center
Andrew.V.Tangborn@nasa.gov
301-614-6178
Outline
1. Fourier Transforms.
2. What is a Wavelet?
3. Continuous and Discrete Wavelet Transforms
4. Construction of Wavelets through dilation equations.
5. Example - Haar wavelets
6. Daubechies Compactly Supported wavelets.
7. Data compression, ecient representation.
8. Soft Thresholding.
9. Continuous Transform - Morlet Wavelet
10. Applications to approximating error correlations
Fourier Transforms
A good way to understand how wavelets work and why they are useful is by
comparing them with Fourier Transforms.
The Fourier Transform converts a time series into the frequency domain:
Continuous Transform of a function f(x):
f() =
f(x)e
ix
dx
where
f() represents the strength of the function at frequency , where is
continuous.
Discrete Transform of a function f(x):
f(k) =
f(x)e
ikx
dx
where k is a discrete number.
for discrete data f(x
j
), j = 1, ..., N
f
k
=
N
j=1
f
j
e
(i2(k1)(j1)/N)
The Fast Fourier Transform (FFT) is o(NlogN) operations.
Example: Single Frequency Signal
f(t) = sin(2t)
0 1 2 3 4 5 6 7 8 9 10
1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
1
Signal
0 1 2 3 4 5 6 7 8 9 10
0.5
0.4
0.3
0.2
0.1
0
0.1
0.2
0.3
0.4
0.5
Real Coefficient
Imaginary Coefficient
Fourier Transform
Discrete Fourier Transform (DFT) locates the single frequency and re-
ection.
Complex expansion is not exactly sine series - causes some spread.
Two Frequency Signal
f(t) = sin(2t) + 2 sin(4t)
0 1 2 3 4 5 6 7 8 9 10
3
2
1
0
1
2
3
Signal
0 1 2 3 4 5 6 7 8 9 10
1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
1
Real Coefficients
Imaginary Coefficients
Fourier Transform
Both signal frequencies are represented by the Fourier Coecients.
Intermittent Signal
f(t) = sin(2t) + 2 sin(4t) when .37 < t < .55
f(t) = sin(2t) otherwise
0 1 2 3 4 5 6 7 8 9 10
3
2
1
0
1
2
3
Signal
0 1 2 3 4 5 6 7 8 9 10
0.5
0.4
0.3
0.2
0.1
0
0.1
0.2
0.3
0.4
0.5
Real coefficients
Imaginary Coefficients
Fourier Transform
First frequency is found, but higher intermittent frequency appears as
many frequencies, and is not clearly identied.
What is a Wavelet?
A function that is localized in time and frequency, generally with a zero
mean.
It is also a tool for decomposing a signal by location and frequency.
Consider the Fourier transform:
A signal is only decomposed into its frequency components.
No information is extracted about location and time.
What happens when applying a Fourier transform to a signal that has
a time varying frequency?
0 2 4 6 8 10 12 14 16 18 20
1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
1
The Fourier transform will only give some information on
which frequencies are present, but will give no
information on when they occur.
Schematic Representation of Decomposition
Original Signal
Time >
F
r
e
q
u
e
n
c
y
>
The signal is represented by an amplitude that is changing in time.
There is no explicit information on frequency.
Fourier Transform
Time >
F
r
e
q
u
e
n
c
y
>
The Fourier transform results in a representation that depends only on frequency.
It gives no information on time.
Wavelet Transform
Time >
F
r
e
q
u
e
n
c
y
>
The wavelet transform contains information on both the time location and fre-
quency of a signal.
Some typical (but not required) properties of wavelets
Orthogonality - Both wavelet transform matrix and wavelet functions can be
orthogonal.
Useful for creating basis functions for computation.
Zero Mean (admissibility condition) - Forces wavelet functions to wiggle (oscillate
between positive and negative).
Compact Support - Ecient at representing localized data and functions.
How are Wavelets Dened?
Families of basis functions that are based on
(1) dilations:
(x) (2x)
(2) translations:
(x) (x + 1)
of a given general mother wavelet (x).
How do we use (x)?
The general form is:
j
k
(x) = 2
j/2
(2
j
x k)
where
j: dilation index
k: translation index
2
k/2
needed for normalization
How do we get (x)?
Dilation Equations
Construction of Wavelets
We consider here only orthogonally/compactly supported
wavelets
- Orthogonality means:
j
k
(x)
j
(x)dx =
kk
jj
(x)dx = 1
k
c
k
(2x k)dx = 1
k
c
k
(2x k)dx = 1
k
c
k
1
2
(2x k)d(2x k) = 1
k
c
k
1
2
()d() = 1
k
c
k
(
1
2
)(1) = 1
or
k
c
k
= 2
Simple Examples
- Smallest number of c
k
is 1: just get (x) = , zero support.
- Haar Scaling Function: c
0
= 1, c
1
= 1.
Haar Scaling Function
The scaling function equation is:
(x) = (2x) + (2x 1)
The only function that satises this is:
(x) = 1 if 0 x 1
(x) = 0 otherwise
1 0.5 0 0.5 1 1.5 2
0.5
0
0.5
1
1.5
x
y
Translation and Dilation of (x):
(2x) = 1 if 0 x 1/2
(2x) = 0 elsewhere
(2x 1) = 1 if 1/2 x 1
(2x 1) = 0 elsewhere
so the sum of the two functions is then:
1 0.5 0 0.5 1 1.5 2
0.5
0
0.5
1
1.5
x
y
Haar Wavelet
Wavelets are constructed by taking dierences of scaling
functions
(x) =
k
(1)
k
c
1k
(2x k)
dierencing is caused by the (1)
k
:
so the basic Haar wavelet is:
1 0.5 0 0.5 1 1.5 2
1.5
1
0.5
0
0.5
1
1.5
and the family comes from dilating and translating:
j
k
(x) = 2
j/2
(2
j
x k)
so that the j = 1 wavelets are:
1 0.5 0 0.5 1 1.5 2
1.5
1
0.5
0
0.5
1
1.5
Orthogonality of Haar wavelets
Translation no overlap
1 0.5 0 0.5 1 1.5 2
1.5
1
0.5
0
0.5
1
1.5
Dilation cancellation
1 0.5 0 0.5 1 1.5 2
1.5
1
0.5
0
0.5
1
1.5
Daubechies Compactly Supported Wavelets
The following plots are wavelets created using larger numbers of lter coecients, but having all
the properties of orthogonal wavelets. For example, D
4
has coecients:
c
k
=
1
4
(1 +
3),
1
4
(3 +
3),
1
4
(3
3),
1
4
(1
3)
(Reference: Daubechies, 1988)
0 20 40 60 80 100 120
0.4
0.3
0.2
0.1
0
0.1
0.2
0.3
0.4
0.5
D
4
0 20 40 60 80 100 120
0.4
0.3
0.2
0.1
0
0.1
0.2
0.3
D
12
0 20 40 60 80 100 120
0.4
0.3
0.2
0.1
0
0.1
0.2
0.3
D
20
Continuous and Discrete Wavelet Transforms
Continuous
(T
wav
f)(a, b) = |a|
1/2
_
dtf(t)(
t 1
b
)
where
a: translation parameter
b: dilation parameter
Discrete
T
wav
m,n
(f) = a
m/2
o
_
dtf(t)(a
m
o
t nb
o
)
m: dilation parameter
n: translation parameter
a
o
,b
o
depend on the wavelet used
Fast Wavelet Transform
(Reference: S. Mallat, 1989)
Uses the discrete data:
_
f
0
f
1
f
2
f
3
f
4
f
5
f
6
f
7
_
Pyramid Algorithm o(N) !! - Start at nest scale and calculate dierences
j=0
j=1
j=2
f
1
f
2
f
3
f
4
f
5
f
6
f
7
f
8
a
0,0
b
0,0
a
1,0 a
1,1
b
1,0
b
1,1
a
2,0
a
2,1
a
2,2
a
2,3
b
2,0
b
2,1
b
2,2
b
2,3
and averages
- Use Averages at next coarser scale to get new set of dierences (b
j,k
) and averages
(a
j,k
)
And the coecients are simply the dierences (b
j,k
) and the average for the coarsest
scale (a
0,0
):
_
a
0,0
b
0,0
b
1,0
b
1,1
b
2,0
b
2,1
b
2,2
b
2,3
_
For Haar Wavelet:
a
j1,k
= c
0
a
j,k
+ c
1
a
j,k+1
b
j1,k
= c
0
a
j,k
c
1
a
j,k+1
- The dierences are the coecients at each scale. Averages used for next scale.
How do we store all this? (eg, Numerical Recipes algorithm).
Intermittent Signal - Wavelet Transform
0 1 2 3 4 5 6 7 8 9 10
3
2
1
0
1
2
3
Signal
0 20 40 60 80 100 120
5
4
3
2
1
0
1
2
3
4
j=2
j=3
j=4
j=5
Higher Frequency intermittent signal shows up in the j = 5, 6 scales, and only in the
middle portion of the domain. Localized frequency data is found.
Data Compression - Ecient Representation
0 1 2 3 4 5 6 7 8 9 10
2
1.5
1
0.5
0
0.5
1
1.5
2
Original Localized Data, 128 points
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
2
1.5
1
0.5
0
0.5
1
1.5
2
16 term Wavelet Reconstruction
0 1 2 3 4 5 6 7 8 9 10
1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
1
16 term Fourier Reconstruction
Localized function is represented with more accuracy using 16 wavelet coecients
because only signicant coecients need to be retained. Fourier coecients are global.
Denoising by Soft Thresholding
Basic Idea:
Wavelet Representation highly compresses coherent data into just
a few coecients.
Magnitude of each coecient is relatively large.
White noise contains energy at all time scales and time locations.
Representation is spread to many (if not all) wavelet coecients.
Noise coecients are relatively small in amplitude.
How do we remove the noise contribution in the coecients?
Ans: Shrink all of them just a little.
How much do we shrink each coecient?
t =
_
2log(n)/
n
where n= number of data points and =noise standard deviation.
Example
0 50 100 150 200 250 300
1.5
1
0.5
0
0.5
1
1.5
2
2.5
Noisy Signal with white noise of known variance.
0 50 100 150 200 250 300
1.5
1
0.5
0
0.5
1
1.5
2
Wavelet coecients Shrunk toward zero.
Soft Thresholding takes advantage of the fact that white noise is rep-
resented equally by all coecients.
Data compression means that coherent signal is represented by just a
few coecients with relatively large values. White noise (or non-white
as well) is spread over many coecients, adding just a small amount to
the magnitude of each.
The Continuous Morlet Wavelet Transform
The Morlet mother wavelet is a complex exponential (Fourier) with a
Gaussian envelop which ensures localization:
(t) = exp(i
0
t)exp(t
2
/2
2
)
where
0
is the frequency and is a measure of the spread or support.
Note that while the footprint is innite, the exponential decay creates
an eective footprint which is relatively compact.
Translations and dilations of the Morlet wavelet:
(
b, a)(t) =
1
a
exp
_
i
0
_
t b
a
__
exp
_
_
_
t b
a
_
2
/2
2
_
_
The Morlet wavelet using matlab
In matlab, the Morlet mother wavelet can be constructed using the
command:
[psi,x] = Morlet(-8,8,128);
on 128 grid points, and domain of [-8,8].
8 6 4 2 0 2 4 6 8
1
0.8
0.6
0.4
0.2
0
0.2
0.4
0.6
0.8
1
x