EGR653

Digital and Adaptive Systems

Active Learning Exercise #1 The LS Optimal Filter: The System Identification Problem
The System Identification Problem
Consider Figure 1, where an input signal x[k ] drives an unknown system to produce an output d [k ] . We intend to use these input and output signals to derive an optimal filter that models the unknown system. The difference between the true filter output (which may be corrupted by an additional noise source as shown in the figure) and the estimated filter output y[k ] is the error signal e[k ] . We will derive the filter model to minimize the average squared error E e 2 [k ] .

x[k ]

unknown filter

d [k ]

n[k ]

filter estimate

y[k ]

e[k ]

Figure 1: The System Identification Problem

The Normal Equations

The minimization of the MSE in an optimal FIR filtering scenario results in a set of linear equations. These equations, known as the Normal Equations, can be solved to yield the best fit filter estimate in a Least Squares sense. The Normal Equations are expressed as v R = g , where

Rxx [0] Rxx [ 1] R [1] R [0] xx R = xx M Rxx [N ]

L Rxx [ N ] L Rxx [ N + 1] , O L Rxx [0]

T

g = [Rdx [0] Rdx [ 1] L Rdx [ N ]] , v with Rxx [k ] = E {x[n ]x[n + k ]} , Rdx [k ] = E {d [n]x[n + k ]} and is the N + 1 tap parameter vector. The solution of the Normal Equations results in the LS optimal parameter vector estimate v* 1 LS = (R ) g .

-1-

EGR653

Digital and Adaptive Systems

Procedure
1. Create a MATLAB script to simulate the system identification problem (at your option, you may use Simulink). Choose some suitable FIR filter and filter order as v your true system response, denoted as true . For input x[k ] , create a (very) long vector of AWGN generated with the randn command (seed based on the current v clock). Next, generate the desired signal d [k ] using your true filter true and x[k ] via the filter command. At this stage, do not add any noise to d [k ] . Construct the Normal Equations by estimating Rxx [k ] and Rdx [k ] using the xcorr command. Solve v* the Normal Equations to arrive at your optimal filter estimate LS . Compute the normalized parameter error as (use the MATLAB command norm, which computes the 2-norm): v v* true LS . v

true

Furthermore, compute the error signal e[k ] . In this situation, with no added noise, the normalized parameter error and the variance of the error signal e[k ] should both be very small. If this is the case, your equations are correctly configured and you may move to the next step. 2. We now consider algorithm performance under conditions of noise interference. Assume that AWGN n[k ] is now added to the desired signal d [k ] , where the SNR is defined as SNR =

x2 . n2

2 2 Above, x is the variance of the input signal x[k ] and n is the variance of the

additive noise n[k ] . Generate a plot of normalized parameter error for a range of 10dB SNR 10dB . On the same graph, plot the variance of the error signal e[k ] for each SNR. 3. Discuss the ability of the Normal Equations to predict the LS optimal filter coefficients as the SNR varies. Furthermore, how does the variance of the error signal change with SNR? Is this as expected?

Exercises
1. Clearly describe the equivalence of solving an overdetermined set of equations using the pseudo-inverse and the solution of the Normal Equations using sample-based estimates of the autocorrelation functions. 2. Considering the cost function J [k ], show that the input regressor vector is orthogonal to the optimal error, that is, E [k ]e* [k ] = 0 .

-2-

EGR653

Digital and Adaptive Systems

3. The correlation matrix R is Toeplitz, and hence, positive semi-definite (all non~ ~ negative eigenvalues). Show that T R 0 . 4. In the simulation, we used AWGN as the input signal x[k ] . This spectrally flat wideband signal is an excellent choice for system identification. In the derivation of optimal filters, it is required that the input signal contain sufficient spectral excitation or else the resulting derived filter will be suboptimal. Discuss why you think an input with sufficient spectral excitation is necessary. Contrast your discussion with why you think using a sinusoid as the input signal is suboptimal.

Turn in (due at start of class next week):

A printout of your m-file script, well commented and neatly formatted that implements the entire testing range for Procedure step (2). Include a comment block to identify your name, script name, date and course, instructor, description, etc. Submit this script to Blackboard prior to class. A hardcopy of your plot from Procedure step (2). Discussion for Procedure step (3). Fully detailed solutions to the Exercise problems (hard copy).

-3-