ECE/MATH 520, Spring 2009

Homework Problems 1
Sept.3,26,
20159:33)
Solutions (version: April
2009,

1.5 Suppose you are shown four cards, laid out in a row. Each card has a letter on one side and
a number on the other. On the visible side of the cards are printed the symbols:
S

Determine which cards you should turn over to decide if the following rule is true or false:
If there is a vowel on one side of the card, then there is an even number on the other side.
Ans.: The cards that you should turn over are 3 and A. The remaining cards are irrelevant
to ascertaining the truth or falsity of the rule. The card with S is irrelevant because S is not
a vowel. The card with 8 is not relevant because the rule does not say that if a card has an
even number on one side, then it has a vowel on the other side.
Turning over the A card directly verifies the rule, while turning over the 3 card verifies the
contraposition.
2.7 Show that for any two vectors x, y 2 Rn , |kxk kyk| kx yk.
2.9
Hint: Write x = (x y) + y, and use the Triangle inequality. Do the same for y.
Ans.: We have kxk = k(x y) + yk kx yk + kyk by the Triangle Inequality. Hence,
kxk kyk kx yk. On the other hand, from the above we have kyk kxk ky xk =
kx yk. Combining the two inequalities, we obtain |kxk kyk| kx yk.
3.6 Let 1 , . . . , n be the eigenvalues of the matrix A 2 Rnn . Show that the eigenvalues of the
matrix I n A are 1
1, . . . , 1
n.
Ans.: Suppose v 1 , . . . , v n are eigenvectors of A corresponding to
Then, for each i = 1, . . . , n, we have
(I n
which shows that 1

A)v i = v i
1, . . . , 1

Av i = v i

ivi

= (1

are the eigenvalues of I n

Alternatively, we may write the characteristic polynomial of I n

I n A (1

) = det((1

)I n

(I n

which shows the desired result.

1

A)) = det( [ I n

1, . . . ,

n,

respectively.

i )v i

A.
A as
A]) = ( 1)n A ( ),

3.17a Consider the matrix

2
3
0 1 1
6
7
Q = 41 0 15 .
1 1 0

a. Is this matrix positive definite, negative definite, or indefinite?

Ans.: a. The matrix Q is indefinite, since

1 and

= 2.

5.6
5.5 Let x(t) = [et +t3 , t2 , t+1]T , t 2 R, and f (x) = x31 x2 x23 +x1 x2 +x3 , x = [x1 , x2 , x3 ]T 2 R3 .
Find (d/dt)f (x(t)) in terms of t.
Ans.: We have
Df (x) = [3x21 x2 x23 + x2 , x31 x23 + x1 , 2x31 x2 x3 + 1]
and

By the chain rule,

2
3
et + 3t2
dx
6
7
(t) = 4 2t 5 .
dt
1

d
f (x(t))
dt
= Df (x(t))

dx
(t)
dt

2
3
et + 3t2
6
7
= [3x1 (t)2 x2 (t)x3 (t)2 + x2 (t), x1 (t)3 x3 (t)2 + x1 (t), 2x1 (t)3 x2 (t)x3 (t) + 1] 4 2t 5
1
= (3x1 (t)2 x2 (t)x3 (t)2 + x2 (t))(et + 3t2 ) + (x1 (t)3 x3 (t)2 + x1 (t))(2t)
+ 2x1 (t)3 x2 (t)x3 (t) + 1
= 12t(et + 3t2 )3 + 2tet + 6t2 + 2t + 1.
(Actually, youre not expected or required to expand out and simplify to the last formula.)

5.9
5.8 Let
f1 (x1 , x2 ) = x21

x22 ;

f2 (x1 , x2 ) = 2x1 x2 .
Sketch the level sets associated with f1 (x1 , x2 ) = 12 and f2 (x1 , x2 ) = 16 on the same
diagram. Indicate on the diagram the values of x = [x1 , x2 ]T for which f (x) =
[f1 (x1 , x2 ), f2 (x1 , x2 )]T = [12, 16]T .
2

Ans.: We have that

{x : f1 (x) = 12} = {x : x21

x22 = 12},

and
{x : f2 (x) = 16} = {x : x2 = 8/x1 }.
To find the intersection points, we substitute x2 = 8/x1 into x21 x22 = 12 to get x41
12x21 64 = 0. Solving gives x21 = 16, 4. Clearly, the only two possibilities for x1 are
x1 = +4, 4, from which we obtain x2 = +2, 2. Hence, the intersection points are located
at [4, 2]T and [ 4, 2]T .
The level sets associated with f1 (x1 , x2 ) = 12 and f2 (x1 , x2 ) = 16 are shown as follows.
x2
f2(x1,x2) = 16
f1(x1,x2) = 12
f1(x1,x2) = 12
3
(4,2)

2
1
1

12

2 3

12

x1

(4,2)

f2(x1,x2) = 16

5.9 Write down the Taylor series expansion of the following functions about the given points x0 .
5.10
Neglect terms of order three or higher.
x2

a. f (x) = x1 e

+ x2 + 1, x0 = [1, 0]T

b. f (x) = x41 + 2x21 x22 + x42 , x0 = [1, 1]T

c. f (x) = ex1

x2

+ ex1 +x2 + x1 + x2 + 1, x0 = [1, 0]T .

Ans.: a. We have
f (x) = f (xo ) + Df (xo )(x

1
xo ) + (x
2
3

xo )T D2 f (xo )(x

xo ) + .

We compute
Df (x) = [e
"

D2 f (x) =

x2

, x1 e

x2

0
e
x2
e
x1 e

+ 1],
#

x2

x2

Hence,
"

"
#
x1 1
0
1
f (x) = 2 + [1, 0]
+ [x1 1, x2 ]
2
x2
1
1
= 1 + x1 + x2 x1 x2 + x22 + .
2

#"
#
1 x1 1
+
1
x2

b. We compute
Df (x) = [4x31 + 4x1 x22 , 4x21 x2 + 4x32 ],
"
#
2
2
12x
+
4x
8x
x
1 2
1
2
D2 f (x) =
.
2
8x1 x2
4x1 + 12x22
Expanding f about the point xo yields
"
#
x1 1
1
f (x) = 4 + [8, 8]
+ [x1
2
x2 1
= 8x21 + 8x22

16x1

1, x2

1]

"

#"

16 8
8 16

x1
x2

#
1
+
1

16x2 + 8x1 x2 + 12 + .

c. We compute
Df (x) = [ex1 x2 + ex1 +x2 + 1, ex1 x2 + ex1 +x2 + 1],
#
"
ex1 x2 + ex1 +x2
ex1 x2 + ex1 +x2
2
D f (x) =
.
ex1 x2 + ex1 +x2 ex1 x2 + ex1 +x2
Expanding f about the point xo yields
"
#
x1 1
1
f (x) = 2 + 2e + [2e + 1, 1]
+ [x1
2
x2

"

#"
#
2e 0
x1 1
1, x2 ]
+
0 2e
x2

= 1 + x1 + x2 + e(1 + x21 + x22 ) + .

6.2 Show that, if x is a global minimizer of f over , and x 2 0 , then x is a global
6.3
minimizer of f over 0 .
Ans.: Suppose x is a global minimizer of f over , and x 2 0 . Let x 2 0 . Then,
x 2 and therefore f (x ) f (x). Hence, x is a global minimizer of f over 0 .
4

6.7 Consider the following function f : R2 ! R:

6.8
"
#
" #
1
2
3
f (x) = xT
x + xT
+6
4 7
5
a. Find the gradient and Hessian of f at the point [1, 1]T .
b. Find the directional derivative of f at [1, 1]T in the direction of maximal rate of increase.
c. Find a point that satisfies the FONC (interior case) for f . Does this point satisfy the
SONC (for a minimizer)?
Ans.: a. The gradient and Hessian of f are
"

#
" #
1 3
3
rf (x) = 2
x+
3 7
5
"
#
1 3
F (x) = 2
.
3 7
Hence, rf ([1, 1]T ) = [11, 25]T , and F ([1, 1]T ) is as shown above.
b. The direction of maximal rate of increase is the direction of the gradient. Hence, the
directional derivative with respect to a unit vector in this direction is

(rf (x))T rf (x)

= krf (x)k.
krf (x)k
p
At x = [1, 1]T , we have krf ([1, 1]T )k = 112 + 252 = 27.31.
rf (x)
krf (x)k

rf (x) =

c. The FONC in this case is rf (x) = 0. Solving, we get

" #
3/2
x=
.
1
The point above does not satisfy the SONC because the Hessian is not positive semidefinite
(its determinant is negative).
6.17 An art collector stands at distance of x feet from the wall where a piece of art (picture) of
6.19
height a feet is hung, b feet above his eyes, as shown below.

Picture

Eye

x
Find the distance from the wall for which the angle subtended by the eye to the picture is
maximized.
Hint: (1) Maximizing is equivalent to maximizing tan();
(2) If = 2 1 , then tan() = (tan(2 ) tan(1 ))/(1 + tan(2 ) tan(1 )).
Ans.: Let 1 be the angle from the horizontal to the bottom of the picture, and 2 the angle
from the horizontal to the top of the picture. Then, tan() = (tan(2 ) tan(1 ))/(1 +
tan(2 ) tan(1 )). Now, tan(1 ) = b/x and tan(2 ) = (a + b)/x. Hence, the objective
function that we wish to maximize is
f (x) =

(a + b)/x b/x
a
=
.
2
1 + b(a + b)/x
x + b(a + b)/x

We have

a2
(x + b(a + b)/x)2

f (x) =

b(a + b)
x2

Let x be the optimal distance. Then, x must satisfy the FONC. Now, the point x = 0 does
not satisfy the FONC (why?). Therefore, x must be an interior point of the constraint set
= {x : x 0}. Hence, we have f 0 (x ) = 0, which gives
1

b(a + b)
= 0
(x )2
p
)
x =
b(a + b).

6.29
6.27 Line Fitting. Let [x1 , y1 ]T , . . . , [xn , yn ]T , n 2, be points on the R2 plane (each xi , yi 2 R).
We wish to find the straight line of best fit through these points (best in the sense that
the average squared error is minimized); that is, we wish to find a, b 2 R to minimize
n

f (a, b) =

1X
(axi + b
n i=1

yi )2

a. Let
n

1X
X =
xi
n i=1
n

1X
=
yi
n i=1

1X 2
=
x
n i=1 i

X2

Y2 =

1X 2
y
n i=1 i
n

1X
=
xi yi .
n i=1

XY

Show that f (a, b) can be written in the form z T Qz 2cT z + d, where z = [a, b]T ,
Q = QT 2 R22 , c 2 R2 , and d 2 R, and find expressions for Q, c, and d in terms of
X, Y , X 2 , Y 2 , and XY .
b. Assume that the xi , i = 1, . . . , n, are not all equal. Find the parameters a and b for
the line of best fit in terms of X, Y , X 2 , Y 2 , and XY . Show that the point [a , b ]T is
the only local minimizer of f .
P
Hint: X 2 (X)2 = (1/n) ni=1 (xi X)2 .

c. Show that if a and b are the parameters of the line of best fit, then Y = a X + b (and
hence once we have computed a , we can compute b using the formula b = Y a X).

Ans.: a. We write
n

1X 2 2
f (a, b) =
a xi + b2 + yi2 + 2xi ab 2xi yi a 2yi b
n i=1
!
!
n
n
X
X
1
1
= a2
x 2 + b2 + 2
xi ab
n i=1 i
n i=1
!
!
!
n
n
n
1X
1X
1X 2
2
xi yi a 2
yi b +
y
n i=1
n i=1
n i=1 i
" P
#" #
Pn
n
1
1
2
x
x
a
i
i
i=1
n
= [a b] n1 Pi=1
n
1
b
i=1 xi
n
" n
#
"
#
n
n
a
1X
1X
1X 2
2
xi yi ,
yi
+
y
n i=1
n i=1
n i=1 i
b
= z T Qz

2cT z + d,

where z, Q, c and d are defined in the obvious way.

7

b. If the point z = [a , b ]T is a solution, then by the FONC, we have rf (z ) = 2Qz

P
2c = 0, which means Qz = c. Now, since X 2 (X)2 = n1 ni=1 (xi X)2 , and the xi are
not all equal, then det Q = X 2 (X)2 6= 0. Hence, Q is nonsingular, and hence
"
#"
# 2 XY (X)(Y ) 3
1
X XY
1
X 2 (X)2
5.
z = Q 1c =
= 4 (X 2 )(Y
) (X)(XY )
2
2
2
X X
Y
X
(X)
2
2
X

(X)

Since Q > 0, then by the SOSC, the point z is a strict local minimizer. Since z is the only
point satisfying the FONC, then z is the only local minimizer.
c. We have

a X +b =

XY (X)(Y )
X 2 (X)2

X+

(X 2 )(Y )
X2

(X)(XY )
=Y.
(X)2