Professional Documents
Culture Documents
Walter Sosa-Escudero
Walter Sosa-Escudero
Walter Sosa-Escudero
Walter Sosa-Escudero
Walter Sosa-Escudero
Walter Sosa-Escudero
Assumption 1 (linearity):
i = 1, . . . , n
1
y1
2
y2
u=
= .
Y = .
.
.
.
.
K K1
yn n1
X=
x11
x12
..
.
x1n
u1
u2
..
.
un
n1
x21 . . . xK1
x22
xK2
..
..
.
.
xKn nK
Walter Sosa-Escudero
y1
x11 x21 xK1
..
..
. =
x22
.
yn
x1n
xKn
1
.. +
.
K
u1
..
.
un
Y = X + u
This is the linear model in matrix form.
Walter Sosa-Escudero
A a square m m matrix.
If (A) = m |A| =
6 0
If (A) < m |A| = 0
X a n K matrix, with (X) = K (full column rank):
(X) = (X 0 X) = k
This results guarantees the existence of (X 0 X)1 based on
the rank of X.
Walter Sosa-Escudero
Walter Sosa-Escudero
Y1
Y = ...
Yk
E(Y ) = =
E(Y1 )
E(Y2 )
..
.
E(YK )
Walter Sosa-Escudero
V (Y )
E[(Y )(Y )0 ]
E(Y2 2 )2
..
.
E(Yk K )2
V (Y1 )
Cov(Y1 , Y2 )
V (Y2 )
...
..
Cov(Y1 YK )
.
V (YK )
Conditional Expectations
Z
E(Y |X = x) =
y fY |X dy
Walter Sosa-Escudero
i = 1, 2, . . . , n
Walter Sosa-Escudero
Walter Sosa-Escudero
Assumption 3: No Multicollinearity
(X) = K, w.p.1
Rank?
All columns of the realizations of X must be linearly
independent.
Careful: this prohibits exact linear relations between columns
of X.
The model admits non-exact relations and/or non-linear
relations.
Examples.
Walter Sosa-Escudero
i = 1, . . . , n
Walter Sosa-Escudero
Walter Sosa-Escudero
Walter Sosa-Escudero
Summary
Linearity: Y = X + u.
Walter Sosa-Escudero
The Intercept
Consider the case x1i = 1, i = 1, . . . , n
yi = 1 + 2 x2i + + K xKi + ui ,
i = 1, . . . , n
Walter Sosa-Escudero
Interpretations
E(yi |X) = 1 + 2 x2i + + K xKi
Walter Sosa-Escudero
Walter Sosa-Escudero
Walter Sosa-Escudero
Walter Sosa-Escudero
i = 1, . . . , n.
Walter Sosa-Escudero
n
X
0 (y X )
e2i = e0 e = (y X )
i=1
= argmin SSR()
Walter Sosa-Escudero
Result: = (X 0 X)1 X 0 Y
= e0 e = (Y X )
0 (Y X )
SSR()
= Y 0 Y 0 X 0 Y Y 0 X + 0 X 0 X
= Y 0 Y 20 X 0 Y + 0 X 0 X
In the second line, note that 0 X 0 Y is a scalar, and hence it is
that is how we obtain the
trivially equal to its transpose, Y 0 X ,
result in the third line.
SSR can be easily shown to be a strictly convex, differentiable
so first order conditions for a stationary point are
function of ,
sufficient for a global minimum.
Walter Sosa-Escudero
e0 e
=0
Using the derivation rules introduced before:
e0 e
= 2X 0 Y + 2X 0 X = 0
Walter Sosa-Escudero
Walter Sosa-Escudero
Algebraic Properties
Walter Sosa-Escudero
ei = 0
i=1
Walter Sosa-Escudero
Goodness of Fit
First check some easy results, when there is an intercept in the
model:
Yi = Yi
Start with Yi = Yi + ei . Take averages in both sides,
ei = 0 by the previous property.
Pn
i=1 (Yi
P
P
Y )2 = ni=1 (Yi Y )2 + ni=1 e2i
Walter Sosa-Escudero
(Yi Y )2 =
X
X
(Yi Y )2 +
e2i
i=1 (Yi Y )
i=1 (Yi Y )
This is the (centered) coeficient of determination: the proportion
of the total variability explained by the fitted linear model.
Walter Sosa-Escudero
Walter Sosa-Escudero
P 2
P 2
e
Yi
= P 2 = 1 P i2
Yi
Yi
Yi2 = Y 0 Y
(Y + e)0 (Y + e)
Y 0 Y + e0 e + 2Y 0 e
Y 0 Y + e0 e + 20 X 0 e
Y 0 Y + e0 e
=
=
by the orthogonality property.
Walter Sosa-Escudero
Estimation of 2
Walter Sosa-Escudero