1

ECO3021S 2013
Weeks 1 and 2
Econometrics
Wooldridge, Appendix B and C
Fundamentals o !ro"a"ilit#
$
%at&ematical Statistics
'at&erine E#al
2
C&anges
(nlike pre)ious #ears, *e *on+t "e doing t&e la*
o iterated expectations, or t&e proo t&ereo, t&us
t&ese slides taken out o t&e originals posted on
)ula
3
&ttp,--***.lickr.com-p&otos-doug/////-
4
Some 0uestions o t&e Week
W&at is an estimator1
2o* do *e measure "ias in an estimator1
W&at is t&e dierence "et*een a cd and a pd1
3oes sample si4e matter or an estimator1
W&at is a consistent estimator1 ?
W&at are t&e ormulae or E567, Co)56,87, 9ar5671

5
What is an experiment?
:t can "e repeated infinitely,
:t &as well defined outcomes,
;&ink coin toss, t&ro* o t&e die, gender o a "a"#
at "irt&, outcomes o a randomised controlled
drug trial etc etc...
Eac& time *e perorm an experiment
5i.e. perorm anot&er trial o t&e experiment7,
*e ma# get a dierent outcome
6
W&at is a <andom 9aria"le1
:t takes on numerical values,
:ts outcome is determined "# an experiment
E.G. We toss a coin ten times and count t&e
num"er o &eads *e o"tain = t&is is our <9 6
6 can take on )alues in >0,1,2,3,?,@,A,B,/,C,10D
Eac& outcome is t&e result o a trial o t&e
experiment
7
But
#ou said numerical )alues, &o* is a coin toss *it&
&eads and tails numeric1
We must assign )alues to eac& o our outcomes,
like 1 or &eads, 0 or tails
Note
Capitals, 6,8, E used to denote random )aria"les
Smalls denote particular outcomes, x F 2, # F ?
<9s can "e discrete or continuous
8
9er# common <9 is Bernoulli
Called a binary <9, 6GBernoulli 5H7
!56 F 17 F H
!56 F 07 F 1 = H
;akes on onl# )alues o 0 and 1.
E.G.
!5an airline customer arri)es7 F 0./0
!5customer doesn+t arri)e7 F 0.2
:s it important to kno* H1 2o* can *e estimate it1
9
Discrete !"
#inite or counta"l# inite num"er o )alues
6 &as k possi"le )alues >x
1
, x
2
, I, x
k
D
Wit& !ro"a"ilit#56Fx
i
7 F p
i
or eac& x
i
,
>p
1
, p
2
, I, p
k
D
NB" Eac& 0 J p
i
J 1, Kp
i
F 1
:s a Bernoulli <9 discrete1
3o : need to tell #ou !56F07 or a Bernoulli <91
10
$robability Density #unction o 6,
5x
L
7 F !56 F x
L
7 F p
L
, L F 1,2, I, k
5x7 is t&e pro"a"ilit# t&at 6 takes on t&e )alue x.
For <9s 6 and 8, *e &a)e
6
and
8
e.g. 6 is t&e num"er o ree t&ro*s made "# a
"asket"all pla#er out o 2 attempts
8ou+re gi)en t&e pd,
507 F 0.2, 517F0.??, 527 F 0.3A
11
12
% continuous ! is a <9 *&ic& takes on an# real
)alue *it& 4ero pro"a"ilit#
i.e. a continuous <9 can take on so man#
possi"le )alues t&at t&e pro"a"ilit# t&at it takes on
an# one is 4ero.
For a continuous <9, 56 F x
i
7 F 0
:s t&at useul1
<at&er talk a"out t&e cd F5x7 F !56JF x
L
7
&umulative Distribution #unction
13
14
;&e C3F means *e can talk a"out a ran'e of
values, and t&e pro"a"ilit# t&at 6 lies in t&em.
!5a JF 6 M "7 F F5x7 F N 5x7 dx


area under t&e cur)e "et*een a and "
;&e cd is t&e integral o t&e pd 5continuous <97
What else do we (now?
0 JF F5x7 JF 1
Area under entire cur)e o F sums to 1.
For 6 a discrete <9,
C3F F !56 JF x
L
7 F Kp
L
FKpd

o)er all xJF x
L
15
E.G.
6 is num"er o ree t&ro*s made "# a "asket"all
pla#er out o 2 attempts. 6 can take on >0,1,2D
Ooing "ack to t&e pd on slide 11,
W&at is !56 PF 171
i.e. *&at is pro"a"ilit# t&at a pla#er makes at least
one ree t&ro*1
16
!roperties o C3Fs
For an# num"er c, !56 P c7 F 1 Q F5c7
F 1 = !56 J c7
For an# num"ers aJ",
!5a J 6 JF "7 F F5"7 = F5a7 F !56 J "7 = !5x J a7
We use cds onl# or continuous <9s,
&ence *e don+t dierentiate "et*een J and JF
17
)oint Distribution
or 6,8, 2 discrete <9s

x,#
5x,#7 F !56Fx, 8F#7
2 <9s are independent i

x,#
5x,#7 F
x
5x7
#
5#7

x
5x7 is called t&e mar'inal pd
18
*ndependence
: 6, 8 are independent, R so are O567 and 2587
6
1
, 6
2
, I, 6
n
discrete <9s,
R
5x
1
, x
2
, I, x
n
7 F !56
1
F x
1
, 6
2
F x
2
, I, 6
n
F x
n
7
F 5x
1
75x
2
7I5x
n
7
iff t&e 6s are independent
:n a seSuence o Bernoulli trials, are t&e coin lips
independent1
19
Binomial 3istri"ution
Oi)en 6 F 8
1
T 8
2
T I T 8
n
W&ere eac& 8
i
&as pro"a"ilit# o success H, and
t&e 8
i
are independent, t&en 6 G"inomial5n,H7,
20
&onditional 3istri"utions
Bayes ule
!58F#U6Fx7 F
8U6
5#Ux7 F
6,8
5x,#7

6
5x7
i.e.
&onditional +,- . /oint -,+
mar'inal -
21
W&at i 6, 8 are independent1 W&at can *e sa#
a"out
8U6
5#Ux7 or
6U8
5xU#7 1
!58F#U6Fx7 F
8U6
5#Ux7
F
6,8
5x,#7

6
5x7
F
6
5x7
8
5#7 F
8
5#7

6
5x7
3oes t&is &old or all t#pes o <9s1
?
22
Expected !alue
E567 F *eig&ted a)erage o)er outcomes o 6
6 a discrete <9
6 continuous <9
W&# is E567 F H *&en 6GBernoulli 5H71
23
Vote
We can get an ans*er or t&e expected )alue
*&ic& is not one o t&e outcomes. :s t&at odd1
24
$roperties of Expected !alues
g567 some unction o 6, 6 a discrete <9
6 a continuous <9
25
!roperties o Expected !alues
6, 8 discrete <9s t&en g56,87 also <9
:
;&en
26

E5c7 F c

For constants a,", E5a6 T "7 F aE567 T "

Expected )alue o a sum is t&e sum o t&e

expected )alues 5i.e. eac& a F 17
: 6 F 8
1
T 8
2
T I T 8
k
, i.e. 6 is BinomialG5n,H7
and eac& 8
i
GBernoulli5H7
27
EWO56,87X F K
k
LF1
K
k
&F1
O5x
&,
#
L
7
6,8
5x
&
,#
L
7
: 6, 8 independent, s&o* E5687 F E567E587
E5687 F K
k
LF1
K
k
&F1
68
6,8
5x
&
,#
L
7
F K
k
LF1
K
k
&F1
68
6
5x
&
7
8
5#
L
7
(by defn of independence)
F K
k
LF1
6
6
5x
&
7 K
k
&F1
8
8
5#
L
7
(take Xs out of sum of Y as they are constants in
that sum)
F E567E587
(by definition)
28
0edian
6 F >Q?, 0, 2, /, 10, 13, 1BD
%edian o 6 F /
;&e middle )alue or odd sets.
Even sets &a)e t*o median )alues, or t&e
a)erage.
6 F >Q@,3,C,1BD
%ed567 F 3 or C, or t&e a)erage o t&e t*o F A
29
If X is continuous, then the median of X, say,
m, is the value such that one-half of the area
under the pdf is to the left of m, and one-
half of the area is to the right of m.
Vote,
%ed567 F E567 *&en 6 is s#mmetric
30
31
Vo* *e start to ask 1uestions like,
2o* ar is 6 rom E567,
or as *e *ill call it, 2, or 2
x
1
W&at does t&e distribution look like1

Fat, skinn# and tall, ske*ed to one side or t&e
ot&er...
We *ant to kno* t&e )ariance o 6
= &o* muc& it )aries around t&e mean
32
33
!ariance 3 4tandard Deviation
a)erage distance o 6 rom its mean 2
Q 9ar567 denoted 5
6
or 5
x
6
Q W&# do *e sSuare t&e distance to t&e mean1
Q 9ar567 F EW56QY7
2
X F E56
2
7 = Y
2
34
!ariance !roperties

9ar567 F 0
i t&ere is a const c, s.t. !56 F c7 F 1,
and i so, t&en E567 F c.

9ar 5a6 T "7 F a

2
9ar567

9ar567 F H51QH7 or 6GBernoulli5H7

35
4tandard Deviation
5 or 5
x
Z F sd567 F T [9ar567
sd5c7 F 0
sd5a6 T "7 F UaUsd567
36
4tandardisin' a <9
6G<95mean Y, std de) Z 7
S&o* t&at,
E5E7 F 0
9ar5E7 F 1
3emonstrated simpl# ater B.2@, around !age B2/
37
4(ewness and 7urtosis
<ound a"out !age B2C
: expect t&at #ou+re amiliar *it& t&ese concepts,
t&oug& : *ill put t&e rele)ant ormulae on a
ormula s&eet = "ut :+ll expect #ou to kno* *&ic&
ormula does *&at.
38
&ovariance
2o* do 6 and 8 )ar# *it& eac& ot&er1

Q : E567 F 0 or E587F0,
R Co)56,87 F E5687
39
&ovariance
&8!9" 6, 8 independent R Co)56,87 F 0
: pro)ing, remem"er E5687 F E567E587 i 6,8
independent
NB"
Co)56,87 F 0 does VO; R t&at 6,8 independent
E.g. *it& )aria"les 6 and 8 F 6
2
, t&e# &a)e
Co)56,87 F 0 5C&eck7 "ut are clearl# related.
40
&8!6" For constants a
1
, "
1
, a
2
, "
2
%ake sure #ou can s&o* t&e a"o)e. Note to alter
co)56,87, multipl# 6 "# some constant. Vot good\
&8!:" Cauc&# Sc&*art4 :neSualit#
UCo)56,87U JF Z
x
Z
#
Note" &ov;-,-< . E556QY
x
756QY
x
77 . !ar;-<.
41
So Co)56,87 F measure o linear relationship
NB, Co)56,87 depends on units of measurement
<at&er turn to &orrelation
Corr56,87 F ]
68
F Co)56,87 F Z
68
F ]
x#
sd567sd587 Z
6
Z
8

Co)56,87 $ Corr56,87 &a)e same sign "ecause

Z
8,
Z
8
are al*a#s positi)e.

Corr56,87 F 0 i Co)56,87 F 0
42
&89" Q1 JF Corr56,87 JF 1.
So Corr56,87 F 1 R a perect positi)e linear
relations&ip "et*een 6 and 8, *e can *rite
8 F aT"6
Corr56,67 F Co)56,67-Z
x
2
F - 9ar567-9ar567 F 1
&86"
Correlation is in)ariant to units o measurement
Corr5a
1
6 T "
1
, a
2
8 T "
2
7 F Corr56,87 i a
1
, a
2
P 0
Corr5a
1
6 T "
1
, a
2
8 T "
2
7 F QCorr56,87 i a
1
, a
2
J 0
43
!roperties
Co)56,87 measures linear dependence
Corr56,87 F 1 R 6,8 perfectly linearly related
i.e. 8 F a T "6 or some constants a and "P0
Corr56,87 F 0 R no linear relations&ip
<emem"er t&ere "e some ot&er t#pe o nonQlinear
relations&ip.
44
9ariance o Sums
!%:"
9ar5a6 T "87 F a
2
9ar567 T "
2
9ar587 T 2a"Co)56,87
: Co)56,87 F 0, t&en,
9ar56 T 87 F 9ar567 T 9ar587
9ar56Q87 F 9ar567 T 9ar587
45
9ariance o Sums
!%=" $airwise uncorrelated variables
Oi)en >6
1
, 6
2
, ..., 6
n
D, *it& Co)56
i
, 6
L
7 F 0 or all i, L
R
9ar5a
1
6
1
T...T a
n
6
n
7 F a
1
2
9ar56
1
7 T...T a
n
2
9ar56
n
7
;&e variance of the sum is eSual to t&e sum of
the variances i all t&e a
i
F 1.
46
For 6 G Binomial5n,H7,
i.e. 6 F 81 T 82 T I T 8n,
and eac& 8 is an independent Bernoulli5H7 <9.

W&at is 9ar567?
8ou s&ould "e a"le to calculate t&e a"o)e. ;&e
onl# missing piece is t&at 9ar58
i
7 F HQH
2
or eac& 8
i
47
Conditional Expectation
EW8U6FxX
e.g. W&at is expected income, or t&ose *it&
education F 12 #ears1
We still do a *eig&ted sum, "ut the wei'hts are
now different = t&e# are t&e conditional pd.
We can s&o* EW8U6X grap&icall# or in a ta"le.
48
Conditional Expectation
<at&er use a unction to descri"e it or eac& le)el
o education.
E.g. E5WAOEUE3(C7 F 1.0@ T .?@E3(C
Conditional expectations can take on an# sort o
unctional orm.
49
50
51
!roperties o CE
&E9"
EWc567U6X F c567
i.e. i *e kno* 6, *e also kno* c567. Functions o
6 "e&a)e as constants *&en *e condition on 6.
&E6"
EWa5678 T "567U6X F a567EW8U6X T "567
&E:"
6,8 are independent R EW8U6X F EW8X
E.g. EW68 T 26
2
U6X F 6EW8U6X T 26
2
.
52
: (,6 independent, $ E5(7 F 0,
E5(U67 F E5(7 F 0 5"# deinition o independence7
J^ea)e out CE.?QCE.AP
Conditional 9ariance
9ar58U6Fx7 F EW8
2
U6X = 5EW8U6X7
2
E.g. 9ar5Sa)ingsU:ncome7 F ?00T0.2@:ncome
&!9"
: 6 and 8 are independent,
R
9ar58U67 F 9ar587
53
Vormal and ot&er distri"utions...
We rel# &ea)il# on normal>Gaussian distri"utions
Q to simplify pro"a"ilit# calculations, "#
assuming our 6s are normall# distri"uted
Q to conduct inference
6GVormal5Y,Z
2
7

Will "e gi)en on a ormula s&eet.
54
55
;&e normal distri"ution is s#mmetric,
so median567 F E567
Birt& *eig&ts, test scores, unemplo#ment rates
tend to ollo* a normal distri"ution.
:ncome, price )aria"les tend not to.
We can transorm )aria"les to "e normalised, e.g.
"# using log567 instead o 6. ;&en 6 is lo'
normal.
?
56
Standard Vormal EGVormal50,17
2as pd,

C3F denoted _547 = ?;@< . AB;@<. $;CD@<
_547 = &a)e ta"le o )alues, no ormula = W&#1
_547 = ta"le O1 in text
W&at is _547 or 4 JF Q3.1 or 4 PF 3.11

57
58
!roperties
?;@< . AB;@<. $;CD@<
!5E P 47 F 1 = _547
!5EJ Q47 F !5EP47
!5aJEJ"7 F _5"7 = _5a7
!5UEUPc7 F !5EJQc7 T !5EPc7
F 2!5EPc7
F 2W1 = _547X
59
60
$N9"
: 6GVormal5Y,Z
2
7, t&en 56QY7-ZG Vormal50,17
So i 6GVormal53,?7, *&at is !56JF 171
!56JF 17
F !556 = 37-2JF 51Q37-27
F !5E JF Q17 F 0.1@/B
F !5EPF 17
F 1 = !5EJF 17
F 1 = 0./?13 F 0.1@/B
^ook at ex B.A on ! B?0.
61
$N6" 6 G Vormal5Y,Z
2
7
R a6 T " G Vormal5aY T ",a
2
Z
2
7.
$N:" Oi)en 6 and 8 Lointl# normall# distri"uted,
6,8 independent i Co)56,8 7 F 0.
$N=" An# linear com"ination o independent,
identicall# distri"uted normal <9s GVormal
NB" 8
1
, 8
2
, ..., 8
n
independent <9s,
Eac& 8
i
G Vormal5Y,Z
2
7,
R
8 "ar F 58
1
T 8
2
T... T 8
n
7-n G Vormal5Y,Z
2
-n7.
W&#1
62
: 6GVormal51,C7,
*&at is t&e distri"ution o 8 F 26 T 31 !B?0.
63
Ot&er 3istri"utions
&hi 41uared"
^et E
i
, i F 1,2,...n,

"e independent <9s

We *rite 6GC&i SSuared5n7 or 6G6
2
n
C&i SSuared al*a#s nonQnegati)e
Vot s#mmetric
Given, E567 F n, 9ar567 F 2n
64
65
t distri"ution
^et EGVormal50,17, 6G6
2
n
, E, 6 independent
;Gt
n
; is similar to normal distn, "ut more spread out,
as t gets larger, it approac&es t&e std normal.
Given, E5;7 F 0 or nP1
9ar5;7 F n-5nQ27 or nP2
66
67
# distri"ution
6
1
G6
2
k1
, 6
2
G6
2
k2
, assume 6
1
, 6
2
independent
FGF
k1,k2
Order o t&e degrees o reedom is important
5numerator is irst7.
Note, 8ou &a)e t&e tools to calc t&e exp )alue
and )ariance o t&e t and c&i sS distri"utions
68
69
Appendix C, !opulations,
!arameters $ <andom Sampling
W&at+s a population1 And learnin'1 Statistical
inference1 4ample1
Why onl# a sample1
Eow could #ou take samples1 W&at+s a random
sample1
W&at are point and interval estimates1
3o t&e# come out t&e same no matter *&ic&
sample is dra*n1
3oes A3; presence reduce crime1 W&at *ould
#our hypotheses "e1 2o* s&-*ould #ou test
t&em1 2o* could #ou test t&em 5i.e. &o*
s&ouldn+t #ou17
70
Appendix C, !opulations,
!arameters $ <andom Sampling
Statistical Inference ! learning something about
a population given the availability of a sample
from the population
$opulation F *ell deined group o su"Lects
Fearnin' F estimation $ &#pot&esis testing
Eg, return to education, program e)aluation
71
Statistical :nerence
1. :denti# population o interest
2. Speci# a model or relations&ip o interest
3. %odels in)ol)e probability distributions
*&ic& depend on unkno*n parameters o t&e
model
!arameters determine direction $ stren'th o
relations&ips = e.g. return to education, eect o
neig&"our&ood *atc& programs on crime
72
Sampling

8 = random )aria"le representing a population, *it&

probability density function 5#,H7 H is unkno*n

!ro"58
1
F #7 F !ro"58
2
F #7 F I F !ro"58
n
F #7 F 5#,H7,
8
1
, 8
2
, I, 8
n
are independent random )aria"les *it&
common pd 5#,H7 t&en >8
1
, 8
2
, I, 8
n
D is a random
sample rom 5#,H7
E.O. Famil# income rom nF100 amilies
>B20000,2/000,@?0000,1.@m, 20m, @000, I, 1000D
73
Sampling

;&is stu is trick#. :.e.,

Q Bernoulli )aria"le = does passenger i arri)e
or t&eir lig&t1 !ro"a"ilit# t&e# do is Bernoulli
distri"ution.

!58
i
F 17 F H, !58
i
F 07 F 1QH

!assengers independent, t&ereore 8

1
, 8
2
etc
are independent, identically distributed
5i.i.d7, and >8
1
, 8
2
, I, 8
n
D are a random sample
rom densit# 5#, H7, *&ere &ere 5#,H7 is a
Bernoulli distn, and popn is all passengers
74
Eac& random sample not necessaril# t&e same
5t&e luck o t&e dra*17
:s it appropriate to assume random samplin'1
E.g. rom a (C; class, or gender 5ma#"e7, race,
academic a"ilit#, income 5pro"a"l# not71 And rom
all matric graduates1
F can "e any prob distn = normal, Bernoulli,
Binomial, t, F, 6
2

75
Finite Sample !roperties o
Estimators
#inite F t&ese sample properties &old or an# si4e
o sample, "ig or small 5"ut *e don+t let n R `7.
Oi)en a random sample >8
1
, 8
2
, I, 8
n
D,
dra*n rom a population t&at depends on H,
an estimator o H is a rule *&ic& assigns eac&
possi"le outcome o t&e sample a )alue o H
e.g. W F &58
1
, 8
2
, I, 8
n
7
;&e rule s&ouldn+t c&ange *it& dierent samples.
76
Example,
>8
1
, 8
2
, I, 8
n
D a random sample *it& mean Y
An estimator o Y could "e,
3enoted t&e sample avera'e G + bar
77
78
An estimator W o a parameter H can "e
expressed, W F &58
1
, 8
2
, I, 8
n
7
h is Lust an# old unction.
W is a <9, as it+s a unction o <)s.
For * t&e sample a)erage, e.g. * F 0.2, t&is is a
point estimate
2o* do *e c&oose between estimators1
W+s samplin' distribution tells us t&e likeli&ood
o certain e)ents &appening = it is important.
79
NB" Eere be dra'ons...
W&en talking a"out E5W7 or 9ar5W7
W&ere W is an estimator, t&is is v different to
*&en W is a <9.
: gi)en W F &58
1
, I, 8
n
7
t&en E5W7 F W5&58
1
, I, 8
n
77
W&ic& is a "it more complicated t&an "eore.
80
(n"iasedness\\\ 5tattoo *ort&#7
W is an unbiased estimator o H i,
E5W7 F H
or all )alues o H
So or a gi)en sample, *ill W F H1 Or close to H1
NB" Hnbiased definition" : *e take infinitely
many samples, and o"tain W in eac& sample,
t&en t&e a)erage o all t&e Ws *ould eSual H
81
: W is a B*%4ED estimator o H, t&en,
Bias5W7 F E5W7 = H \F 0
Q ;&e "ias depends on h, and on t&e pdf of +.
Q For e.g. :s t&e sample avera'e un"iased1
^ooks like no. W&#1 Because E58 "ar7 F Y.
82
83
Sample 9ariance
For random sample >8
1
, 8
2
, I, 8
n
D
Wit& E587 F Y, and 9ar587 F Z
2
3eine t&e sample variance estimator as,
3oes it look like our deinition o )ariance rom
"eore1
84
Sample 9ariance
;&is estimator is unbiased = E5S
2
7 F Z
2
. ;&is
result #ou deinitel# can s&o*=ask me i #ou can+t.
We divide by nI9 and not n "ecause Y is
estimated, and not kno*n. : *e kne* n, *e *ould
calculate t&e a"o)e *it& Y replacing 8 "ar, and
di)iding t&roug& "# n and not nQ1.
85
(n"iased estimator F good estimator ?
%a#"e, "ut )ariance is also important.
+ou can 'et an unbiased estimator which is
pretty bad. E.g. nF 100, (se rule W F 8
1
58
1
is
t&e irst o"ser)ation o t&e sample7 to estimate Y,
and t&us E5W7 F Y 5take enoug& samples,
calculate W, and t&us E5W7 *ill F Y7. So W is
un"iased, "ut a prett# dum" t&ing to do 5t&ro*s
out all ino7.
We care a"out t&e samplin' distribution too.
86
Sampling 9ariance
87
9ariance o t&e Sample A)erage
2o* spread out is t&e estimator1
W&ere did *e use t&e independence o t&e 8
i
s1
;o summarise,
For random sample >8
1
, 8
2
, I, 8
n
D *it& 8
i
G 5Y,Z
2
7,
t&en E58 "ar7 F Y, "ut 9ar58 "ar7 F Z
2
-n
88
So no* *&at1
9ar58 "ar7 F Z
2
-n
:s 9ar58 "ar7 a <91 3o *e al*a#s kno* it1
W&at &appens to 9ar58 "ar7 as n J K1
We preer estimators *it& small variance = more
predicta"le.
9ar58
1
7 F Z
2
5Because it+s a random dra* rom t&e
popn7 and 9ar58 "ar7 F Z
2
-n. W&ic& do *e preer1
89
90
Eicienc#
: W
1
and W
2
are un"iased estimators o H,
W
1
is efficient relati)e to W
2
i
9ar5W
1
7 JF 9ar5W
2
7
For all H
Can *e al*a#s c&oose an estimator using
eicienc#1 Sometimes not, sometimes t&e# &a)e
dierent )ariances or dierent )alues o H.
S&ould *e compare t&e eicienc# o "iased
estimators1 Vope\ See page B@B or e.g.
%SE pB@B
91
As#mptotic !roperties o Estimators
9ar58
1
7 F Z
2
,
9ar58 "ar7 F Z
2
-n
W&at &appens to 8
1
and 8 "ar as n R `1
:s adding more data al*a#s good1
W
n
an estimator o H rom sample >8
1
, 8
2
, I, 8
n
D
o si4e n. W
n
is a consistent estimator o H i
!5 U W
n
= H U P a 7 R 0 as n R `, or e)er# a P 0.
: W
n
is consistent, plim;W
n
< . L
92
93
VB, re B:AS
3o *e actuall# take lots and lots o samples1 ;&at
*ould cost a lot o mone# rig&t1
:t+s a thou'ht experiment = *e don+t actuall# do
it, except ma#"e in a computer simulation = "ut
*e t&ink a"out it to ans*er Suestions a"out what
values t&e estimator *ould gi)e in eac& o t&ose
samples, and whether t&e a)erage o t&ose
)alues *ould eSual t&e population )alue
3o #ou kno* t&e popn )alue1 Vo, "ut *e s&o*
alge"raicall# t&at E5W7 F popn )alue
94
VB, unbiased = &as to do *it& *&at &appens i
*e take lots o samples, calculated W in eac&
sample, and take t&e expected )alue o t&e Ws,
and see i t&e# eSual H
&onsistency = *&at &appens to W
n
as n R `1
;&e eSuation is a s&ocker, "ut t&e intuition s&ould
make sense,
:ntuiti)e explanation, : W
n
is consistent, W
n

"ecomes more and more concentrated around H
5Wn R H7 as n R `
95
Consistenc# is a minimal re1uirement.
i.e. i an estimator doesn+t e)en tend to H as
n R `, t&en it+s not muc& use to us.

For W
n
an un"iased estimator o H, another way
of sayin' that W
n
is consistent, "esides
plim5W
n
7 F H, is to sa# !ar;W
n
< J M as n J K.
96
^et >8
1
, 8
2
, I, 8
n
D "e independent, identicall#
distri"uted random )aria"les *it& mean Y. ;&en,
plim 58
n
"ar7 F Y
;o estimate Y, *e Lust need to pick a large enoug&
sample to get ar"itraril# close to Y
:n englis&, : #ou pick a "ig enoug& random
sample, #our sample a)erage *ill tend to*ards
#our popn a)g.
^a* o ^arge Vum"ers
97
98
!roperties o pro"a"ilit# limits,
Oi)en H a parameter,
deine b F g5H7 or a continuous unction g5H7.
Suppose plim5W
n
7 F H.
3eine an estimator o b "# O
n
F g5W
n
7.
;&en $F*0 9,
plim ';W
n
< . ';plim W
n
< F g5H7 F b
;&e plim 5o a n o W7 F a n 5o t&e plim o W7,
i t&e n is continuous7.
99
!roperties o !ro"a"ilit# ^imits,
$F*0 9,
plim ';W
n
< . ';plim W
n
< F g5H7 F b
$F*06" i plim5;
n
7 F c and plim 5(
n
7 F d, t&en,
;i< plim5;
n
T (
n
7 F c T d
;ii< plim5;
n
(
n
7 F cd
;iii< plim5;
n
-(
n
7 F c-d
We can combine consistent estimators to get
ot&er consistent estimators = see egs on pBA0.
100
We kno* t&e sample variance is un"iased
:t is consistent or Z
2
5s&o*ing t&is is complex "ut
possi"le, "ut not reSuired or t&is course7.
;&e sample standard deviation S F sSrt5S
2
7 is
not un"iased, "ut it is consistent or Z.
101
Not covered in lectures"
As#mptotic Vormalit#
!age BA0 mid*a#, !BA1
Oeneral Approac&es to !arameter Estimation
!BA2, !BA3, &al o !BA?
102
:nter)al Estimation, Conidence
:nter)als
Estimated rate return to education F <20000-#ear
Q &o* close is t&is to population rate o return1
Q t&e standard deviation tells us &o* it is
distri"uted, "ut not&ing a"out *&ere it is in
relation to t&e true popn )alue
We *ant to kno* t&e probability t&at a random
inter)al contains t&e population value.
103
Suppose *e+re gi)en a population G Normal;2, 9<,
5Lust letting )ariance F 1 or eg sake7
let >8
1
, 8
2
, I, 8
n
D "e random sample rom popn
;&en + barN Normal;2, 9>n<
We kno* *e can standardise 8 "ar to Std Vormal
And *e kno*,
!5Q1.CAJEJ1.CA7 F .C@ or EGVormal50,17
W&#1 rom ta"le O
5detailed expl i needed on slide 131-27
104
W&# do *e care t&at *e can standardise 8 "ar1
Because t&en *e can *rite, *it& 8 "ar
standardised,
8ou can transorm t&e a"o)e into t&e "elo*,
i.e. pro"5a random inter)al contains Y7 F 0.C@
105
4o we 'et a OPQ confidence interval
W&ic& sa#s t&at t&e random inter)al,
Contains Y *it& pro"a"ilit# 0.C@
;&at+s cool. W&#1
Vote t&at t&oug& *e talked a"out t&is or t&e
estimator 8 "ar, it &olds or an# normal estimator
*&ic& *e can standardise 5also don+t orget t&at
)ariance o 1 *e must still deal *it&7
106
;&is random inter)al contains Y *it& pro"a"ilit#
0.C@ = 8ES #ou can sa# t&is
;&e pro"a"ilit# t&at Y is in t&e 5calculated7
inter)al is 0.C@
= V0 #ou mustn+t sa# t&is.
Once *e+)e calculated our inter)al rom our
sample, it is Lust num"ers, and Y is eit&er in it or
not in it. B(;,
: *e *ere to take infinitely many samples and
calculate t&e inter)al a"o)e or eac& o t&em,
C@e o t&e inter)als *ill contain Y
107
108
;&is assumes t&e )ariance F 1
%ore generall#, *it& population G Vormal5Y, Z
2
7,
3o *e kno* Z1 We must use an estimate,
4ample standard deviation
2o*e)er no* t&e random inter)al does not
contain Y *it& C@e pro"a"ilit#, "ecause *e
replaced Z *it& s, and s depends on t&e sample.
109
We turn to t&e t distribution, rom t&e std normal
We are gi)en 5no need to pro)e7
Vo* *e *ant
!5Qc J t
nQ1
J c7 F 0.C@
c F CB.@
t&
percentile, s.t. C@e area is "t*n Qc $ c.
We calculate t&e OPQ &.* no* *it& c, not 1.CA,
3on+t kno* c *it& certaint# Q it depends on do nQ1
110
111
We o"tain c rom ta"le O2
e.g. i n F 20, d F nQ1 F 1C, c F 2.0C3,
W# "ar = 2.0C3 s-[20, #T2.0C3 s-[20X
A 10051Qc7 percentile in t
nQ1
is o"tained as,
i.e. a 10051Q0.17 F C0t& percentile CF is,
W# "ar = c
0.0@
s-[n, # T c
0.0@
s-[nX
112
We need to pick c and kno* nQ1 5do7 to pick c.
(suall# *e pick C@t& percentile 5i.e. c F 0.17

;&e a"o)e is mess#. 2elp can "e &ad. We kno*,
4tandard deviation sd 58 "ar7 F Z-[n
But *e don+t kno* Z
So standard error is se5 8 "ar7 F s-[n
113
114
Example C.2 on !age BA/
:s t&e a)erage c&ange F 01
We o"tain a C@e C.:. O WQ2.2/, Q0.2X
;&is inter)al doesn+t include 4ero, so *e can sa#
*it& C@e pro"a"ilit#, t&e a)erage c&ange in scrap
rates in t&e population is not 4ero.
Not covered G asymptotic &*s on $a'e RRM
115
<ule o ;&um" or C@e C.:.
As sample si4e n increases,
t distn R standard normal
Also, c
c-2
R 1.CA as n R `
<ule o ;&um" or C@e C.:,
;&is is &and#, #ou don+t need to use t&e ta"le. We
use 2 "ecause it+s a good enoug& approx to 1.CA.
<ead and understand example C3 !age BB0
116
2#pot&esis ;esting
*ant to ask speciic 1s *it& yes>no answers
3oes Lo" training increase producti)it#1
Are coloured students discriminated against in
admissions1
3o speed cameras reduce accidents1
117
E.g. An election *it& candidates A, B
A recei)es ?2e o )ote, B gets @/e
A is con)inced s&e+s "een c&eated
We sample 100 )oters, get @3e *&o )oted or A
Null Eypothesis, 2
0
, H F .?2
!resume 2
0
is true, until *e pro)e ot&er*ise 5i.e.
need enoug& e)idence to reLect 2
0
7
%lternative Eypothesis, 2
1
, H P .?2
2o* muc& e)idence is enoug& to reLect 2
0
1
118
;#pe : and :: errors
Sype *"
<eLect 2
0
*&en it is actuall# true
Sype **"
Fail to reLect 2
0
*&en it is actuall# alse
W&at is pro"a"ilit# t&at *e make eit&er error1
2#pot&esis tests attempt to minimise t&e
pro"a"ilit# o making a Sype * error = called t&e
si'nificance level I T.
119
T . $;e/ect E
M
, E
M
true<
We *ant T to "e as lo* as possi"le
(sual )alues T F 0.10, 0.0@, 0.01
Oi)en T, minimise pro"a"ilit# o t#pe :: error,

Or maximise power of the test
!o*er F 1Q!5;#pe :: error7
$ower s&ould F 1 i 2
0
is alse
120
2#pot&esis ;ests in a Vormal !opn
We create test Statistic t, decide on T,
and t&en ind c 5rom ta"le7, "ased on distri"ution
o t 5i.e. *&ic& ta"le7, assuming E
M
is true 5p&e*\7
2
0
, Y F Y
0
9 sided test" 2
1
, Y P Y
0
or 2
1
, Y J Y
0
6 sided test" 2
1
, Y f Y
0
*0$8S%NS" We kno* ; F 5# "ar = Y
0
7 G;
nQ1
se5# "ar7
121
<eLection <ules,
9 tailed test"
c F 10051Qc7 percentile in t
nQ1
<eLect 2
0
*&en t P c
6 tailed test"
c F 10051Qc-27 percentile in t
nQ1
<eLect 2
0
*&en tPc or t J Qc
3o *e e)er sa# t&at *e accept 2
0
at @e le)el1
VO\ We sa# *e fail to re/ect.
122
123
124
Examples,
Oi)en a 2 tailed test, c F 0.0@, n F 22,
;&en c F 2.0/ 5rom ta"le O27
<eLect 2
0
i UtU P 2.0/
1 tailed test, c F 0.0@, n F 3A
;&en c F 1.AC
See example C.? !age BB/
!^EASE ensure #ou can read )alues o O2.
125
p )alues
T is t&e probability #ou reLect 2
0
*&en it is true.
8ou pick T, #ou mig&t "e oka# *it& &ig& c
<esearc&er A and B come up *it& dierent
conclusions "ased on dierent cs.
For smaller c, #ou need more e)idence to reLect
2
0
, as c is &ig&er 5c&eck in ta"les7.
Oi)en a t stat )alue 5t7, *&at+s t&e largest T suc&
t&at : still ail to reLect 2
0
1 ;&is is our p value.
126
p F !ro"5o"ser)e ; as large as *e do U 2
0
is true7
1 sided, p value F !5;Pc U 2
0
is true7 F 1Q_5c7
Example,
: p F 0.0A@
At c F @e, *e *ill ail to reLect 2
0
At c F 10e, *e *ill reLect 2
0
Vot )er# exact 5) *icked7, "ut #ou could sa#,
p )alue is t&e pro"a"ilit# t&at #ou gaccepth 2
0
127
4mall p values = strong e)idence against 2
0
,
estimate is signiicant at p le)el o signiicance.
e.g. t F 2./@, p )alue F 1Q _52./@7 F .002
!ro"a"ilit# o seeing a t as large as 2./@ is .002
i.e. )er# unlikel#. We reLect 2
0
.
We assume V is large enoug& to use standard
normal ?;c< and not t&e t distn.
:s it eas# to calculate p )alues1
Vo, "ut Stata can do it easil#.
Ex C.A !BB/, Vice p value summary on pB/1
128
129
C:s and 2#pot&esis ;esting
Ater *e calculate C:, *e can do &#pot&esis tests
;&ese *ill all "e 2 sided, and at c&osen le)el
e.g. C@e C: or Y
2
0
, Y F Y
0
: Y
0
is in t&e C:, we fail to re/ect E
M
: Y
0
is not in t&e C:, we re/ect E
M
VB, man# null &#pot&eses *ill not get reLected
!B/? Example C./, C.C
130
!ractical )s. Statistical Signiicance
2
0
, Y F 0,
4tatistical
&o* "ig is t&e t1 W&en *ill t "e large1
$ractical
sign and magnitude o estimate, context.
A consistent test reLects 2
0
*it& pro"a"ilit# R 1
as n R ` *&en 2
1
is true.
131
Votation
H, a population parameter
H &at 5Hi7, estimate $ estimator o H
H tilde 5HG7, %nother estimate-estimator o H

132
Suppose population G Normal;2, 9<,
let >8
1
, 8
2
, I, 8
n
D "e random sample rom popn
;&en + barN Normal;2, 9>n<
We can standardise 8 "ar to Std Vormal
!5Q1.CAJEJ1.CA7 F .C@ or EGVormal50,17
!5Q1.CAJEJ1.CA7
F 2!50JEJ1.CA7 by symmetry
F 25!5EJ1.CA7 Q !5EJ077 by defn
F 250.CB@ = 0.@7 F 250.?B@7 F 0.C@ using table"#
133
So "asicall# : made #ou start *it& some random
inter)al, 5Q1.CA JF E JF 1.CA7, *it& E std normal.
;&en : asked #ou *&at t&e pro"a"ilit# *as t&at E
*as in t&at inter)al = t&e ans*er *as 0.C@
So t&en *e kne* t&at
!5Q1.CAJEJ1.CA7 F .C@ or EGVormal50,17
So transorm an# <9 into a std normal as ollo*s,