Stat7055 l05 Samp-Dist

STAT7055
LECTURE 5
Sampling Distributions
STAT7055 - Lecture 5
1 / 69
Introduction
Statistical Inference
I
Statistical inference: Determining the behaviour

of a population by studying a sample from that
population.
I
I
I
Population parameters describe populations, e.g.,

population mean and population variance 2 .
Population parameters usually unknown.
To find out more about them, we take a random sample
of data.
Calculate statistics from sample, e.g., sample mean X

2
and sample variance s .
Use X and s2 to estimate and 2 .
2 / 69
Introduction
Statistical Inference
I
So we make inferences about population parameters

using sample statistics.
But suppose we take a new sample - we would then
and s2 .
get a different X
There is variability associated with these sample
statistics.
It would be nice to know how they differ from
sample to sample.
3 / 69
Introduction
Sampling Distribution
I
Sampling distribution: The distribution of a

sample statistic that would arise if we were to
repeatedly take samples from a population.
Knowing the sampling distribution of a sample
statistic can tell us how accurately we are
estimating a population parameter.
We will focus on the sampling distribution of the
sample mean and the sampling distribution of the
sample proportion.
4 / 69
Sampling Distribution of Sample Mean
Example 1
Example 1
I
Suppose that we roll a fair four-sided die and let X

be the number that comes up.
The population can be thought of as an infinite
number of rolls of the die.
Lets repeatedly draw samples of size 2, i.e., roll the
die twice.
Find the sampling distribution of the sample mean,
X.
5 / 69
Example 1
Probability Distribution of X
x
1 2 3 4
p(x)
I
1
4
1
4
1
4
1
4
Remember that probability distributions represent

populations.
We can calculate the population mean and
population variance from the probability distribution
in the usual way.
6 / 69
Example 1
Mean and Variance of X

x
1 2 3 4
p(x)
= E(X) =
1
4
1
4
1
4
1
4
(x p(x))
all x
5
2
2
= V (X) = E(X 2 ) (E(X))2
2
15
5
5
=
=
2
2
4
=
7 / 69
Example 1
Sampling Distribution of X
I
Lets calculate the sample mean for all possible

samples of size 2:
1
1
First 2
roll 3
4
I
=1
= 1.5
=2
= 2.5
Second roll
2
3
=2
= 1.5 X
= 2.5
=2
X
=3
= 2.5 X
= 3.5
=3
X
= 2.5
=3
= 3.5
=4
Note that each possible sample of size 2 has

1
probability 16
of being drawn.
8 / 69
Example 1
Sampling Distribution of X
I
We can determine the sampling distribution of X

from the previous table:
x
1.5
2.5
3.5
p(
x)
1
16
2
16
3
16
4
16
3
16
2
16
1
16
We can calculate this sampling distribution exactly

since we can completely list all possible samples of
size 2.
9 / 69
Example 1
Mean of X
x
1.5
2.5
3.5
p(
x)
1
16
2
16
3
16
4
16
3
16
2
16
1
16
=
E(X)
(
x p(
x))
all x
5
2
= E(X) =
=
10 / 69
Example 1
Variance of X
x
1.5
2.5
3.5
p(
x)
1
16
2
16
3
16
4
16
3
16
2
16
1
16
2
2
E(X)
2
55
5
=
8
2
5
=
8
V (X) 2
=
=
2
2
=E X
V (X)
11 / 69
Example 2
Example 2
I
I
I
What happens when we cant list all possible

samples of a certain size?
The best we can do is to repeatedly draw samples,
calculate sample statistics and approximate the
sampling distribution.
Suppose that X U (0, 100).
X is continuous, so we clearly cannot list all
possible samples (of any size!).
Lets run a little experiment...
12 / 69
Example 2
Experiment
1.
2.
3.
4.
5.
Generate some data.

Display this data in a histogram.
Analyse some sample statistics.
Repeat with different sample sizes (n).
See what happens!
13 / 69
Example 2
n = 20
14 / 69
Example 2
n = 20
15 / 69
Example 2
Histogram and Some Statistics
n
20
X 53.23572497
26.900781
s
2
s 723.6520183
16 / 69
Example 2

I
Why does the histogram not look more like a

uniform distribution?
17 / 69
Example 2

I
Why is the sample mean not equal (or close) to:

E(X) =
a + b 0 + 100
=
= 50?
2
2
Why is the sample variance not equal (or close) to:

(100 0)2
(b a)2
=
= 833.33?
V (X) =
12
12
18 / 69
Example 2
n = 1000
19 / 69
Example 2
n
1000
X 49.3829
s 29.35405
s2 861.66
20 / 69
Example 2
n = 32000
21 / 69
Example 2
n
32000
X 49.97777
s 28.86928
s2 833.4354
22 / 69
Example 2
Summary
As n increases, the distribution of the observations

in the sample will look more like the distribution of
the population from which the sample was drawn.
As n increases, the sample statistics (such as the
sample mean and sample variance) will converge
towards their population values.
23 / 69
Example 2
Another Experiment
Lets now re-do the previous experiment, but with

several samples at the same time.
Well examine the sample means of the samples,
and analyse their characteristics.
See what happens when we change both the sample
size (n) and the number of samples generated (m).
24 / 69
Example 2
Another Experiment
25 / 69
Example 2
n = 20 and m = 20
26 / 69
Example 2
n = 20 and m = 20
27 / 69
Example 2
Histogram and Some Statistics of the

Sample Means
n
20
m
20
X 52.6569567
X
sX 7.19465383
s2X 51.7630437
28 / 69
Example 2
n = 20 and m = 256
I
I
Distribution looks funny...

Lets re-try the exercise, but increase the number of
samples generated from m = 20 to m = 256.
We will keep the number of observations in each
sample to n = 20, as before.
29 / 69
Example 2
n = 20 and m = 256
30 / 69
Example 2
n = 20 and m = 256
31 / 69
Example 2

Sample Means
n
20
m
256
X 50.39852
X
sX 6.387126
s2X 40.79538
32 / 69
Example 2
Summary
I
The sampling distribution of the sample means

looks like a normal distribution!
This phenomenon, called the Central Limit
Theorem, will occur regardless of the underlying
population distribution (in this case, a uniform
distribution).
Also, the mean of the sampling distribution of the
sample means is equal to the population mean of X.
33 / 69
Example 2
n = 100 and m = 256
Lets see what happens when we increase the

number of observations in each sample from n = 20
to n = 100.
Well leave the number of samples to be m = 256,
as before.
34 / 69
Example 2
n = 100 and m = 256
35 / 69
Example 2

Sample Means
n
100
m
256
XX 49.96038
sX 3.000858
s2X 9.005149
36 / 69
Example 2
Comparison of n = 20 to n = 100
I
Compare the results from when n = 20 and

m = 256 to when n = 100 and m = 256.
n
20
m
256
XX 50.39852
sX 6.387126
s2X 40.79538
n
100
m
256
XX 49.96038
sX 3.000858
s2X 9.005149
37 / 69
Example 2
Comparison of n = 20 to n = 100
n = 100
n = 20
38 / 69
Example 2
Summary
I
The variance and standard deviation of the sample

means reduce significantly when the sample size n
increases!
This is because the variance of the sampling
distribution of the sample means is equal to:
2
X
2
=
n
with the standard deviation equal to:
X =
n
39 / 69
Example 2
Standard Error
I
Note that when we talk about the standard

deviation of a sample statistic, this is not the same
as the population standard deviation of X (our
data).
To make this distinction clear, the standard
deviation of the sampling distribution of a statistic
is called the standard error (SE) of the statistic.
And thats why we use the notation X , where the
subscript indicates that it is the standard deviation
of X.
40 / 69
Central Limit Theorem
Mean and Variance of Sample Mean

I
So the sample mean is given by:

= 1 (X1 + . . . + Xn )
X
n
Since random sample, each Xi is independent and

E(Xi ) = and V (Xi ) = 2 .
Therefore, we can use the laws of expected value
and
and variance to calculate X = E(X)
2
X
= V (X).
41 / 69

=E
X = E(X)

1
(X1 + . . . + Xn )
n
1
E (X1 + . . . + Xn )
n
1
= (E(X1 ) + . . . + E(Xn ))
n
1
= ( + . . . + )
n
1
= n =
n
=
42 / 69

2
X
= V (X) = V

1
(X1 + . . . + Xn )
n
1
V (X1 + . . . + Xn )
n2
1
= 2 (V (X1 ) + . . . + V (Xn ))
n

1
= 2 2 + . . . + 2
n
1
2
= 2 n 2 =
n
n
=
43 / 69

I
If X is a random variable (discrete or continuous)

with mean and variance 2 , then in general:

2
2
N X = , =
as n
X
X
n
Which means if we standardise:

X
Z N (0, 1) as n
44 / 69
How Large Should n Be?

I
Generally, it depends on the population distribution

of X.
I
If X has a normal distribution, then the sample mean

has a normal distribution for all sample sizes.
If X has a distribution that is close to normal, the
approximation is good for small sample sizes (e.g.,
n 20).
If X has a distribution that is far from normal, the
approximation requires larger sample sizes (e.g., n > 50).
45 / 69
Example 1
Example 1
I
Consider the marks of all students who took an

economics test. If marks are normally distributed,
with mean equal to 72 and standard deviation equal
to 9, find:
(a) The probability that any one student will have a mark
over 78.
(b) The probability that a sample of 10 students will have an
average mark over 78.
46 / 69
Example 1
Solution - Part (a)

I
I
Let X be the mark of a randomly selected student.

Then X N = 72, 2 = 92 .

X 78 72
P (X > 78) = P
>
9
= P (Z > 0.67)
= 1 P (Z < 0.67)
= 1 0.7486
= 0.2514
47 / 69
Example 1
Solution - Part (a)
47 / 69
Example 1
Solution - Part (b)

I
be the average mark of the sample of 10

Let X
students.

2
92
2
By CLT, X N X = = 72, X = n = 10 .
> 78) = P
P (X
78 72
X
>
9
X
10
= P (Z > 2.11)
= 1 P (Z < 2.11)
= 1 0.9826
= 0.0174
48 / 69
Example 1
Solution - Part (b)
48 / 69
Example 1
Example 1
I
Why does the probability drop from 25.14% to

1.74%?
I
Interpretation:
I
Standard deviation of sample means (or the standard

error ) is always smaller than standard deviation of single
observations.
Sample means have less variation than single
observations.
E.g., much more likely to find an individual who is

very smart, than a random sample of 10 students
who are all very smart.
49 / 69
Example 2
Example 2
Given a normal distribution with a mean of 50 and a

variance of 100, find the probability that the sample
mean of a sample of 25 observations will differ from
the population mean by less than 4 units.
50 / 69
Example 2
Solution
I
We only care about the distance between the

sample mean and population mean.
That is, we dont care if the sample mean is larger
than the population mean or smaller than the
population mean, as long as the difference between
them is less than 4 units.
| < 4).
So we want to find P (|X
51 / 69
Example 2
Solution
| < 4) = P (4 < X
< 4)
P (|X
!
4
4
X
<
<
=P
10
10
25
25
= P (2 < Z < 2)
= 0.9544
52 / 69
Example 3
Example 3
I
The number of accidents per week at a hazardous

intersection varies randomly with mean 2.2 and
standard deviation 1.4.
This distribution is discrete and so is certainly not
normal.
What is the approximate probability that there are
fewer than 100 accidents at the intersection in a
year?
53 / 69
Example 3
Solution
I
I
I
Let Xi be the number of accidents that occur in the

ith week of the year.
We know E(Xi ) = 2.2 and V (Xi ) = 1.42 for all i.
We want to find the probability of fewer than 100
accidents in the year, i.e.:
!
!
P52
52
X
100
i=1 Xi
<
P
Xi < 100 = P
52
52
i=1
< 1.923)
= P (X
54 / 69
Example 3
Solution
I
But we know from the CLT that

2
1.4
N 2.2,
X
52
Therefore,

< 1.923 = P
P X
<
1.923 2.2
1.4
52
= P (Z < 1.43)
= 0.0764
55 / 69
Sampling Distribution of Sample Proportion
Binomial Distribution
I
I
I
Recall the Binomial distribution that was introduced

in Week 3.
If X Bin(n, p), then X was counting the number
of successes in n independent Bernoulli trials.
We assumed that p was known.
But in reality, p is an unknown population
parameter.
56 / 69
Sample Proportion
I
Therefore, just like we did with the population mean

, we need to estimate the population proportion p
from a sample.
For example, suppose we are interested in whether
Coke or Pepsi is the more popular soft drink.
Let p denote the population proportion of people
who prefer Coke over Pepsi.
57 / 69
Sample Proportion
I
If X denotes the number of people who prefer Coke

over Pepsi in a randomly selected sample, what is a
reasonable estimate of p?
If n is the size of the sample, then a reasonable
estimate of p is the sample proportion of people
who prefer Coke over Pepsi:
p =
X
n
Lets investigate p in a little more detail...

58 / 69
Sample Proportion
I
Another way we can express X is:

X=
n
X
Xi
i=1
where
(
1 person i prefers Coke over Pepsi
Xi =
0 otherwise
I
So X is a sum of independent Bernoulli random

variables.
59 / 69
Probability Distribution of Xi
xi
p(xi ) 1 p p
= E(Xi ) = 0 (1 p) + 1 p
=p

2 = V (Xi ) = E Xi2 (E(Xi ))2

= 02 (1 p) + 12 p p2
= p(1 p)
60 / 69
Sample Proportion
I
Notice that p is just the sample mean of the

Bernoulli variables:
Pn
X
Xi
= i=1
p =
n
n
Therefore, from slides 41, 42 and 43, we know that:

p = E(
p) = = p
p(1 p)
2
2
p = V (
p) =
=
n
n
61 / 69

I
And we can also apply the CLT to find the sampling

distribution of p!
The CLT tells us that for sufficiently large n:

p(1 p)
2
2
= N p,
p N p = , p =
n
n
If we standardise p, we then get:
p p
q
= Z N (0, 1)
p(1p)
n
62 / 69
Example 1
Example 1
I
(Q9.41 p327) A psychologist believes that 80% of

male drivers when lost continue to drive hoping to
find the location they seek rather than ask
directions. To examine this belief, he took a random
sample of 350 male drivers and asked each what
they did when lost. If the belief is true, determine
the probability that less than 75% said they
continue driving.
63 / 69
Example 1
Solution
I
I
I
Let p be the sample proportion of drivers that keep

driving.
We know that n = 350 and the population
proportion is p = 0.8.
We want to find P (
p < 0.75).
Since n = 350 is very large, we can apply the CLT
and conclude that:

0.8(1 0.8)
p(1 p)
= N 0.8,
p N p,
n
350
64 / 69
Example 1
Solution
I
Therefore:
0.75 0.8
p p
<q
P (
p < 0.75) = P q
p(1p)
n
0.8(10.8)
350
= P (Z < 2.34)
= 0.0096
65 / 69
Example 2
Example 2
(Q9.43 p327) An accounting professor claims that

no more than one-quarter of undergraduate business
students will major in accounting. What is the
probability that in a random sample of 1200
undergraduate business students, 336 or more will
major in accounting?
66 / 69
Example 2
Solution
I
I
I
Let X be the number of students that major in

accounting out of the random sample of 1200.
Then X Bin(n = 1200, p = 0.25).
We want:
P (X 336) = P (X = 336) + P (X = 337) + . . .

. . . + P (X = 1200)
67 / 69
Example 2
Solution
I
We can use the CLT to approximate this probability:

336
X
P (X 336) = P
1200 1200
P (
p > 0.28)
0.28 0.25
p p
= P q
<q
p(1p)
n
0.25(10.25)
1200
= P (Z > 2.40)
= 1 0.9918
= 0.0082
68 / 69
Reference
Reference
Chapter 9.
I
Keller, 9th edition.
69 / 69

Stat7055 l05 Samp-Dist

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Stat7055 l05 Samp-Dist

Uploaded by

Copyright:

Available Formats

STAT7055

Statistical inference: Determining the behaviour

Population parameters describe populations, e.g.,

Calculate statistics from sample, e.g., sample mean X

So we make inferences about population parameters

Sampling distribution: The distribution of a

Sampling Distribution of Sample Mean

Suppose that we roll a fair four-sided die and let X

Sampling Distribution of Sample Mean

Remember that probability distributions represent

Sampling Distribution of Sample Mean

Mean and Variance of X

Sampling Distribution of Sample Mean

Lets calculate the sample mean for all possible

Note that each possible sample of size 2 has

Sampling Distribution of Sample Mean

We can determine the sampling distribution of X

We can calculate this sampling distribution exactly

Sampling Distribution of Sample Mean

Sampling Distribution of Sample Mean

Sampling Distribution of Sample Mean

What happens when we cant list all possible

Sampling Distribution of Sample Mean

Generate some data.

Sampling Distribution of Sample Mean

Sampling Distribution of Sample Mean

Sampling Distribution of Sample Mean

Histogram and Some Statistics

Sampling Distribution of Sample Mean

Histogram and Some Statistics

Why does the histogram not look more like a

Sampling Distribution of Sample Mean

Histogram and Some Statistics

Why is the sample mean not equal (or close) to:

Why is the sample variance not equal (or close) to:

Sampling Distribution of Sample Mean

Sampling Distribution of Sample Mean

Histogram and Some Statistics

Sampling Distribution of Sample Mean

Sampling Distribution of Sample Mean

Histogram and Some Statistics

Sampling Distribution of Sample Mean

As n increases, the distribution of the observations

Sampling Distribution of Sample Mean

Lets now re-do the previous experiment, but with

Sampling Distribution of Sample Mean

Sampling Distribution of Sample Mean

Sampling Distribution of Sample Mean

Sampling Distribution of Sample Mean

Histogram and Some Statistics of the

Sampling Distribution of Sample Mean

Distribution looks funny...

Sampling Distribution of Sample Mean

Sampling Distribution of Sample Mean

Sampling Distribution of Sample Mean

Histogram and Some Statistics of the

Sampling Distribution of Sample Mean

The sampling distribution of the sample means

Sampling Distribution of Sample Mean

n = 100 and m = 256

Lets see what happens when we increase the

Sampling Distribution of Sample Mean

n = 100 and m = 256