Common Continuous Distributions

Statistics 104
Autumn 2004

Taken from Statistics 110 Lecture Notes

c 2004 by Mark E. Irwin

Copyright

Common Continuous Distributions

Uniform
Exponential
Normal
Gamma
Cauchy

Continuous Distributions

Uniform Distributions
This distribution describes events that are equally likely in a range (a, b).
As mentioned before, it is what people often consider as a random number.
The PDF for the for the uniform distribution (U (a, b)) is
1
f (x) =
;
ba
The CDF is

0
F (x) =

xa
ba

axb

x<a
axb
x>b

The first two moments of the uniform are

a+b
;
E[X] =
2
Continuous Distributions

(b a)
Var(X) =
;
12

SD(X) =

(b a)2
12
2

k=3
k=2
k=1
k=0.25

1.0
0.0

0.5

f(x)

1.5

U(0,1)
U(2,3)

Uniform CDFs

U(k,k) CDFs

f(x)
2

0
x

Continuous Distributions

0
x

U(0,1)
U(2,3)

0.0 0.2 0.4 0.6 0.8 1.0

0.0 0.2 0.4 0.6 0.8 1.0

F(x)

U(k,k) Densities
2.0

0.0 0.2 0.4 0.6 0.8 1.0

f(x)

Uniform Densities

k=3
k=2
k=1
k=0.25

0
x

Exponential Distribution
The exponential distribution is often used to describe the time to an event.
The PDF for the for the exponential distribution (Exp()) is
f (x) = ex;

The CDF is

F (x) =

x0

1 ex x 0
0
x<0

The first two moments of the exponential are

1
E[X] = ;

Continuous Distributions

Var(X) =

1
;
2

SD(X) =

0.4

lambda=0.5
lambda=1
lambda=2

0.0

lambda=0.5
lambda=1
lambda=2

0.8

Exponential CDFs

f(x)

0.0 0.5 1.0 1.5 2.0

f(x)

Exponential Densities

10

10

The exponential distribution has an interesting property. It is said to be

memoryless. That is, it satisfies
P [T > t + s|T > s] =
=
Continuous Distributions

P [T > t + s and T > s]

P [T > s]
P [T > t + s] et+s
= s = et
P [T > s]
e
5

So the chance that what we are observing survives another t units of time
doesnt depend on how long we have observed it so far.
This property limits where the exponential distribution can be used. For
example, we wouldnt want to use it to model human lifetimes.
Another potential drawback is that the parameter describes the
distribution completely. Once you know this, you also know the standard
deviation, skewness, kurtosis, etc.
Even with these drawbacks, the exponential distribution is widely used.
Examples where it may be appropriate are
Queuing theory - times between customer arrivals
Times to relapse in leukemia patients
Times to equipment failures
Distances between crossovers during meiosis
Continuous Distributions

The parameter can often be thought of as a rate or a speed parameter.

The exponential distribution can be parameterized in terms of the mean
time to the event, = 1 . With this parameterization the PDF and CDF
are

f (x) =

Continuous Distributions

ex/
;

x0

F (x) =

1 ex/ x 0
0
x<0

Normal Distributions
The normal distribution is almost surely
the most common distribution used in
probability and statistics. It is also
referred to as the Gaussian distribution,
as Gauss was an early promoter of its
use (though not the first, which was
probably De Moivre). It is also what
most people mean when they talk about bell curve. It is used to describe
observed data, measurement errors, an approximation distribution (Central
Limit Theorem).
The PDF for the for the normal distribution (N (, 2)) is
1
(x)2 /2 2

f (x) =
e
2

Continuous Distributions

The two parameters of the distribution are the mean () and the variance
( 2). A special case is the standard normal density which has = 0
and 2 = 1 and its PDF is often denoted by (x). As we shall see, once
we understand the standard normal (N (0, 1)), we understand all normal
distributions.

The CDF for the normal distribution doesnt have a nice form. The CDF
for the standard normal is often denoted by (x) which is of the form

Z
(x) =

2
1
eu /2du
2

The CDF for any other normal distribution is based on (x).

Continuous Distributions

N(0,sigma^2) Densities

0.4

0.8

N(mu,1) Densities

0.6
0.2
0.0

0.0

N(0,sigma^2) CDFs

0
x

Continuous Distributions

N(mu,1) CDFs

f(x)
2

0
x

mu = 0
mu = 1
mu = 1

0.0 0.2 0.4 0.6 0.8 1.0

0.0 0.2 0.4 0.6 0.8 1.0

f(x)

sigma = 1
sigma = 0.5
sigma = 2

0.4

f(x)

0.2
0.1

f(x)

0.3

0
1
1

sigma = 1
sigma = 0.5
sigma = 2

10

The normal distribution is an example of a symmetric distribution, with the

point of symmetry being the mean . For a symmetric distribution with
mean 0
P [X x] = P [X x]

Theorem. Let X N (, 2) and let Y = aX + b. Then Y N (a +

b, (a)2)
Continuous Distributions

11

Proof.
FY (y) = P [Y y]
= P [aX + b y]

yb
= P X
a

yb
= FX
a
Therefore
fY (y) =
=

d
FX
dy

yb
a

1
yb
fX
|a|
a

Note that to this point, we havent made any assumptions about the
distribution of X. Any linear transformation of a RV gives this relationship.
Continuous Distributions

12

Now if X N (, 2), then

1
1 y b a
exp
fY (y) =
2
a
|a| 2
which is the density for a N (a + b, (a)2). 2
From this all normal density curves must
have the same basic shape and if X
N (, 2)
f (x) =
and

1
x

x
F (x) =

One consequence of this, is that to get probabilities involving normal

distributions we only need a single function, or table.
Continuous Distributions

13

If X N (, 2), then
X
Z=
N (0, 1)

The values Z are sometimes

referred to as the standard scores.
Suppose for example that blood
pressure (X) can be modelled
(approximately) by a normal
distribution with = 120 and
= 20.

X ~ N(mu,sigma^2)

x = mu + z sigma

Z ~ N(0,1)

If we are interested in the

P [X 140], this is the same as the P [Z 1] since
140 120
z=
=1
20
Table 2 of Rice gives the CDF of the standard normal.
Continuous Distributions

14

Note that this table

is actually taken from
Moore and McCabes
Introduction to the
Practice of Statistics.
However it is the same
a Table 2 in Rice.
z is the standard
normal variable. The
value of P for z
equals 1 minus the
value of P for +z; for
example P for 1.62
equals 1 0.9474 =
0.0526.

Continuous Distributions

15

We can use the table to get the

following probabilities

0.020

Blood Pressure Density

P [X 140] = P [Z 1] = 0.8413

0.010
0.000

f(x)

P [X 140]

60

80

95100

120

140

160

180

Blood Pressure

P [X 95]

95 120
P [X 95] = P Z
20
= P [Z 1.25]
= 1 P [Z 1.25]

Continuous Distributions

16

Using the fact that P [Z 1.25] = 1 P [Z 1.25] (table flip),

P [Z 1.25] = 1 0.8944 = 0.1056. Therefore
P [X 95] = 1 0.1056 = 0.8944
P [95 X 140]

0.010
0.000

f(x)

0.020

Blood Pressure Density

60

80

P [95 X 140] = P [X 140] P [X 95]

95100

120

140

160

180

Blood Pressure

= P [Z 1] P [Z 1.25]
= 0.8413 0.1056 = 0.7357

Continuous Distributions

17

Gamma Distributions
The gamma distribution can be used model a wide range of non-negative
RVs. It has been used to model times between earthquakes, the size of
automobile insurance claims, rainfall amounts, plant yields.
The PDF for the for the gamma distribution (G(, )) is
1 x
f (x) =
x
e
;
()

x0

The parameter is the shape parameter of the gamma distribution and

is the scale parameter.

The gamma distribution is a generalization of exponential distribution as

Exp() = G(1, ).
Continuous Distributions

18

G(2,lamdba) Densities
1.2

1.2

G(alpha,1) Densities

0.8
0.4
0.0

G(alpha,1) CDFs

G(2,lamdba) CDF

4
x

Continuous Distributions

f(x)
2

alpha=1
alpha=0.75
alpha=2
alpha=4
0

0.0 0.2 0.4 0.6 0.8 1.0

0.0 0.2 0.4 0.6 0.8 1.0

f(x)

alpha=1
alpha=0.75
alpha=2
alpha=4

f(x)

0.0

0.4

f(x)

0.8

alpha=1
alpha=0.75
alpha=2
alpha=4

alpha=1
alpha=0.75
alpha=2
alpha=4
0

19

The normalization constant

Z

() =

x1exdx

is the Gamma function evaluated at .

Some useful properties of the Gamma function are
1. (1) =

R
0

exdx = 1

2. () = ( 1)( 1) (Prove by integration by parts).

3. If n is a positive integer, then (n) = (n 1)! (Direct consequence of
the first two facts).
4. (0.5) =

(Useful in showing that the variance for a normal is 2).

The CDF of the gamma doesnt have a nice closed form so you need tables
or a computer to find probabilities involving the gamma.
Continuous Distributions

20

The first two moments of the gamma are

E[X] =

Var(X) =

SD(X) =

;
2

Proof. Let X G(, 1). Then

Z

E[X ] =

1 x

x
Z

=
0

nx

e
dx
()

x+n1ex
dx
()

( + n)
()

So
( + 1) ()
E[X] =
=
=
()
()
Continuous Distributions

21

and

( + 2) ( + 1)()
E[X] =
=
= ( + 1)
()
()

This implies that

Var(X) = E[X 2] (E[X])2 =
Let Y =

X
.

Then Y G(, ) as
(y)1ey
fY (y) =
()

Thus
2

E[X]
E[Y ] =
= ;

Continuous Distributions

Var(X)

Var(Y ) =
= 2;
2

22

Beta Distributions
Beta distributions are useful for data that occur in fixed, finite intervals
The PDF for the for the beta distribution (Beta(, )) is
xa1(1 x)b1
f (x) =
;
(a, b)

0x1

The function (a, b) is known as the Beta function and is

Z

(a, b) =

xa1(1 x)b1dx

=
Continuous Distributions

(a)(b)
(a + b)
23

Beta(a,6)Densities

1
2
3
4
8

4
f(x)

1.5

0.0

0.5

1.0

f(x)

2.0

0.8
1
2
4
6

2.5

Beta(a,a) Densities

0.0

0.2

0.4

0.6
x

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

Like many continuous distributions, the CDF for the beta does not have a
nice form and must be determined through tables or software.
Continuous Distributions

24

The first two moments of the uniform are

E[X] =

a
;
a+b

Var(X) =

ab
(a + b + 1)(a + b)2

Proof.
E[X n] =
=

So
E[X] =
and

1
(a, b)

xnxa1(1 x)b1

(a + n, b)
(a, b)

a
(a + 1, b) (a + 1)(b) (a + b)
=
=
(a, b)
(a + b + 1) (a)(b) a + b

(a + 1)a
(a + 1, b)
E[X ] =
=
(a, b)
(a + b + 1)(a + b)
2

Continuous Distributions

25

which implies
a
Var(X) =
a+b

a+1
a

a+b+1 a+b

a
b
=
a + b (a + b + 1)(a + b)

Continuous Distributions

26

Cauchy Distributions
The Cauchy distribution (also known as the Lorentzian distribution), is
often used for describing resonance behavior. It can also be used to describe
outliers in data sets. However it is more commonly used in probability and
statistics as a distribution that can be used for counter examples.
The PDF for the for the Cauchy distribution (C(, )) is

f (x) =

The CDF is

Continuous Distributions

x 2
1 + ( )

1 1
x
F (x) = + Arctan
2

27

0.15

f(x)

0.25

0
1
1

0.05

f(x)

C(0,sigma) Densities

0
x

0.0 0.1 0.2 0.3 0.4 0.5 0.6

C(mu,1) Densities

1
0.5
2

The Cauchy is another example of a location-scale distribution (the normal

is the first weve discussed). If X C(0, 1), then Y = + X is C(, ).

Continuous Distributions

28

The Cauchy distribution is known as a heavy tailed distribution. Its tails

decay to 0 very slowly.

0.4

Normal vs Cauchy

0.2
0.0

0.1

f(x)

0.3

N(0,1)
C(0,1)

So slowly in fact, that the Cauchy has no moments. For all n,

Z

|xn|f (x)dx =

Continuous Distributions

29