4 Random variables and probability distributions

Example. Consider tossing four coins. The possible outcomes are then S = fHHHH; HHHT; : : : ; THHH; HHTT; HTHT; : : : ; TTHH; HTTT; THTT; : : : ; TTTH; TTTTg

Suppose we are interested in the number of heads in each a toss (of the four coins). Let variable X indicate that number, so that X (HHHH) = 4, X (HHHT) = : : : = X (THHH) = 3, etc. The value of X is determined by the random experiment. Conse-

quently a variable whose value is determined by a random experiment is called a random variable.

Figure. Random variable as a function: X : S!I R. The probability distribution of the random variable consist of the values of the variable and the associated probabilities.
Example. In the previous example the probabilities [denoted as P (X = x)] for each possible values can be calculated under the symmetry assumption to be x P (X = x) 0
1 16

A random variable is an uncertain quantity whose value depends on change. Mathematically a random variable is dened as a function with denition set (domain) the sample space S and value set (the range) the real numbers. So a random variable is a function with domain S and range I R, the real numbers (usually), i.e., X:S!I R
27

1
1 4

2
3 8

3
1 4

4
1 16

Note.

P
x2SX

P (X = x) = 1, where SX = X (S ), the set of

possible values of the random variable (the range space).

28

Note. It is usual practice to distinguish the random variable and its value. Capital letters (like X above) for the random variables and lower case letters (like x above) for the ~, Y ~ , etc. values are usual. Notations like X or x, y, etc. are also common in literature for random variables. Sometimes the notations are not dierentiated. In such cases it should be clear from the context which one is in question. A discrete random variable can assume a countable number of values
Example. Typical examples of discrete random variables are the number of children in a family, the result of a toss of a cube, the number of heads in the previous coin-toss example, etc. Also a random variable with range Z = f: : : 2; 1; 0; 1; 2; : : :g (the whole numbers) is discrete.

The probability distribution of a random variable X is a function f satisfying (1) f (x) 0; for all x (2)
R x f (x) dx = 1; if X continuous P x f (x) = 1; if X discrete

Note. In the discrete case f (x) = P (X = x). In the continuous case this does not hold! In the continuous case f (x) is usually called a probability density function. The cumulative distribution function, F (x), of a random variable is
( P
F (x) = P (X x) =

Rx

ix P (X

= i); X discrete

f (t) dt; X continuous 1

A continuous random variable can take any value in an interval of values (the value set is uncountable) Note. A random variable can also be a mixture of continuous and discrete.
29

Note. The distribution function is useful in probability calculations:

(i) P (a < X b) = P (X b) P (a X ) = F (b) F (a) (ii) P (X > c) = 1 P (X c) = 1 F (c)

30

Expected value The expected value of a discrete random variable is dened as = E [X ] =

X
x

Variance and Standard Deviation The variance of a random variable is dened as

( R1 P
2 = Var [X ] =
1

xP (X = x) =
Z 1
1

X
x

xf (x):

(x )2f (x) dx; X continuous )2f (x); X discrete:

x (x

If X is continuous then = E [X ] =
( P

That is xf (x) dx: Var [X ] = E (X )2: The standard deviation is the positive square root of the variance
q

Generally, if h is a function, and Y = h(X ),

E [Y ] = E [h(X )] =

R1

x h(x)P (X = x); X discrete

h(x)f (x) dx; X continuous

= Properties:

Properties:
(1) E [c] = c; c a constant (2) E [aX + bY ] = aE [X ] + bE [Y ]; a and b constants
Example. Monthly sales of a certain product X is distributed as 5; 000 6; 000 7; 000 8; 000 9; 000 x P (X = x) 0:2 0:3 0:2 0:2 0:1 Then E [X ] = 0:2 5; 000 + + 0:1 9; 000 = 6; 700: Suppose the prot function of the product is h(X ) = 2X 8; 000 (Eur). Then using (1) and (2) above, the expected prot is E [h(X )] = 2E [X ] 8; 000 = 2 6; 700 8; 000 = 5; 400 Eur:

(1) Var [X ] = E (X )2 = E [X 2 ] 2 (2) Var [c] = 0; c a constant (3) Var [aX + b] = a2Var [X ] Chebysev's Theorem: Let X be a random variable with E [X ] = and Var [X ] = 2 . Then for any constant k > 1 P (jX j < k ) 1 1=k2:
32

31

Bi-variate random variables Consider two random variables X and Y (say) together. The joint probability distribution function of X and Y is a function satisfying
(1) f (x; y) 0; for all x; y R R (2) x y f (x; y) dx dy = 1; if X and Y are continuous P P x y f (x; y ) = 1; if X and Y are discrete

Covariance and Correlation Covariance of X and Y is dened as xy = Cov [X; Y ] = E [(X x )(Y y )]; where x = E [X ] and y = E [Y ]. Correlation of X and Y is dened as xy xy = xy
Example. Suppose that in a population we have the following distribution of gender, X , with 0 = male, 1 = female, and political aliation, Y , with 0 = Democrat, 1 = Republican, 2 = Other. The joint probability distribution f (x; y) and marginal distributions fx (x) and fY (y) are as follows Y nX 0 1 2 fX (x) Now E [X ] E [Y ] E [X 2 ] E [Y 2 ] E [XY ] = = = = = 0 0:56 + 1 0:44 = 0:44 0 0:47 + 1 0:40 + 2 0:13 = 0:66 02 0:56 + 12 0:56 = 0:56 02 0:47 + 12 0:40 + 22 0:13 = 0:92 0 0 0:20 + + 1 2 0:07 = 0:24 0 0:20 0:30 0:06 0:56 1 0:27 0:10 0:07 0:44 fY (y) 0:47 0:40 0:13 1:00

The joint cumulative distribution function is dened as

( P
F (x; y ) =
ix j y f (i; j ) X , and Y Rx Ry f (s; t) ds dt X and 1 1

discrete Y continuous

Random variables X and Y are independent if f (x; y) = fX (x)fY (y) for all x and y; where fX (x) and fY (y) are the probability distribution functions of X and Y , respectively.
Note. Suppose X and Y are discrete, then for example X f (x; y ): fX (x) =
y

33

34

So
2 x 2 y xy

= = = =

E [X 2 ] (E [X ])2 = 0:56 0:442 = 0:3664 0:92 0:662 = 0:4844 0:24 (0:44)(0:66) = 0:0504 xy =(x y ) = 0:0504=

Example. Independence and correlation X nY 1 2 3 fY (y) 6 0:2 0 0:2 0:4 7 0 0:2 0 0:2 8 0:2 0 0:2 0:4 fX (x) 0:4 0:2 0:4 1

xy

(:3664)(:4844) 0:120

Properties: X and Y random variables, a, b, c, and d constants

(1) Cov [X; Y ] = E [XY ] E [X ]E [Y ] (2) Cov [aX + c; bY + d] = abCov [X; Y ] (3) 1 xy 1 (4) Var [aX + bY ] = a2Var [X ] + b2Var [Y ] + 2abCov [X; Y ] (5) If X and Y are independent, then E [XY ] = E [X ]E [Y ]; Cov [X; Y ] = 0; and xy = 0
2. Note. Cov [X; X ] = Var [X ] = x

We easily calculate that xy = 0, but for example f (1; 6) = 0:2 6 = 0:4 0:4 = fX (x)fY (y). Thus X and Y are uncorrelated but not independent.

Several random variables (multivariate case) Consider random variables X1; X2; : : : ; Xm. Let 2, Cov [X ; X ] = , E [Xi] = i, Var [Xi] = i i j ij and ij = ij =(ij ). The covariance matrix of X1 ; : : : ; Xm is
2 1 12 B 2 B 21 2 =B . . B . . . @ . m1 m2

Note. Covariance and correlation measure linear dependency between X and Y . Note. If Cov [X; Y ] = 0 then xy = 0, and we say that X and Y are uncorrelated. However, if X and Y are uncorrelated it does not imply that X and Y are independent!
35

: : : 1m : : : 2m . ... . . 2 : : : m

1 C C C C A

Note. ij = ji. So is symmetric.

36

In practice more useful is the correlation matrix 12 : : : 1m 1 : : : 2m . . ... . . 1 m1 m2 : : : Again P is symmetric, i.e., ij = 1; : : : ; m.
B P=B B 21 @ . . 0

Some discrete probability distributions (a) Bernoulli distribution Let X be a random variable with possible values 0 and 1, and let P (X = 1) = p. Then we can write the distribution function of X as
(

1 C C C A

ji, i; j =

Example. Let Xi denote the annual return of a stock i = 1; : : : ; m, and let E [Xi ] = i . Construct a portfolio Xp = w1 X1 + + wpXm
th stock. Assume where wi indicate the fraction invested in the Pi m for the shake of simplicity that wi 0, and w = 1. i=1 i

fX (x) =

px(1 p)1x ; if x = 0 or x = 1 0; otherwise;

and denote X Ber(p), with the meaning "X is Bernoulli distributed with parameter p". An example of a Bernoulli random variable is the result of a toss of a coin with Head, say, equal to one and Tail equal to zero. An experiment of this kind is called a Bernoulli trial Note. p is called the probability of "success", and q = 1 p the probability of "failure". E [X ] = 0 (1 p) + 1 p = p Var [X ] = E [X 2 ](E [X ])2 = pp2 = p(1p) = pq
38

The mean return of the portfolio is p = E [Xp ] = and variance

2 = Var [X ] p p m X i=1

wi E [Xi ] =

m X i=1

wi i

= =

Pm Pm Pm
i=1 j =1

wi wj ij

w2 2 i=1 i i

+2

Pj1 Pm
i=1

ww j =2 i j ij

Note. Adoption of matrix notations gives a more elegant forms for the formulas p = w0
2 = w0 w; p where w = (w1 ; w2 ; : : : ; wm)0 is the vector of the portfolio weights,

= (1 ; : : : ; m)0 the vector of expected values of the returns, and the prime indicates transposition. 37

(b) Binomial distribution Suppose a Bernoulli trial is repeated independently n times (e.g. tossing a coin n times). Then probabilities of x success, x = 0; 1; : : : ; n can be calculated with the binomial distribution
8 < n px (1 p)nx; x = 0; 1; : : : ; n x fX (k) = : 0; otherwise n

A binomial variable random variable is a sum of independent Bernoulli variables, i.e., X = Pn i=1 Xi, Xi Ber(p). So = E [X ] =
n X i=1

E [Xi] =

n X i=1

p = np;

2 = Var [X ] = np(1 p) and =

q

where

np(1 p):

x with

n! x!(n x)!

Example. Two percents of a product are defective. If a lot of 100 items are ordered what is the probability that there are no defective items? What is the probability that there are at least two defective items. Let X denote the number of defective items in the lot. Then possible values of X are X = 0; 1; : : : ; 100. Binomial model is suitable here, so X Bin(100; 0:02): (0:02)0 (0:98)100 0:137 0 P (X 2) = 1 P (X 1) = 1 (P (X = 0) + P (X = 1)) = 1 (0:98)100 100(0:02)(0:98)99 0:597 Note. The expected value is E [X ] = 2. P (X = 0) =

m! = m(m 1)(m 2) 1 is the factorial. Note. By denition 0! = 1. Notation X Bin(n; p).

39

100

40

(c) Poisson distribution The Poisson distribution is a model for describing a random variable that counts the number of occurrences in a particular interval of time. E.g. then number of equipment failure per week, number of trac accidents per month, number of customers visiting a service point per day, etc. Then in principle there is no upper limit, so X = 0; 1; 2; : : :. Poisson distribution: X Po() x e ; for x = 0; 1; 2; : : : x! where e = 2:71828 : : : is the base of the natural logarithm. f (x) = E [X ] = and Var [X ] = :
41

The Poisson distribution can be used also as an approximation for the binomial probabilities if p is "small" and n is large (rules of thumb: p 0:05 and n 20). Then = np is used.
Example. Using the Poisson approximation in the above example with = np = 100 0:02 = 2, we get P (X = 0) = and P (X 2) = 1 P (X 1) = 1 e2 2e2 0:594 : : : Pretty close! 20 2 e 0:135 0!

42

Continuous distributions In the case of continuous random variables the distribution is described in terms of density functions. (a) Uniform distribution X U (a; b) with a < b The density function (df) of a uniform distribution in the interval [a; b] is
8 <

= E [X ] = 2 = Var [X ] =

1 (a + b) 2

1 (b a)2 12 Note. In the simulation studies the uniform distribution U (0; 1) has a central role. (b) Exponential distribution Exponential distribution has proven to be a useful for modeling the (random) time between two occurrences of Poisson random variable. For example, if the number of cars arriving to a service station per minute has a Poisson distribution, then the time between two consecutive cars has an exponential distribution (time is measured on a continues scale). X is exponentially distributed, denoted as X Exp( ) if the density function is f (x) = ex; x 0; > 0
44

f (x) =

: 0; otherwise

1 ba ;

for a x b

The cumulative distribution function (cdf) is

8 > 0; for x < a > > Z x < xa ; for a x b F (x) = f (t) dt = > a > ba > : 1; for x > b

43

F (x) =

Z x
0

f (t)dt = 1 ex ; = E [X ] = 1

x 0:

1 2 = Var [X ] = 2
Example. Let X denote the time in hours a machine is operating properly such that X Exp(2). What is the probability that the machine will work the next hour without breaking? The answer is P (X 1) = 1 F (1) = 1
0

2e2x dx = 1 (1 e21 ) = 0:1353:

45