You are on page 1of 38

Mathematical Foundation

For
Communication
Engineering
V. R. Gupta
Assistant Professor
Department of Electronics and
Telecommunication
YCCE, Nagpur
Course code : ET 951
Course Name : Mathematical Foundations for
Communication Engineering
Course objective:
1. To develop an understanding of intermediate probability theory.
2. To develop an understanding of random variables.
3. Understanding the theory associated with sequences of random
variables.
4. To develop abstract and critical reasoning by studying logical
proofs and the axiomatic method as applied to basic probability.

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Course outcomes:
At the conclusion of the course, students will be able to
Apply elementary probability concepts, such as
disjoint/independent events, conditional probability, and total
probability to solve practical probability problems.
Apply discrete random variables and probability mass functions to
compute probabilities and expected values in a variety of
applications.
Apply probability theory to analyze and reduce transmission errors
in digital communication systems.
Analyze pairs of random variables in terms of their joint
probabilities, covariance, and correlation coefficient.
V. R. Gupta, Assist.Prof., ET, YCCE,
Nagpur
Apply continuous random variables and probability density
functions to compute probabilities and expected values in a variety
of applications.
Use Matlab to calculate probabilities and simulate the
performance of systems containing randomness.
Develop probability models and Prove elementary probability
results.
Perform rigorous probability calculations involving random
variables.
Obtain distributional information about a random variable.
Determine the distribution of a functional combination of given
random variables V. R. Gupta, Assist.Prof., ET, YCCE,
Nagpur
Course Assessment:
Grading is based on the following components that are weighted
as described below:
Home Assignments :5%
Open book Test :2%
Quiz :2%
Attendance : 1%
Mid Sem. Exam-1 : 15%
Mid Sem. Exam-2 : 15%
End Sem. Exam. : 60%
V. R. Gupta, Assist. Prof., ET,
YCCE, Nagpur
UNIT-1: Introduction to Probability :
Definitions, scope and history; limitation of classical and relative-frequency-
based definitions. Sets, fields, sample space and events; axiomatic definition of
probability. Combinatory: Probability on finite sample spaces. Joint and
conditional probabilities, independence, total probability; Bayes rule and
applications
UNIT-2: Random variables :
Definition of random variables, continuous and discrete random variables,
cumulative distribution function (cdf) for discrete and continuous random
variables; probability mass function (pmf); probability density functions (pdf)
and properties. Jointly distributed random variables, conditional and joint
density and distribution functions, independence; Bayes rule for continuous
and mixed random variables.
UNIT-3: Function of random a variable.
pdf of the function of a random variable; Function of two random variables;
Sum of two independent random variables. Expectation: mean, variance and
moments of a random variable. Joint moments, conditional expectation;
covariance and correlation; independent, uncorrelated and orthogonal random
variables. Random vector: mean vector, covariance matrix and properties.

V. R. Gupta, Assist.Prof., ET,


YCCE, Nagpur
UNIT-4: Some special distributions:
Uniform, Gaussian and Rayleigh distributions; Binomial, and Poisson
distributions; Multivariate Gaussian distribution
Vector-space representation of random variables, linear independence, inner
product, Schwarz Inequality
Elements of estimation theory: linear minimum mean-square error and
orthogonality principle in estimation; Moment-generating and characteristic
functions and their applications, Bounds and approximations: Chebysev
inequality and Chernoff Bound.

UNIT-5: Sequence of random variables and convergence:


Almost sure (a.s.) convergence and strong law of large numbers; convergence
in mean square sense with examples from parameter estimation; convergence in
probability with examples; convergence in distribution. Central limit theorem
and its significance.

UNIT-6: Random process


Basic definitions, important Random processes, continuous-time linear systems
with random inputs white noise, classification of random processes, WSS
processes and LSI systems.
V. R. Gupta, Assist.Prof., ET,
YCCE, Nagpur
Assignments
Assign. Date Last Date of
Topic Remark
No. given submission

1.
Based on Unit I & II
2.
Based on Unit III & IV
3. Based on Unit V and VI
(Self Learning topic)

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Text Books:
Probability and Random Processes, by H. Stark, J.W Woods,
Pearson Education, 2000.
Probability, Random Variables and Stochastic Processes, by
A. Papoulis, S. U. Pillai, McGraw Hill, 2002.
Reference Books:
Probability and Stochastic Processes, by R D Yates, D J
Goodman, John Wiley and Sons, 1992.

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Why study probability?

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
What is probability?
Mathematical model
Help us to study physical systems in an
average sense.

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Different Kinds of Probability
Probability as intuition

Probability as the Ratio of Favorable to Total


Outcomes (Classical Theory)
Probability as a Measure of Frequency of
Occurrence
Probability Based on an Axiomatic Theory
V. R. Gupta, Assist. Prof., ET,
YCCE, Nagpur
Probability as intuition
Deals with judgments based on intuition.

Eg: Joe will be wary about letting his nine-


year-old daughter Jane swim in the local pond,
if Frank reports that Bill thought that he might
have seen an alligator in it.

V. R. Gupta, Assist. Prof., ET, YCCE,


Nagpur
Probability as the Ratio of Favorable to Total
Outcomes (Classical Theory)
Probability of an event is computed a priori.

NE
where: NE is the no. of ways E can occur

N N is the no. of all possible outcomes.

Eg: we throw a pair of unbiased dice and ask what is


the probability of getting a seven?

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
1st Die
2nd
Die
1 2 3 4 5 6
1 2 3 4 5 6 7
2 3 4 5 6 7 8
3 4 5 6 7 8 9
4 5 6 7 8 9 10
5 6 7 8 9 10 11
6 7 8 9 10 11 12
The total no of outcomes is 36 if we keep the Dice distinct.
The number of ways of getting a seven is N 7 = 6.

6 1
P[ getting a 7]
36 6
V. R. Gupta, Assist. Prof., ET,
YCCE, Nagpur
Probability as a Measure of Frequency of
occurrence
The Probability of an event E is computed as.

nE
P[ E ] lim
n n
here n n and therefore 0 P[ E ] 1
E

We can never perform the experiment an infinite number of


times so we can only estimate P[E] from a finite number of
trials.
We postulate that approaches a limit as n goes to
nE n
infinity.

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Probability Based on an Axiomatic Theory
Experiment: an experiment is a procedure we perform that
produces some result. Often letter E is used to designate an
experiment. (e.g., the experiment E5 might consist of tossing a
coin five times)
Outcome: An outcome is a possible result of an experiment.
The Greek letter (xi ) is often used to represent outcomes.
(e.g., the outcome 1 of experiment E5 might represent the
sequence of tosses head-head-tail-head-tail; however, the
more concise HHTHT might also be used.)
V. R. Gupta, Assist. Prof., ET,
YCCE, Nagpur
Event: An event is a certain set of outcomes of an experiment.
(e.g., the event C associated with experiment E5 might be C={all
outcomes consisting of an even number of heads}) .

Sample Space: The sample space is the collection of set of all


possible distinct (collectively exhaustive and mutually exclusive)
outcomes of an experiment. The letter S is used to designate the
sample space, which is the universal set of outcomes of an
experiments.

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Set Theory
or
Set Algebra
Set: A set is a collection of objects called elements.

E.g., car, apple, pencil is a set whose elements are a car, an


apple & a pencil. The set heads, tails has two elements.
The symbol
denotes set inclusion and the symbol is the
opposite of
.
How to define a set?

A {RTM University , SGB niversity , PuneUniversity}


B {x 2 x 1, 2, 3,.....}

C {all RTM University students who arein Final Year BE}

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Subset: A subset B of a set A is another set whose elements are
also elements of A.

BA

Set Equality: A B
The definition of set equality is A B iff B A and A B

Universal set: It is a set of all things that we could possibly


consider in a given context. All set under consideration will be
subsets of a set S, which we shall call space or universal set.
V. R. Gupta, Assist. Prof., ET,
YCCE, Nagpur
Null set or empty set: By definition the set that contains no
elements is called as Null set or empty set. This is denoted by
{}.

S={set of all universities}

If a set consist of n elements, then the total number of its subsets


equals 2n .

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Set Operations

C B A or A B C

A B C Thus, for any A,


{} A S

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Unions and Intersections

A B or AUB
B
A This operation is comulative and associative.
AUB B U A
( A U B) U C A U ( B U C )

if B A, then, A U B A
AU A A
A U{} A
AUS S

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Unions and Intersections
AB or A I B
This operation is comulative ,

A B associative and distributive.


AB BA
( AB)C A( BC )
A( B U C ) AB U AC )
if A B, then, AB A
AA A
A{} {}
AS A
V. R. Gupta, Assist. Prof., ET,
YCCE, Nagpur
Mutually Exclusive Set
Two sets A and B are said to be mutually
exclusive or disjoint if they have no
B common elements, that is, if
A
AB {}

several sets A1 , A2 , A3 ......... are called mutually exclusive if

Ai Aj {} for every i and j i

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Partitions
A partitions U of a set S is a collection
A1
of mutually exclusive subsets Ai of S
whose union equals S.
B
A2 An

A1 U ........ U An S where Ai Aj {} j i
Thus,
U [ A1 ,........, An ]

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Complements
The complement of a set A is the

set consisting of all elements of S


A
that are not in set A. Thus

AU A S AA {} A A S {}

if B A, then B A;
if A B, then B A;
V. R. Gupta, Assist. Prof., ET,
YCCE, Nagpur
Demorgans Law
AB

A U B AB

AB A U B
AB

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
If in a set identity we replace all sets by their
complements, all unions by intersections, and all
intersections by unions, the identity is preserved.

A( B U C ) AB U AC

A( B U C ) A U B U C A U BC
AB U AC ( AB )( AC ) ( A U B )( A U C )

A U BC ( A U B )( A U C )

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Duality Principle
We know that, S {} and {} S .
If in an set identity all overbars are removed, the identity
is preserved.
If in a set identity we replace all unions by intersections,
all intersections by unions, and the sets S and {} by the
sets {} and S, the identity is preserved.

A( B U C ) AB U AC S UA S
A U BC ( A U B )( A U C ) {} A {}
V. R. Gupta, Assist. Prof., ET,
YCCE, Nagpur
c
AUB AI B A

A B A B A B
V. R. Gupta, Assist. Prof., ET,
YCCE, Nagpur
Probability Space
The space, S or is called the certain event, its
elements experimental outcomes, and its subsets
events.
The empty set {} is the impossible event.
The event { i} consisting of a single element i is an
elementary event .

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
In the applications of probability theory to physical
problems, the identification of experimental outcomes is
not always unique.

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
In the relative frequency interpretation of various results, we shall
use the following terminology:

Trial: A single performance of an experiment will be called a trial.

An event A occurs during this trial if it contains the element {i}

The certain event occurs at every trial.

The impossible event never occurs.

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Axiomatic definition of Probability
Probability is a set function P[.] that assigns to every event E a
number P[E] called the probability of E such that
1. P [ E ] 0

2. P [ ] = 1

3. P [ E U F ] = P [ E ] + P [ F ] If EF=

These conditions are the axioms of the theory of probability

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Properties
The probability of the impossible event is zero.
4. P[ ] 0
5. P[ EF ] P[ E ] P[ EF ]
c

6. P[ E ] 1 P[ E c ]
7. P[ E U F ] P[ E ] P[ F ] P[ EF ]
8. P[ E ] P[ B] P[ AB ] c
iff B A
These conditions are the axioms of the theory of probability

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur
Equality of Events
Two events A and B are called equal if they consist of the same
elements.

They are called equal with probability 1 if the set AB c U Ac B


consisting of all outcomes that are in A or in B but not in AB
has zero probability. .
AB c U Ac B

V. R. Gupta, Assist. Prof., ET,


YCCE, Nagpur

You might also like