Muhammad Ali

Lecturer in Statistics GPGC Mardan.

M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

Introduction
Definition of Econometrics
literally interpreted econometrics means "economic measurement". Econometrics may be defined as
the social science in which the tools of economic theory, mathematics, and statistical inference are
applied to the analysis of economic phenomena(variable).Econometrics can also be defined as
"statistical observation of theoretically formed concepts OR alternatively as mathematical economics
working with measured data. so economic theory attempts to defined the relationship among different
economic variables.

Methodology of Econometrics
Following are the main steps in methodology of econometrics
1. Specify mathematical equation to describe the relationship between economic variables.
2. Design methods and procedures based on statistical theory to obtain representative sample
from the real world.
3. Development of methods for estimating the parameters of the specified relationships
4. Development of methods of making economic forecast for policy implications based on
estimated parameters.

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

What are the goals of econometrics

Econometrics help us to achieves the following three goals:
1. Judge the validity of the economic theories.
2. Supply the numerical estimates of the co-efficient of the economic relationships which may be
then used for some sound economic policies.
3. Forecast the future values of the economic magnitude with certain degree of probability.

The Nature of Econometrics Approach

The first step of every econometrics research is the specification of the model, a model is simply a set of
mathematical equations. If the model has only one equation it is called a single-equation model.
Whereas if it has more than one equation it is called a multi-equation model. Now let us consider the
following model.
Y=0 + 1X
where
Y= Consumption expenditure
0 = Intercept
1= Slope or co-efficient of regression
X= income
This is a deterministic model showing the relationship between consumption and income.
The non-deterministic or stochastic models can be written as:
Yi=0 + 1Xi+ i

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

where " i " is known as the disturbance or residual term, and hence it is a random variable. It is

also

called probabilistic model. The disturbance or error term represent all those factors

affect

consumption but not taken into account. This equation is an example of an

that

"Econometric

Model".

In other words it is an example of a "linear regression Model". In this case the response

variable 'Y' is

linearly related to the predictor variable 'X' but the relationship between the two is

not exact. The

second step is the estimation of model by appropriate econometric method, this will

include

the

following steps.
1. Collection of Data for the variables included in the model.
2. Choice of appropriate econometric technique for the estimation of technique used is Regression
Analysis in Statistics).
3. The third step is to develop the suitable criteria to find out whether estimates obtained are
in agreement with the expectations of the theory that has been tested, that is to decide whether
the estimates of the parameters of the theoretically meaning full and statistically significant.
4. The final step is to use the estimated model to predict the future value of the response variable.

Deterministic and Stochastic Models

A relation between X and Y is said to be deterministic if for each value of predictor variable X,
there is one and only one corresponding values of response variable Y. On the other hand the
relation between X and Y is said to be stochastic or probabilistic if for a predictor value of X 4
there is a whole probability distribution of values of Y, that is 'Y' is a random variable and 'X' is a
fixed mathematical variable measured without error.

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

Types of Econometrics
Econometrics can be divided into two main braches
1. Theoretical
2. Applied

1. Theoretical econometrics:
It is concerned with development of the appropriate methods for measuring the economic relationship
specified by the econometrics models. This type of econometrics depends much on mathematical
statistics, single equation and simultaneous equations techniques, the methods used for measuring
economic relationships.

Y = a + bx simple
Y = a + bx0 + bx 2 Simul tan eous
Y = a + bx1 + bx 2 simula tan eous

2. Applied econometrics:
Applied econometrics Describe the practical value of economic research. It deals with the applications of
econometric methods developed in the theoretical econometrics to the different fields of economics
such as the consumption functions, demand and supply, fraction etc. The applied econometrics has
made it possible to obtained numerical results from these studies which are of great importance to the
planners.

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

The Role of Econometrics:

The important role of econometric is the estimation and testing of economics models. The first step in
the process is the specification of the model in the mathematical form. The 2nd step is the to convert
the relevant data from the economy. The thirdly use the data to estimate the

parameters

of

the

models and finally we will carry out tests on the estimated model in an attempt to judge whether it
constitutes a sufficiently realistic picture of the economy being studied or whether somewhat different
specification is to be estimated.

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

Regression Analysis
Definition of Regression
Regression analysis is statistical technique for investigating and modeling the relationship
between variables. The term regression was first time introduced by "Francis Galton". In his
paper Galton found that the height of the children of unusually tall or unusually short parents
tends to move towards the average height of the population. Galton law of universal regression
was confirmed by his friend Karl Pearson, more than a thousand records of heights of members
of family groups. He found that the average height of sons of a group of tall fathers was less
than their father height and the average height of sons of a group of tall fathers was less than
their fathers height and the average height of sons of a group of short fathers was greater
than their fathers height, thus "regressing tall and short sons alike toward the average height of
all men. In the world of Galton this was "regression to mediocrity".

Modern Interpretation of Regression

Regression analysis is the study of the dependence of one variable the dependent variable, on
one or more other variables, the explanatory variable.

Objective of Regression
The objective of regression analysis is to estimate or predict the average value of the response
variable on the basis of the known or fixed values of the predictor variable.

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

The Simple Linear Regression Model

Simple linear regression is the most commonly used technique for determining how one
variable of interest(the response variable)is affected by changes in another variable(the
explanatory variable)The terms "response" and "explanatory" mean the same thing as
"dependent" and "independent", but the former terminology is preferred because the
"independent" variable may actually be interdependent with many other variables as well.
Simple linear regression is used for three main purposes:
1. To describe the linear dependence of one variable on another.
2. To predict values of one variable from values of another, for which more data are available.
3. To correct for the linear dependence of one variable on another, in order to clarify other
features of its variability. Linear regression determines the best-fit line through a scatter plot of
data, such that the sum of squared residuals is minimized; equivalently, it minimizes the error
variance. The fit is "best" in precisely that sense: the sum of squared errors is as small as
possible. That is why it is also termed "Ordinary Least Squares" regression. Model of the simple
linear regression is given by:
Yi=0 + 1Xi+ i
Where
0 = Intercept
1= Slope or co-efficient of regression

i =random error

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

An important objective of regression analysis is to estimate the unknown parameters 0 and 1 in the
regression model. This process is also called fitting the model to the data, The parameters 0 and 1 are
usually called regression coefficients. The slope 1 is the change in the mean of the distribution of Y
producing a unit change in X. The intercept 0 is the mean of the distribution of the response variable Y
when X=0.

Estimation of the parameters by OLS(ordinary least squares)

Ordinary least squares (OLS) or linear least squares is a method for estimating the unknown parameters
in a linear regression model. This method minimizes the sum of squared error or residual.
Mathematically the sum of square of error can be w
2

^
^

s = ei = y i xi I

i =1
n

Differentiating equation ( I ) with respect to and

^
^

xi = 0

i
^

^
^

= 2 y i xi xi = 0

simplifying the above equations we get the following normal equations:

^

Yi = n + X i ii
^

Yi X i = X i + X i iii
These are called normal equations.
From equation (ii) we get:
8

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

=Y X
Now substituting the value of in equation ( ii ) we get:
^

Yi X i = (Y X ) X i + X i

Yi X i = Y X i X X i + X i
^

Yi X i Y X i = X i X X i
^

It is important to know that

X iYi Y X i
2
Xi X Xi
X iYi Yi X i / n
2
2
X i ( X i ) / n

and

are not the same as

and

because they are based on a single

sample rather than the entire population. If you took a different sample, you would get different values
for

and

. Let's call

and

the OLS estimators of

and

. One of the main goals of

econometrics is to analyze the quality of these estimators and see under what conditions these are good
estimators and
d under which conditions they are not. Once we have
more
ore

The

variables.

second

is

The

the

first

estimates

is

of

the

the

fitted

error

terms,

and

values,

which

we

, we can construct two

or

estimates

will

call

of y:

the residuals:

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

Assumptions of the Classical linear Regression Model:

Following are the few important assumptions of the classical linear regression model:
1. Linearity: The regression model is linear in the parameters. i.e.
Yi

=0 +1Xi + ui

2. Non stochastic X: Values of the independent or repressor variable assumed to be fixed in

the repeated sampling.
3. Zero mean of the error term: The expected value or mean of the random disturbance
term given the value of X is zero. i.e.
E ( Ui/Xi) = 0
4. Homoscedasticity: Variance of the ui for all the observations remain the same.i.e.
V ( u1) = 2 V(u2) = 2 V(u3) = 2

V(un)=2

10

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

5. No Autocorrelation
correlation between error term: There is no correlation between any two
error terms, symbolically:
COV( i , j )=0

6. No relationship between predictor va

variable
riable and error term.
E(ui,Xi)=0
7. The number of observations 'n' must be greater than the number of

parameters to be estimated. Alternatively,, the number of observations 'n' must be

greater than the number of explanatory variables. The values of predictor
tor variable (X) in a given
sample must not all be the same. Technically Var(X) must be a finite positive number.

11

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

8. The regression model is correctly specified. Alternatively, there is no specification

bias or error in the model used in empirical analysis. There is no perfect multicollinearity. That is
, there are no perfect linear relationships among the explanatory variables.

Interpretation of Coe
cients/Parameters
The interpretation of the coe
cients
cients from a linear regression model is fairly straightforward.
straig
The
estimated intercept (b0) tells us the value of y that is expected when x = 0. This is often not very useful,
because many of our variables dont have true 0 values (or at least not relevant ones
ones-like education or
income, which rarely if ever have 0 values). The slope (b1) is more important, because it tells us the
relationship between x and y. It is interpreted as the expected change in y for a one-unit
one
change in x.

Properties of Least -Square

Square Estimators:
1. The least square estimators are lin
linear
ear function of the actual observation on Y.
we have
=

i =1

i =1

( Xi X ) ( Y Y ) / ( Xi X )2
n

Yi (Xi--- Xi ) -- Yi

( Xi X ) /

i =1

( Xi X )2
i =1

Yi (Xi--- Xi ) /

i =1

( Xi X )2 ;
i =1

12

as

i =1

(Xi--- Xi ) = 0

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

Wi

Where

xi /

Similarly we have

= y x
=

/ --(

) X

[1 / n X w ] Y
n

Hence both

and

2. The least square estimators

i =1

, are expressed as linear function of the Y's.

and

, are unbiased estimators of and .

we have
n

= Wi Yi
i =1

wi ( + Xi + i)
i =1

Now

i =1

i =1

i =1

wi = xi / xi 2

i =1

i =1

i =1

------------ I )
wi + wi xi + wi i ------------(

i =1

i =1

= ( xi x) / xi = 0 -----------(( A )
2

n
n
n

2
2
2
2
wi = xi / xi = xi /( xi ) 2 = 1 / xi ----------( B )

i =1
i =1
i =1
i =1
i =1

n
x
i
w
x
=

i i
2

i =1
i =1 x i
n

xi
( x i ) =
=1
2

x
i

13

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

n equation ( I )
Substituting these values in
^

= (0) + wi ( xi x ) + w i i

wi xi + x wi + wi i

[1 + 0] + wii

+ wii

-----------( I )

Taking expectation on both sides we get

g

^
E

Then
Now

+ wi E ( i )

+0

since E ( i ) = 0

is an unbiased estimate of .

1 / n xw i Yi

1 / n xw ( + xi + i )

1
1
1

+ x i + i xwi xwi xi xwi i

n
n
n

1
x 1
+
+ i x wi x wixi x wii
n
n
n

1
(n ) + x + 1 i x(0) x(1) x wii
n
n

By using equation A , B , and C.

=

1
n

+ i x wi i

Taking expectation on both sides:

14

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)
E( )

Thus

Econometrics
BS Economics

+ E ( i ) x wi E ( i )

+0+0

1
n

since E( i ) = 0

is an unbiased estimator of .

15

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

Multiple Linear Regression Model

Definition
A linear regression model that involves more than one predictor variable is called
multiple linear regression model. In this case the response variable is a linear function of
two or more than two predictor variables. A multiple linear regression model with "p"
predictor variables is given by:

Yi = o + 1 X 1 + 2 X 2 + ... + p X p + i

0 , 1 ,..., p are parameters and to be estimated from the sample data. These
parameters are also called regression coefficients, the parameters

j ( j = 1,2,3... p ) represent the expected change in the response Y and percent change
in Xj, where all the remaining predictor variables Xi's (ij) are held constant. For the
reason these parameters are

often called partial regression coefficients. A multiple

linear regression model with two predictor variables is given by:

Yi = 0 + 1 X 1 + 2 X 2 + i
The corresponding regression model estimated from sample data is given as:
^

Y = 0 + 1 X1 + 2 X 2
^
^

Where 0 , 1 , and 2 are estimates of the parameters 0 , 1 , and 2 .

16

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

Ordinary Least Square Criteria to find Estimates

The least square function

i =1

^
^

Yi 0 + 1 X 1 + 2 X 2

^
^
^

Yi 0 1 X 1 2 X 22 --------------( I )

e i2

^
^

The function 'S' is to be minimized with respect to 0 , 1 , and 2 . For this purpose we have to
^
^

differentiate Equation ( I ) with respect to 0 , 1 , and 2 .

(S )
^

( 0 )

2
ei = ^
^

0 i =1
0

(S )
^

( 1 )

^
^
^

i
0
1
1
2 X 2 = 0 -------( II )

i =1
n

^
^
^
n 2
n

e
=
Y

i
0
1
1
2 X 2 = 0 ------( III)
^
^ i

1 i =1 1 i =1
2

(S )

2
=
ei = ^
^
^

( 2 ) 2 i =1
2

17

^
^
^

i
0
1
1
2 X 2 = 0 ----( IV )

i =1
n

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

From ( II )
^

^
^
2 Yi 0 1 X 1 2 X 2 ( 1) = 0

i =1

i =1

i =1

i =1

i =1

Yi 0 1 Xi 2 X 2 = 0
i =1

i =1

Yi = 0 + 1 Xi + 2 X 2
n

i =1

Yi = n

i =1

+ 1 X 1 + 2 X 2 (V )

i =1

From ( III )
^
^
^

2 Yi 0 1 X 1 2 X 2 ( X 1 ) = 0

Yi X 1 0 X 1 1 X 1 2 X 1 X 2 = 0
^

Yi X 1 = 0 X 1 + 1 X 1 + 2 X 1 X 2 (VI )
2

^
^
^

From ( IV ) 2 Y i 0 1 X 1 2 X 2 ( X 2 ) = 0

Yi X 2 0 X 2 1 X 1 X 2 2 X 2 = 0
^

Yi X 2 = 0 X 2 + 1 X 1 X 2 + 2 X 2 (VII )
2

Equation ( V ), ( VI ), and ( VII ) are called normal equations.

From ( V ) we get
18

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

Yi / n = n / n
i =1

+ 1 X1 / n + 2 X 2 / n

Y = 0+ 1 X1 + 2 X 2
^

0 = Y 1 X1 2 X 2
substituting value of 0 in equation ( VI) and ( VII) we get
^
^
^
^
2

From (VI ) Yi X 1 = Y 1 X 1 2 X 2 X 1 + 1 X 1 + 2 X 1 X 2

^
^
^
^

2
From(VII ) Yi X 2 = Y 1 X 1 2 X 2 X 2 + 1 X 1 X 2 + 2 X 2

Yi X 1 Y X 1 = 1 X 1 X 1 X 1 + 2 X 1 X 2 X 2 X 1 A
^

Yi X 2 Y X 2 = 1 X 1 X 2 X 1 X 2 + 2 X 2 X 2 X 2 B
Now

)(

x1 y = X 1 X 1 Y Y

)
(

x1 y = X 1 Y Y X 1 Y Y

x1 y = X 1 Y Y , sin ce
Y Y = 0
x1 y = X 1Y Y X 1
x1 = ( X 1 X 1 ) 2
2

x1 = ( X 1 X 1 ) X 1 X 1
2

19

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

x1 = X 1 X 1 X 1 X 1 X 1 X 1
2

x1 = X 1 X 1 X 1 sin ce ( X 1 X 1 ) = 0
2

Similarly
x 2 = X 2 X 2 X 2 , sin ce ( X 2 X 2 ) = 0
2

)(

x2 y = X 2 X 2 Y Y

x2 y = X 2 Y Y X 2 Y Y

)
(

x 2 y = X 2Y Y X 2 ; sin ce Y Y

Substituting these results in equation A and B we obtained the normal equations in

deviation form as follow:1

Solving the above normal equations for 0^ and 1^ i.e. multiplying equation ( C ) by
x22 an equation( d ) by x1x2 and subtract it, we will get the following estimates
of 1.
^

x1 y x 2 2 x 2 y x1 x 2

2
2
x1 x 2 ( x1 x 2 )2

1 =

Similarly multiplying equation ( C ) by ' x1x2 ' and equation ( d ) by x12 and
subtracting we will get
^

x 2 y x1 2 x1 y x1 x 2

2
2
x1 x 2 ( x1 x 2 )2

2 =

20

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

Standardized coefficients:
In statistics, standardized coefficients or beta coefficients are the estimates resulting from an analysis
carried out on independent variables that have been standardized so that their variances are. Therefore,
standardized coefficients refer to how many standard

deviations a dependent variable will change,

per standard deviation increase in the predictor variable. Standardization of

the coefficient is usually

done to answer the question of which of the independent variables have a greater effect on
the dependent

variable in

a multiple

regression analysis,

when

the

variables

are

measuredindifferent unitsof (forexample, income measuredin dollars and familysize measuredin numbe
r of individuals). A regression carried out on original (unstandardized) variables produces unstandardized
coefficients. A regression carried out on standardized variables produces standardized coefficients.
Values for standardized and unstandardized coefficients can also be derived subsequent to either type
of analysis. Before solving a multiple regression problem, all variables (independent and dependent) can
be standardized. Each variable can be standardized by subtracting its mean from each of its values and
then dividing these new values by the standard deviation of the variable. Standardizing all variables in a
multiple regression yields standardized regression coefficients that show the change in the dependent
variable measured in standard deviations.

Advantages
Standard coefficients' advocates note that the coefficients ignore the independent variable's scale of units,
which makes comparisons easy.

21

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)

Econometrics
BS Economics

Disadvantages
Critics voice concerns that such a standardization can be misleading; a change of one standard deviation
in one variable has no reason to be equivalent to a similar change in

another

variables are easy to affect externally, e.g., the amount of time spent on an action.
cholesterol level are more difficult, and some, like height or age, are

predictor.

Some

Weight

or

impossible to affect externally.

Goodness of Fit (R2)

The Coefficient of Determination, also known as R Squared, is interpreted as the goodness of fit of

regression. The higher the coefficient of determination, the better the variance that the dependent
variable is explained by the independent variable. The coefficient of

determination is the overall

measure of the usefulness of a regression. For example, if R2 is 0.95. This means that the variation in the
regression is 95% explained by the independent variable. That is a good regression.

Now,

if

the

Coefficient of Determination, or R2,is 0.50. Its means that the variation in the regression is 50%
explained by the independent variable. This is not a good regression. Note that R2 lies between '0' and
'1'. If R2=1, it means that the fitted model explains 100% of the variation in response variable 'Y'. On the
other hand if R2=0, the model does not explain any of

the variation of 'Y'. The Coefficient of

Determination can be calculated as the Regression sum of squares, RSS, divided by the total sum of
squares,SST
Coefficient of Determination =

RSS
TSS

Mathematical formula of the coefficient of Determination is given as under:

22

Muhammad Ali
Lecturer in Statistics GPGC Mardan.
M.sc (Peshawar University)
Mphil(AIOU Islamabad)
^

R =
2

Econometrics
BS Economics

)(

)(

1 X1 X 1 Y Y + 2 X 2 X 2 Y Y

Yi Y

Problems with the coefficient of Determination

First, let's consider that the Coefficient of Determination will increase as more independent variables are
added. It does not matter if those independent variables help to explain the variation of the dependent
variable, the R Square (Coefficient of Determination) will increase as more independent variables are
added. This brings us to the concept of Adjusted R Squared. The adjusted R Squared takes into account
only the independent variables that assist in explaining the variation of the dependent variable.
The adjusted R Squared is different than the Coefficient of Determination, because the adjusted R
Squared will only increase if the independent variables are helpful in an explanatory nature. The
adjusted R Squared may be negative and must be lower than the original R Square (original Coefficient
of Determination).

23