St.

Mary’s University
Faculty of Business and
Economics
Department of Economics
Course Name- Econometrics
Course Code- ECON3112
Credit Hour- 4
Instructor Name -Dereje Mengist
October, 2024

SMU Econometrics for Finance

10/9/2024 1
 Chapter two: The Classical
Linear Regression
Model(CLRM)
2.1. Methods of Least Squares
• Terminology and Notation
Y=f(x)

10/9/2024 SMU Econometrics for Finance 2

 Cont’d
• Dependent variable: the variable that is influenced
by the independent variable(s).
• For example, in a Multiple Linear Regression
Model(MLRM), output is influenced by
independent variables like fertilizers cost, labor
cost, pesticides cost etc.
• Independent variable: a variable, whose values
does not depend up on other variable, but
influences dependent variable.
• Examples include, fertilizers cost, pesticides cost
etc.

10/9/2024 SMU Econometrics for Finance 3

 2.2. Simple and Multiple
Regressions
Regression Analysis
• Economic theories are mainly concerned with the
relationships among various economic variables.
• These relationships, when phrased in mathematical
terms, can predict the effect of one variable on
another.
• The functional relationships of these variables
define the dependence of one variable upon the
other variable(s) in the specific form.
• The specific functional forms may be;
• linear, quadratic, logarithmic, exponential,
hyperbolic, or any other form.

10/9/2024 SMU Econometrics for Finance 4

 Cont’d
• The concept of regression Analysis

The objective of linear

regression analysis: is to
estimate and/or predict the
mean or average value of the Regression analysis:
dependent variable on the basis
of the known or fixed values of
the explanatory variables.

It is with a view to estimate and/or That is to estimate the

It is concerned with the predict the (population) mean or
study of the dependence of average value of the dependent in
population regression function
terms of the known or fixed (in (PRF) on the basis of sample
one variable on one or more repeated sampling) values of the regression function (SRF) as
other variables. latter. accurately as possible.

10/9/2024 SMU Econometrics for Finance 5

 Cont’d

• Simple, or two-variable, regression analysis: if we are

studying the dependence of a variable on only a
single explanatory variable.
– E.g. consumption expenditure on real income
• Multiple regression analysis: if we are studying the
dependence of one variable on more than one
explanatory variable.

10/9/2024 SMU Econometrics for Finance 6

 Simple Linear Regression

• Represented by single equation regression model

• Y = f(x)
• The dependent variable expressed as a function of
only a single explanatory variable
• Causal relationship between variables flow in one
direction only.
• Example:

10/9/2024 SMU Econometrics for Finance 7

 Multiple Linear Regression

• Dependent variable explained by more than one

explanatory variables.
• Example; Y=f( X, Z, K, O)
• Regression equation of Y on X.
• Variation in C = systematic variation + random
variation.
• Consumption=f( Income, Wage rate)

10/9/2024 SMU Econometrics for Finance 8

 Important forms of relation
• In this chapter we shall consider a simple linear
regression model.
• i.e. a relationship between two variables related in a
linear form.
• We shall first discuss two important forms of relation:
• stochastic and non-stochastic,
• Note: among which we shall be using the former in
econometric analysis.

10/9/2024 SMU Econometrics for Finance 9

 Stochastic and Non-stochastic
Relationships
• Econometricians say relationship between variables
(X and Y) are generally in exact (stochastic).

10/9/2024 SMU Econometrics for Finance 10

 Cont’d

A. Non-Stochastic Model:
• A relationship between X and Y, characterized as Y =
f(X) is said to be deterministic or non-stochastic if for
each value of the independent variable (X) there is
one and only one corresponding value of dependent
variable(Y).
• Example:
• Without the error/or disturbance term (u), the
relationship is said to be exact/deterministic,
otherwise stochastic or inexact.

10/9/2024 SMU Econometrics for Finance 11

 B. Stochastic /Inexact
Relationship
• A relationship between X and Y is said to be
stochastic if for a particular value of X there is a
whole probabilistic distribution of values of Y.
• In such a case, for any given value of X, the
dependent variable Y assumes some specific value
only with some probability.
• Stochastic model: is a model in which the dependent
variable is not only determined by the explanatory
variable but also others variables which are not
included in the model.
• Example:

10/9/2024 SMU Econometrics for Finance 12

 Cont’d

• Existence of the disturbance is justified in the following

points:
• Omission of other variables
• Measurement error/data collection difficulties.
• Randomness in human behavior
• Imperfect specification of the model
• Poor proxy variable
• Note: In regression analysis we are concerned with a
stochastic or statistical relationship and not of a
deterministic or non stochastic or mathematical
relationship.

10/9/2024 SMU Econometrics for Finance 13

 Cont’d
• Assume a supply function where by the supply for a certain
commodity depends on its price (other determinants taken to
be constant) and the function being linear, the relationship can
be put as:

• For a particular value of P, there is only one corresponding

value of Q.
• This is a deterministic (non stochastic) relationship since for
each price there is always only one corresponding quantity
supplied.
• All the variation in Q is due solely to changes in P, and that
there are no other factors affecting the dependent variable.

10/9/2024 SMU Econometrics for Finance 14

 Cont’d
• If this were true all the points of price quantity pairs,
if plotted on a two-dimensional plane, would fall on a
straight line.
• However, if we gather observations on the quantity
actually supplied in the market at various prices and
we plot them on a diagram we see that they do not
fall on a straight line.

10/9/2024 SMU Econometrics for Finance 15

 Cont’d

Note: The derivation of the observation from the line

may be attributed to several factors.
• Omission of variables from the function
• Random behavior of human beings
• Imperfect specification of the mathematical form of
the model
• Error of aggregation
• Error of measurement

10/9/2024 SMU Econometrics for Finance 16

 Cont’d
• To take into account the above sources of errors we
introduce in econometric functions a random variable
which is usually denoted by the letter ‘u’ or ‘ℇ’
• And is called error term or random disturbance or
stochastic term of the function,
• So called because ‘u’ is supposed to ‘disturb’ the
exact linear relationship which is assumed to exist
between X and Y.
• By introducing this random variable in the function
the model is rendered stochastic of the form:

10/9/2024 SMU Econometrics for Finance 17

 Cont’d
• Thus a stochastic model is a model in which the
dependent variable is not only determined by the
explanatory variable(s) included in the model but also
by others which are not included in the model.
• In order to take all these sources of error into account,
we introduce the stochastic/random disturbance term
into our econometric models and hence the complete
simple econometric model becomes:
• Yi= α + βXi + εi
• Variation in Y = Explained Variation + Unexplained
Variation
10/9/2024 SMU Econometrics for Finance 18
 Simple Linear Regression
•
model
A simple linear regression model: It is a relationship
between two variables related in a linear form.
• The true relationship which connects the variables
involved is split into two parts:
– a part represented by a line and
– a part represented by the random term ‘u’.

10/9/2024 SMU Econometrics for Finance 19

 Cont’d
• The scatter of observations represents the true
relationship between Y and X.
• The line represents the exact part of the relationship
and the deviation of the observation from the line
represents the random component of the relationship.
• These points diverge from the regression line by U1
,U2,…….Un.

– The first component is the part of Y explained by

the changes in X and
– The second is the part of Y not explained by X,
that is to say the change in Y is due to the random
influence of ui.
10/9/2024 SMU Econometrics for Finance 20
 Con’t

10/9/2024 SMU Econometrics for Finance 21

 2.3. Assumptions and
Properties
• The objective of a regression analysis is not only
estimate the unknown parameters, β‘s, (coefficients ).
• Y = f(X) + U = Β0+ Β1X + Ui
• The classical made important assumption in their
analysis of regression.
– Some assumptions are related to Y and X
– Some assumptions are related to X and X
– Some assumptions are related to U
• The most important of these assumptions are
discussed as follows.

10/9/2024 SMU Econometrics for Finance 22

 A. The model is linear in
parameters
• The model should be linear in the parameters
regardless of whether the explanatory and the
dependent variables are linear or not.
• This is because if the parameters are non-linear it is
difficult to estimate them since their value is not
known but you are given with the data of the
dependent and independent variable.

10/9/2024 SMU Econometrics for Finance 23

 Cont’d
• Check yourself whether the following models satisfy
the above assumption or not.

• Linearity in variables implies that an equation is

linear model if it is expressed in a straight line.
• The parameters are raised to their first degree.
Note: Linear regression means linear in parameter but it
may not be linear in the explanatory variable.

10/9/2024 SMU Econometrics for Finance 24

 B. Ui is a Random Real Variable
• This means that the value which u may assume in any
one period depends on chance;
– it may be positive,
– negative or
– zero.
– Every value has a certain probability of being
assumed by u in any particular instance.

10/9/2024 SMU Econometrics for Finance 25

C. Zero Mean Value of the Error
term
• That is; given the value of X the mean or expected
value of the disturbance term is zero.
• Technically, the conditional mean value of ε is zero.
• Mathematically,

10/9/2024 SMU Econometrics for Finance 26

 D. The variance of the random
variable(U) is constant in each
period: (The assumption of
Homoscedasticity )
• Equal variance of the error term. Given the value of X, the
variance of the error term(u) is the same for all observations.
• The variation of each ℇi around all values of the explanatory
value is the same.
• The dispersion of the disturbance is the same.

• This constant variance is called homoscedasticity assumption and

• The constant variance itself is called homoscedastic variance.

10/9/2024 SMU Econometrics for Finance 27

E. The Random Variable (U) has
a Normal Distribution
• This means the values of u (for each x) have a bell
shaped symmetrical distribution around their zero
mean and constant variance

• Normality Test

10/9/2024 SMU Econometrics for Finance 28

 F. The random terms of
different observations (Ui , Uj)
are independent. (The
assumption of no
autocorrelation)
• This means the value which the random term
assumed in one period does not depend on the value
which it assumed in any other period.

10/9/2024 SMU Econometrics for Finance 29

 G. The random variable (U)
is independent of the
explanatory variables.
• There is no correlation between the random variable
and the explanatory variable.
• If two variables are unrelated their covariance is zero.

10/9/2024 SMU Econometrics for Finance 30

 H. The explanatory variables
are measured without error
• Y = f(X) + Ui
• U absorbs the influence of omitted variables and
possibly errors of measurement in the y’s.
• i.e., we will assume that the regressors are error free,
while y values may or may not include errors of
measurement.

10/9/2024 SMU Econometrics for Finance 31

 Additionally
• The regression model is correctly specified.
• There is no perfect multicollinearity (this holds in the
case of multiple linear regression model).
• The number of observations ( n ) must be greater than
the number of parameters ( k ) to be estimated (in
multiple linear regression
• Assumption of dependent variable:
• We have two assumptions of dependent variables:
• The dependent variable Yi is normally distributed and
• Successive values of the dependent variable are
independent.
10/9/2024 SMU Econometrics for Finance 32
 Methods of Estimation
• Specifying the model and stating its underlying
assumptions are the first stage of any econometric
application.
• The next step is the estimation of the numerical
values of the parameters of economic relationships.
• The parameters of the linear regression model can be
estimated by various methods.
• Three of the most commonly used methods are:
• Method of moments (MM)
• Ordinary least square method(OLS)
• Maximum likelihood method (MLM)
• But, here we will deal with the OLS methods of
estimation.
10/9/2024 SMU Econometrics for Finance 33
 The Ordinary Least Squares
(OLS Method)
• The model Yi= α + βXi + εi is called the true
relationship between Y and X because Y and X
represent their perspective population value, and α
and β are called the true parameters since they are
estimated from the population value of Y and X.
• But it is difficult to obtain the population value of Y
and X because of technical or economic reasons.
• So we are forced to take the sample value of Y and X.
• The parameters estimated from the sample value of Y
and X are called the estimators of the true parameters
α and β and are symbolized as

10/9/2024 SMU Econometrics for Finance 34

 con’t

10/9/2024 SMU Econometrics for Finance 35

 con’t

• Meaning, the residuals should be small.

• Therefore, when assessing the fit of a line, the vertical
distances of the points from the line are the only
distances that matter.
• The OLS method calculates the best-fitting line for a
data set by minimizing the sum of the squares of the
vertical deviations from each data point to the line
(the Residual Sum of Squares, RSS)
• Minimize RSS= ei2
• We will used differential calculus

10/9/2024 SMU Econometrics for Finance 36

 Cont’d
• Why the sum of the squared residuals?
• Why not just minimize the sum of the residuals?
• To prevent negative residuals from cancelling
positive ones.
• If we use, all the error terms ei would receive equal
importance no matter how closely/ widely scattered
the individual observations are from SRF.

10/9/2024 SMU Econometrics for Finance 37

 Con’t

10/9/2024 SMU Econometrics for Finance 38

 Con’t

• Rearranging we will get:

• Divide both sides by “n”:

• Rearranging the above equation we obtain:

10/9/2024 SMU Econometrics for Finance 39

 Cont’d
• Substituting the values of α we get:

10/9/2024 SMU Econometrics for Finance 40

 Con’t

Example 1:
• First find the intercept and slope of a function
• Write the mathematical relationship between x and y

10/9/2024 SMU Econometrics for Finance 41

 Con’t
Example 2:
• You are given a data on saving and income of five
households as follows:

• The slope varies. But we need to establish a linear r/hip

between x and y.
• The relationship is not exact.
• So math’s failed to do so. But econometrics can make it.
How?

10/9/2024 SMU Econometrics for Finance 42

 Cont’d

• Given a relationship of the form: -𝑌𝑖=𝑎+𝑏𝑋𝑖+ε𝑖

10/9/2024 SMU Econometrics for Finance 43

 Cont’d
• Solving for Example 2: -
i Y = Saving X = Income X2 XY
1 200 500 250,000 100,000
2 100 300 90,000 30,000
3 600 1000 1,000,000 600,000
4 700 800 640,000 560,000
5 400 450 202,500 180,000
Total 2000 3050 2,182,500 1,470,000
Mean 400 610

• 𝑏= 𝑋𝑌−n𝑥𝑦 𝑋2−𝑛𝑥2= 1,470,000−5∗610∗4002,182,500−5∗6102=

250,000/322,000 = 0.78
• 𝑎=𝑦−𝑏𝑥=400−0.78∗610=−73.5
• The Y = -73.5 + 0.78X Is the estimated regression line.

SMU Econometrics for

10/9/2024 44
Finance
 Statistical Properties of Least
Square Estimators
• There are various econometric methods with which
we may obtain the estimates of the parameters of
economic relationships.
• We would like to an estimate to be as close as the
value of the true population parameters i.e. to vary
with in only a small range around the true parameter.
• How are we to choose among the different
econometric methods, the one that gives ‘good’
estimates?
• We need some criteria for judging the ‘goodness’ of
an estimate.

10/9/2024 SMU Econometrics for Finance 45

 Cont’d
• ‘Closeness’ of the estimate to the population parameter is
measured by the mean and variance or standard deviation of
the sampling distribution of the estimates of the different
econometric methods.
• We assume the usual process of repeated sampling i.e. we
assume that we get a very large number of samples each of
size ‘n’;
• We compute the estimates β’s from each sample, and for
each econometric method and we form their distribution.
• We next compare the mean (expected value) and the
variances of these distributions and
• We choose among the alternative estimates the one whose
distribution is concentrated as close as possible around the
population parameter.
10/9/2024 SMU Econometrics for Finance 46
 Con’t

10/9/2024 SMU Econometrics for Finance 47

 Multiple Linear Regression
Analysis
• In simple regression we study the relationship between a
dependent variable and a single explanatory (independent
variable);assume that a dependent variable is influenced by
only one explanatory variable.
• However, many economic variables are influenced by
several factors or variables.
• For instance;
• In decision to investment studies we study the relationship
between quantity invested (or either to invest or not) and
interest rate, share price, exchange rate , etc.
• The demand for a commodity is dependent on price of the
same commodity, price of other competing or
complementary goods, income of the consumer, etc.
10/9/2024 SMU Econometrics for Finance 48
 Cont’d
• Hence the two variable model is often inadequate in practical
works.
• Therefore, we need to discuss multiple regression models.
• The multiple linear regression is entirely concerned with the
relationship between a dependent variable (Y) and two or more
explanatory variables (X1, X2,…, and Xn).
• Why do we need multiple regression?
1. One of the motivation for multiple regression is the omitted
variable bias in the simple regression analysis.
• It is the primary drawback of the simple regression but multiple
regression allows us to explicitly control for many other factors
which simultaneously affect the dependent variable.

10/9/2024 SMU Econometrics for Finance 49

 Example: wages vs.education
• Imagine we want to measure the (causal) effect of an
additional year of education on a person’s wage.
• If we want to the model: wage = β0 + β1 educ + u and
interpret β1 as the ceteris paribus effect of educon wage,
we have to assume that educ and u are uncorrelated.
• Consider a different model now: wage = β0 + β1 educ +
β2 exper + u, where exper is a person’s working
experience(in years).
• Since the equation contains experience explicitly, we will
be able to measure the effect of education on wage,
holding experience fixed.
• Multiple regression analysis is also useful for
generalizing functional relationships between variables.

10/9/2024 SMU Econometrics for Finance 50

 Simple Regression vs. Multiple
Regression
• Most of the properties of the simple regression model
directly extend to the multiple regression.
• We derived many of the formulas for the simple
regression model; however, with multiple variables,
formulas can get difficult when explanatory variables
are more than two.
• As far as the interpretation of the model is concerned,
there’s a new important fact : the coefficient βj
captures the effect of jth explanatory variable,
holding all the remaining explanatory variables fixed.

10/9/2024 SMU Econometrics for Finance 51

 Con’t

• Multiple regression analysis is an extension of simple

regression analysis to cover cases in which the
dependent variable is hypothesized to depend on
more than one explanatory variable.
• Much of the analysis will be a straight forward
extension of the simple regression model.
• In multiple linear regression, we have one dependent
variable Y, and k number of explanatory variables.
• The relationship between a dependent & two/more
independent variables is linear in parameters, and
may not be linear in variables.

10/9/2024 SMU Econometrics for Finance 52

 Con’t

10/9/2024 SMU Econometrics for Finance 53

What changes as we move from
simple to multiple regression?

• Potentially more explanatory power with more

variables;
• The ability to control for other variables; (and the
interaction of various explanatory variables:
correlations and multicollinearity);
• Harder to visualize drawing a line through three or
more (n)- dimensional space.
• The R2 is no longer simply the square of the
correlation coefficient between Y and X.

10/9/2024 SMU Econometrics for Finance 54

 Con’t

10/9/2024 SMU Econometrics for Finance 55

 Assumptions of the Multiple
Linear Regression
• In order to specify our multiple linear regression
model and proceed our analysis with regard to this
model, some assumptions are compulsory.

• But these assumptions are the same as in the single

explanatory variable model developed earlier except
the assumption of no perfect multicollinearity. These
assumptions are:

10/9/2024 SMU Econometrics for Finance 56

 Con’t

SMU Econometrics for

10/9/2024 57
Finance
 Con’t

10/9/2024 SMU Econometrics for Finance 58

Model With Two Explanatory
Variables
• Estimation of parameters of two-explanatory
variables model

10/9/2024 SMU Econometrics for Finance 59

 con’t

• Since the population regression equation is unknown

to any investigator, it has to be estimated from sample
data.
• Let us suppose that the sample data has been used to
estimate the population regression equation.
• We leave the method of estimation un specified for
the present and merely assume that the equation has
been estimated by sample regression equation, which
we write as:

10/9/2024 SMU Econometrics for Finance 60

 Estimation of Multiple
Regression( An OLS
approach)
• Given sample observation on Y, X1,, & X2, we
estimate the model using method of least square
(OLS)

10/9/2024 SMU Econometrics for Finance 61

 Con’t

10/9/2024 SMU Econometrics for Finance 62

 Con’t

10/9/2024 SMU Econometrics for Finance 63

 Con’t

10/9/2024 SMU Econometrics for Finance 64

 Con’t

10/9/2024 SMU Econometrics for Finance 65

 DM

End of chapter 2

10/9/2024 SMU Econometrics for Finance 66