UNIT 1 MATHEMATICAL MODELLING -

AN OVERVIEW
Structure Page No

1.1 Introduction
Objectives
1.2 Model Classifications
1.3 Mathematical Modelling - What &id Why?
1.4 Classifying Mathematical Models
1.5 Limitations of a Mathematical Model
1.6 Summary
1.7 SolutionslAnswers

1. INTRODUCTION
Mathematics is a very effective tool in solving real world problems. The
critical step in the use of mathematics for solving real world problems is the
building of a suitable mathematical model. A mathematical model is a
conversion of a real world problem into an abstract mathematical problem
involving mathematical concepts such as constants, variables, functions,
equations, inequalities, etc. The process by which a real world problem is
represented and inteipreted in terms of a mathematical model is called
mathematical modelling. Our main aim in this unit is to introduce you to the
I basic concepts of mathematical modelling and discuss the process of
development of a mathematical model.

Translation into a mathematical model is one of the many approaches to

solving real-world problems. Other approaches include experimentations with
! physical models or schematik models or with the red world directly. The
mathematical approach, which is our main concern throughout this come, has
I a number of advantages which we shall be illustrating through examples.
However, to give you a broader idea about various models, we shall start the
I unit by discussing model classifications in Sec. 1.2. In Sec. 1.3 we shall
discuss the process of translating a real world problem into a mathematical
I problem. The need for mathematical modelling is also illustrated in this
section through various examples. Sec. 1.4 introduces you to different types of
, modelling.

Objectives

/I After studying this unit, you should be able to

define a mathematical model;

1t realise the need for developing a mathematical model;
!
1
translate a real world problem into its equivalent mathematical formulation;
I

I identify the type of modelling to be used for a given problem.

i
+ Introduction to
Mathemancal Modelling 1.2 MODEL CLASSIFICATIONS
f
B
A model is an abstraction of reality or a representation of a real object or
situation. It could be a simple drawing of office plans or a complicated
functional representation of a complex machinery part. A model airplane may
be assembled from children's kit, or it may actually contain an engine and a
rotating propeller that enable it to fly like a real plane.

Some models are replicas of the physical properties (relative shape, form, and
weight) of the object they represent. Some are physical models but do not have
the same physical appearance as the object of their representation. Other type
of models deal with symbols, expressions, mathematical equations and
inequalities. Each of these models can be classified into four main categories:
physical models, schematic models, verbal models and mathematical models.

Physical Models

Physical models are prototypes models that look like the objects they represent.
They are more or less the exact replicas of the object being modelled. Scale
models of Taj Mahal, airplanes, buses, ships, ofice complexes, shopping
centres, homes, etc. look exactly like their counterparts but in much smaller
scale. The advantage in having a scaled model is that one can tell exactly what
the object under study looks like, in three dimensions, before making a major
investment. Also, some of these models can even perform as their counterparts
would and this allows you to conduct the study on the model to see how it
might perform under actual operating conditions. Scaled models of airplanes
can be tested in wind tunnels to determine aerodynamic properties and the
effects of air turbulence on their outer surfaces. Models of bridges and dams
can be subjected to multiple levels-of stress from wind, heat, cold and other
factors to test their effects as endurance and safety. Scaled models that behave
in a manner similar to the real objects are less expensive to create and test than
their actual counterparts.

Schematic Models

Schematic models are more abstract than physical models. They do not look
like the physical reality they represent. Graphs and charts are schematic
niodels that provide pictorial representations of mathematical relationships.
Mathematical linear relationship between two variables may be indicated by
plotting a line on a graph. Pie charts, bar charts and histograms can all model
some real situations but /donot bear any physical resemblance to them.
Diagrams, drawings, blueprints and a flow chart describing a computer
program are all exampllzs of schematic models.

Verbal Models

Verbal models use words to represent some object, situation or problem that
exists, or could exist, i.n reality. This could be a simple word presentation of
scenery described in a book to a complex business problem (described in words
and numbers). Verbal. models provide all relevant and necessary information
to solve the problem, make recommendations and suggest alternatives. The
case studies which you must have studied from management text books are
examples of verbal r~nodelsthat expose you to the workings of a business
without having to visit the firm's actual premises. Often these verbal models Mathematical Modelling
provide enough information to be converted into mathematical models.
- An Overview

Mathematical Models

Mathematical models are the most abstract of the four classifications. These
models do not look like their real-life counterparts at all. Mathematical models
are built using numbers, symbols, variables, empirical laws related together by
means of equations or equalities. Mathematical models can take many forms
like statistical models, optimization models, algebraicldifferential equations or
game theoretic models, etc. In this unit and units to follow, we shall
concentrate on mathematical models. We shall be discussing in detail the
process and need of mathematical modelling, types of mathematical models
and we shall formulate some models with the contexts taken fkom biology,
physics, economics, finance, medicine, etc. Before we discuss basic concepts
of mathematical modelling in this unit, you may try the following exercise.

El) Give two examples each of the physical, schematic, verbal and
mathematical models.

Let us now discuss the process of mathematical modelling.

- -

1.3 MATHEMATICAL MODELLING - WHAT AND

WHY?
Mathematics is a rich and interesting discipline. It provides a set of ideas and
tools effective in solving problems which arise in other fields like history,
philosophy, sciences, sociology, political science, life sciences, medical
sciences, engineering, etc. It also provides concepts useful for theoretical
approach in other fields. Mathematics may be applied to specific problems
already posed in mathematical form, or it may be used to formulate such
problems. In theory construction, mathematics provides abstract structures A 'system' is a
which may be used as tools in understanding problems arising in other fields. collection of one or
Problem formulation and theory construction involve a process knovm as more related objects
mathematical modelling or, mathematical model building. A mathematical i.e., physical entities
with specific
model, as we have already mentioned, is an abstract model that uses characteristics or
mathematical language to describe a system. It can be viewed as a attributes.
representation of the essential aspects of an existing system (or a system to be
constructed) which presents knowledge of that system in useable form.

Mathematical modelling usually begins with a situation in the real world.

These situations may arise in different disciplines like engineering, physics,
physiology, psychology, ecology, wildlife management, chemistry, economics,
sports etc. A psychologist, for example, observes certain types of behaviour in
rats running in a maze, a wildlife ecologist notes the number of eggs laid by
endangered sea turtles, or an economist records the volume of international
trade under a specific tariff policy. Each t i e s to observe and predict'fuhue
behaviour. Their efforts may be based completely on intuition, but often they
are the result of detailed study, experience, and observing the similarities
between the current situation and other situations which are better understood.
This study of the system, the accumulation and organisation of information and
stating the problem to be studied is in fact the first step in model building.
Introduction to The next step is to make the problem simpler by making certain assumptions
Mathematical Modelling and approximations. It requires to identify and select the conceptdinformation
to be retained in the problem. For example, the psychologist studying rats in a
maze may decide that it makes no difference that all the rats are black or that
the maze is cons'tructed of wood. On the other hand, it may be significant for
her that all the rats are siblings and the maze is divided into compartments. He
may also assume that a rat is always in exactly one compartment and never half
in one and half in another compartment.

The third step in modelling is to replace the real quantities and processes by
mathematical symbols, a set of variables and a set of equations/inequalitiesthat
establish relationships between these variables. The values of the variables can
be practically anything; real or integer numbers, Boolean values or strings, for
example. The variables represent some properties of the system, for example,
we may measure system outputs often in the form of signals, timing data, event
occurrence (yedno). The actual model is the set of functions that describe the
relations between the different variables.

After the problem is formulated, the fourth step is the study of the resuiting
mathematical system using appropriate mathematical tools and techniques.
This may involve a calculation, solving an equation, proving a theorem, etc.
The motivation is to produce new information about the problem being studied.
It is likely that new information can be obtained by using well-known
mathematical concepts and techniques. If not, we may need to develop new
techniques or adopt tested methods fiom other disciplines.

The final step in the model-building process is the evaluation of a

mathematical model i.e., comparison of the results predicted on the basis of the
mathematical analysis with the real world. It is important to know whether or
not our model gives reasonable answers i.e., Does our model reflect all the
important aspects of the real world problem? If a model is not accurate
enough, then, we need to refine our model. We may need a new formulation, a
new mathematical analysis and hence a new evaluation. It usually happens that
the model-building process proceeds through several iterations, each a
refinement of the preceding, until, finally, an acceptable one is found. Broadly,
we can divide the modelling process as follows:

Formulation which involves the following three steps:

i) Studying the problem: Accumulating and organising the

information about the real world problem. Describing the context
of the problem and stating the problem within this context.

ii) Identifying relevant information: Identifying and selecting the

conceptdinformationwhich are significant and to be retained in the
problem.

iii) Mathematical representation: Replacing real quantities and

processes by mathematical symbols, a set of variables and a set of
equations/inequalitiesthat establish relationship between variables.

Mathematical Analysis which studies the formulated problem using

appropriate mathematical tools and techniques.
 Evaluation which decides whether the model and its analysis explain the lWodelling
phenomenon/problem we are interested in. If it is not a good model then - An Overview
we need to refine it. There is no unique or correct model; but there are
good models and bad models. The skill of modelling lies in being able to
judge which is which.

Pictorially, we can represent the process of mathematical modelling as shown

in Fig. 1 below.

Formulation

I ( Mathematical Analysis
I
a Evaluation

Stop

Fig. 1

After a model is evaluated and found gocd, there are several uses of the model:
i) it helps in our better understanding of 'he real physical system, ii) it can
serve as a tool in the prediction of the fbture state of the system which is
currently unknown, iii) ir. can help in doing trial experiments by changing the
parameter values or in perturbing the system to produce a desirable condition,
i.e., many unknown parameter values can be estimated.

Let cs now look at some simple mathematical models. We shall not go into the
details of their formulations here since you must be already familiar with them.

Example 1 1 Lotka- Volterra Equations

The simplest model of interacting populations was proposed independently by

Lotka in I 925 in the United States and by Volterra in 1926 in Italy. If x and y
are respectively, the predator and prey populations, then their model is in terms
of the following equations:

a,p, y, 6 are positive constants.

y and p represent constant specific birth rate of the prey and mortality rate of
the predator respectively.
Introduction to The predator's specific growth rate a y depends on availability of the prey as
Mathematical Modelling
food source, whereas, the prey death rate 6x depends on the number of
predators. These equations hdve oscillatory solutions but have the
unsatisfactory feature of forming a conservative system, and oscillations of any
magnitude are possible. More realistic versions of this model remove this
degeneracy. This model is discussed in detail in Unit 9, Block-3 of our
undergraduate course MTE- 14 on 'mathematical modelling'.
***
Example 2: Model of a Particle in a Box
In physics, the particle in a box (also known as the infinite potential we8 or
the infinite square well) is a problem consisting of a single particle inside an
infinitely deep potential well, h m which it cannot escape, and which loses no
energy when it collides with the walls of the box. In one dimensional case, the
problem can be described as a single point particle enclosed in a box inside
which it experiences no force i.e., it is at zero potential energy. At the walls of
the box, the potential rises to infinity, forming an impenetrable wall.
In classical mechanics, the problem can be modelled using Newton's laws of
motion and the solution to the problem is trivial: The particle moves in a
straight line, always at the same speed, until it reflects h m a wall. A
quantum-mechanical solution of the problem becomes very interesting and
reveals some decidedly quantum behaviour of the particle that agrees with
observations but contrasts sharply with the predictions of classical mechanics.
According to quantum theory, particle has no definite position or velocity.
Rather, a probabilistic interpretation is given to the state of the particle in terms
of a time-independent wave h c t i o n yl(x) . The square of the wave function
( yl 12.,is a probability density.

The problem of a particle situated in a 1-dimensional infinite square well with

momentum only in the direction of quantum confinement (the x-direction) is
described by the schrodinger eqnation, a basic equation of quantum
mechariics, given by

where h is the reduced planck constant,

m is the mass of the particle,
yr(x) is the complex-valued stationary time-independent wave
function,
V(x) is the spatially varying potential and
E is the energy, a real number.
For the case of the particle in a I'-dimensional box of length L, the potential is
zero inside the box, but rises to i d n i t y at x = 0 and x = L . Thus, the equation
reduces to

the solution to which caj),befound easily under appropriate boundary

conditions. One of the b i b l e choices is
yr(0) = yl (L) = 0 .
 The motivation for this choice is that the particle is unlikely to be found at a
location with a high potential (the potential repulses the particle), thus the
probability of finding the particle, I ~ ( xj2,) must be small in these regions and
decreases with increasing potential. For the case of an infinite potential,
I ~ ( xl2) must be infinitesimally small or 0 , thus yr(x) must be zero in this
region. Thus, we get yr(0) = w(L) = 0.

Example 3: Model ofRational Behaviour for a Consumer

In this model, we assume that a consumer faces a choice of n commodities

labelled 1, 2, ..., n each with a market price p, , p,, ..., p, . The consumer is
assumed to have a cardinal utility function U (cardinal in the sense that it
assigns numerical values to utility), depending on the amounts of commodities
, ,
x , x , ..., xn consumed. The model further assumes that the consumer has a
budget M which she uses to purchase x,, x, ,...,x, in such a way as to
, .
maximise U (x, , x , ..., x ,) The problem of rational behaviour .in this
model then becomes an optimization problem, that is:
max U(x,, x 2 , ..., x,)
n
subjectto: pixi <M,xi 2 O V i=(1, 2, ..., n ) .
i =l
This model is used in general equilibrium theory, particularly to show
existence and optimality of economic equilibria. However, the fact that this
particular formulation assigns numerical values to levels of satisfaction is the
source of criticism. However, it is not an essential ingredient of the theory,
only an assumption.
***
And now an exercise for you.

E2) Give at least two examples of the formulas you are already familiar with
as the mathematical models of the real situations.

Why formulate a Mathematical Model?

As we mentioned earlier, the use of mathematics is one of many approaches for

i understanding and solvinc;a real world problem. Others include
experimentation with scaled physical models in a iaboratory on a smaller scale
simulating all the conditions of the real situation, or with the real world
directly. But these may be highly risky and costly as they may involve the use
of explosive and expensive chemicals or materials. On the other hand,
mathematical approach has many advantages. Mathematical modelling is very
inexpensive and provides systematic approach to problem solving. Once
developed properly, a great deal can be learnt about the real-life situation by
manipulating a model's variables and analysing the results. Mathematical
modelling requires users to accumulate and organize information and, in the
process, to indicate areas where additional information is needed, and hence
increase the understanding of the problem.
Introduction to Let us look at some of the problen~swhich illustrate the use of mathematical
Mathematical Modelling modelling.

For meteorological purposes, rockets are launched so that they r e k h a certain

altitude and record atmospheric conditions such as temperature and pressure.
In such cases we are confronted with the following problem:

Example 4: What is the rocket thrust level and duration necessary to ensure
that the rocket reaches the desired altitude?

For this problem, a solution, based on experimentation involving rocket

launches with different thrust-time, is unacceptable due to the cost and the
uncertainty of success. Further, the solution to this problem cannot be obtained
using scale-model experiments. For this kind of study, a mathematical
approach is preferred.

In large supermarkets you must have seen a number of checkout counters.

Customers prefer to stand in a queue which is the shortest. The problem to be
sorted out by the management is the following:

Example 5: What is the optimum number of check out counters that the
supermarket should have in order to maximize the expected profit?

Information of this kind is frequently needed for planning purposes. Fewer

counters imply long queue lengths and can affect customer goodwill. On the
other hand, if there are too many checkout counters, manning them reduces the
total profit. There is really no scientific alternative to a mathematical treatment
for problems of this kind.
***
You may now think of more situations like this where mathematicaI treatments
of the problem become necessary.

E3) Give two situations from the field of management where mathematical
treatment of the problem is necessary to get the required solution.

Mathematical models can be classified into different categories depending on

the mathematical structure of the underlying formulation. We shall discuss
these classifications in the next section.

1.4 CLASSIFYING MATHEMATICAL MODELS

According to the structure of the models we can classify mathematical models
into the following four types:

(i) Linear vs. Nonlinear

Mathematical models are usually composed by variables, which are

abstractions of quantities of interest in the described systems, and operators
that act on these variables, which can be algebraic operators, functions,
differential operators, etc. If all the operators in a mathematical model present Mafhemaflcal Modelling
linearity, the resulting mathematical model is defined as linear. A model is -An Overview

considered to be nonlinear otherwise.

For example, consider the equation

Eqn. (1) is a one-dimensional diffusion equation (also known as Fick's 11law)

where C (x, t) is the concentration of difhsing substance, x is the space
coordinate and D is the diffusion coefficient. This equation represents the
diffusion of a substance in the x direction due to a concentration gradient in
that direction. In some cases, e.g., difision in dilute solutions, D can be
approximated by a constant, while in other cases, e.g., difhsion in high
polymers, D depends on concentration C .

For the case when D is a constant, Eqn. (1) becomes

which is a linear differential equation and hence is said to be a linear model of

the problem. You may note that Eqn. (1) is a second order linear partial
differential equation with C as dependent variable and t and x as
independent variables. In general Eqn. (2) along with appropriate initial and
boundary conditions is solvable. You must have also come across this equation
and solved it in your differential equation course at the undergraduate level
(Ref. Unit 17, Block-4, MTE-08).

If, on the other hand, D is not a constant but depends on C ,say, for example, if
D = Do expb (C - C, )]where Do, P, C, are constants then Eqn. (1) reduces to

Eqn. (3) is a non-linear PDE and hence the model obtained is a non-linear one.

You may notice that if the model tries to replicate more closely the real
situation (i.e., D is a function of the concentration in this case), the
corresponding mathematics involved is more challenging. (solving a ,
non-linear PDE,in this case).

(ii) Static vs. Dynamic

A static model does not account for the element of time and hence the
variables and relationships describing the system are time-independent.
Consider, for instance, the transportation problem generally associated with
industries:

.
Suppose there are m origins Oi, i = 1, 2, .., m in an industry, where various
amount of a commodity are produced or stored for transportation to n
destinations D j, j = 1, 2, ..., n . It is then required to transport all units of the
commodity fiom all Oi ,exhaustively, to all D j , exactly satisfying all their
Introduction to requirements in such a way that the total transportation cost becomes
Mathematical Modelling minimum. The following assumptions are made:

i) The ith origin can supply exactly ai unit o'f the commodity.

ii) The jth destination can accept exactly b unit.

iii) The cost of transportation of one unit fiom Oi to D is Cij .

This transpoltation problem, therefore, is essentially a minimization problem.

Mathematically, if xi =number of units transported fiom Oi to D j , then we
want to
minimize Z = xx
m

j=l
n

j=l
Cij x i j

n
subject to x i j = a i , i = l , 2, ..., m
j=1

where x i j L O V i, j and a,, b j > O V i, j. (7)

Eqns.(4)-(7) define the transportation problem where all variables are

independent of time. Such a system is a static system. We shall discuss this
model in detail in Unit 5 of this course.

In contrast to the static system, in dynamic system, time plays a very important
role. The relationship between the variables describing the system changes
with time. If you look at the problem of rocket launch then it can be described
A system is in terms of a closed system consisting of two objects-the rocket and the earth.
openfclosed if the The variables describing the rocket are its position and velocity relative to
objects in the system some fixed point on the earth, and the interaction between the two objects is
dotdo not interact
with objects of the
given by the theory of dynamics. In this description we may as well include
super-system which the influence of other planets in the solar system by treating the planets as
do not belong to the belonging to the environment of the system and the system being open. In
system. either case, the variables- i.e. position and velocity of the rocket-change
continuously with time. Hence the system is a dynamic system.

(iii) Discrete vs. Continuous

Mathematical model may be discrete or continuous according as the variables

involved are discrete or continuous. In models involving changes over time,
the changes may take place continuously with time and the variables of the
system are described for all time instants over the interval of interest. On the
other hand, the changes may occur at discrete instants of time and the variables
described only for the re!evant time instants. Thus, one should be clear
whether to treat the time element as continuous or discrete. In certain
instances, the data available are such that a discrete treatment of time is more
appropriate as opposed to continuous. For instance, if a soft drink
manufacturing company is interested in estimating the weekly demand of the
soft drink, then the time element is a week and hence it has to be treated as a
discrete variable.
 Mathematical Model1ing
Models involving continuous variables are often expressed through differential -An Overview
equations-ordinaryor partial, whereas, those involving discrete charactMsation
result in difference equations. Consider for instance, the mathematical model,
referred to as the classical Malthusian population scheme for population
growth given by Thomas R. Malthus (1766- 1834). The model is based on the
idea that the population size for one generation depends on the size of the
previous generation. This is expressed mathematically by the following
equation
I Pt+l =rXPt (8)
where:
t :represents the time period (which could be minutes, hours, weeks,
years, etc.) depending on the species being considered
i
.
pt :represents the population size at time t The units of time could
be hours, days, years, etc.
p ,+,:represents the population size at the next time period. Again it
could be the next hour, next day, next year, etc.; and
r :referred to as the Malthusian factor, is the multiple that
determines the growth rate.

The model given by Eqn. (81 which is a difference equation, is a discrete

model. It allows you to find out the value of p at different discrete time
intervals say, at years 3, 4, 5, etc. You couldn't use this model to find out the
size of p when t = 3.578 because t = 3.578 does not represent a prescribed
discrete time. The value of r in Eqn. (8) has a strong impact on how fast the
population will grow. We shall be discussing this model and other discrete and
continuous population models in detail in Unit 4. Analogous continuous
model of the population growth resulting in differential equation is given by

(9)

where x(t) (> 0) is the size of the population at time t , x,, is the size of the
population at the initial time and r represents the net growth rate. For details
of this model refer to Unit 8, Block-3, MTE-14.

(iv) Deterministic vs. Stochastic

A system is said to be deterministic if the values assumed by the variables or

the changes in the variables are known with certainty. Consider for instance,
the problem of rocket launch considered in Example 4. The variables of t'le
system are the position and velocity of the rocket. The laws of classical
dynamics can be used to describe the motion fairly accurately and the changes
in position and velocity can be predicted with a high degree of certainty.
Hence, in this case we can view the system as Iieing deterministic. On the
other hand, in a stochastic model, randomness is taken into account and
variables are described by probability distributions instead of unique values.
For example, a manufacturer, having trouble deciding whether to build a large
or small facility knows that the solution to this capacity problem depends upon
the volume of demand. High demand would require a large facility while low
Introduction to ckmand would require a small facility. While the manufacmer has no way of
Mathematical Modelling knowing with certainty what the exact demand will be, he can, at least,
determine the probability of the occurrence of each (high or low). For
example, if the manufacturer estimates that the probability of the occurrence of
high dm~andis 70 percent and the occurrence of low demand is 30 percent, he
can use this information along with the monetary value (expected pay-off) of
each situation to construct mathematical models such as pay-off matrices to
find an optimal decision (see Table-I)

Table-1

High demand Low demand

(70%) (30%)
Large facility Rs.240,000 -Rs.60,000
Small facility Rs.120,000 Rs. 100,000

This type of model is said to be a stochastic optimization model. We shall be

discussing some optimization models in Unit 5. Some models of this type are
also discussed in Unit 12, Block4 of MTE-14.

Once again look at the problem of supermarket operation considered in

Example 5. The problem is to determine the optimum number of checkout
counters in a supermarket to maximize the expected profit. The model can be
successfbl once a relationship between some measure of queue (e.g. average
queue length or average time period for which the customer has to wait before
being served) and the number of checkout counters is obtained. The variables
characterizing the system are the number of customers, their arrival rate, their
departure rate, service time, peak period, etc. Here the arrival of customers,
departure of customers and their service time are all random. They cannot be
determined uniquely, rather they are given by certain probability distributions.
Stochastic models based on fitting these probability distributions to the arrival,
departure and service time can be obtained. For the details of some of these
madels you can refer to Unit 14, Block-4 of Mathematical Modelling course
(MTE- 14).

In real life, there is always uncertainty. If the uncertainty is insignificant then

it can be ignored and the system can be treated as a deterministic system. This
is a process of simplification. If the uncertainty is significant then it cannot be
ignored and must be taken into account while characterising the system.

As mentioned earlier, most of the discrete and stochastic models lead to

differencelalgebraic equations whereas the knowledge of algebraicldifferential
equations is required while dealing with lineulnonlinear, staticldynarnic, and
continuous models. The skills you have in algebra, calculus, analysis,
differential equations, optimization and probability theory will be useful for
successfblly dealing with mathematical modelling. The type of mathematics
required depends on the type of model formulated.

''oil may cl%wtry the following exercises.

 Mathematical Modelling

E4) State the types of modelling you will use for the following problems.
Also give reasons in support of your answer.
a) Human motion (e.g. walking, lifting, jumping) involves dynamic
action at one or more joints in the body. If the joints h c t i o n
normally they cause no discomfort otherwise they cause severe pain
when in motion. One way of reducing the pain and discomfort is to
replace the defective joint with an artificial one. For the artificial
joint to be effective it must be capable of executing all the motions
of a normal joint. The problem is to build a model to describe the
hctioning of a joint, say, hip joint, so that it is usefbl to
bioengineers when designing hip replacements.
b) Research and development (R&D) is an important activity in any
modem industrial organization. The R&D manager is faced with
the difficult problem of allocating limited resources (money,
manpower, space, etc.) between a number of competing projects.
The problem is to build a model to help the R&D manager to make
right decisions.
c) In a chemical process, the output quality and yield depend critically
on the levels assigned to relevant factors (temperature, pressure,
concentration, etc.) It is important to optimally select these levels
as well as to monitor and control the system to ensure that they stay
at the desired levels. The problem is to build a model to help
achieve this.
d) The formation of sand dunes and their encroachment into
de-forested lands near deserts has become a serious problem in
many parts of the world. It is needed to predict the spread of desert,
as well as to devise policies to control the spread. The problem is to
build a model to describe the movement of sand dunes.
e) Advertising is a means by which a manufacturer can promote the
product and improve sales and hence, the revenue generated.
However, advertising costs money and is worthwhile only if the
cost of advertising is less than the increase in the sale revenue. The
problem is.to evaluate the effectiveness of different advertising
policies and the selection of the optimal policy. The problem is to
build a model to help solve this problem.

1.5 LIMITATIONS OF A MATHEMATICAL

MODEL
Mathematical modelling is a multistage activity involving various concepts and
techniques. Ideally, a mathematical model ends by returning to its origin. We
need to check whether the model and its analysis explain the phenomenon we
are interested in. The whole art of mathematical modelling lies in its
self-consistency. A mathematical model is an adequate mathematical model if
it captures the salient features of the system associated with the problem and is
capable of yielding a meaningfbl solution to the original problem.

It is rare that we obtain an adequate mathematical model at the first attempt for
the problem under consideration. In general, an iterative procedure is needed
Introduction to where improvements are progressively made until an adequate mathematical
Mathematical Modelling model is obtained. The adequacy of a model is established by checking the
validity of the assumptions made in building the model and by the closeness of
the agreement between the behaviour of the model and the system under
consideration. For example, while developing a model to describe the motion
of a simple pendulum if the time interval of study is sufficiently small, so that
the energy loss due to frictional drag is very small, then the assumption that the
frictional drag is negligibIe is valid. However, if the time interval of study is
large then the assumption of negligible frictional drag is not valid, since the
effect of drag on the pendulum motion is cumulative.

In a rocket launch model, the assumption that the thrust generated by the rocket
is an impulse, is valid if the thrust lasts for a very small fraction of the total
flight time, something that is not known initially. Only after a first solution,
can this be checked and if the need be, the model has to be modified. This
emphasizes the iterative nature of modelling.
To end the unit we now give the summary of what we have covered in it.

1.6 SUMMARY
In this unit we have covered the following points:

1) Mathematical model uses mathematical language to describe a real world

problem.
2) Mathematical models are built using numbers, symbols, variables related
together by means of functions, formulas, empirical laws, equations or
equalities and can take many forms like statistical modeIs, optimization
model, algebraic, differential equations or game theoretic models, etc.
3) The process of mathematical modelling involves three main steps -
formulation, mathematical analysis and evaluation.
4) Depending on the mathematical structure of the underlying formulation,
mathematical models can be classified into lineadnonlinear,
static/dynamic, discrete/continuous and deterministic/stochastic models,
5) The skills in subjects like algebra, calculus, analysis, differential
equations, statistics optimization, matrix theory and probability theory are
useful for dealing with mathematical modelling. The type of
mathematics required depends on the type of model formulated.
6) A mathematical model is adequate if it captures the salient features of the
system associated with the problem and is capable of yielding a
meaningful solution to the original problem.
7) Mathematical modelling is an iterative process where improvements are
progressively made until wi adequate mathematical model is obtained. :

El) Physical model - models of comic book super-heroes which look

exactly like their counterparts but in much smaller
scale.
Schematic model - a diagram showing the sequence of activities that
must be maintained in an assembly-line balancing.
 Verbal model - word problems given in any mathematics book. Mathematical Modelling
Similar examples of various types may be given. - An Overview

Other similar examples !?om your surroundings or real life experience

may be given.

E2) You may give examples of familiar mathematical models from your own
experience.

E3) A firm that assembles computers and computer equipment is to start the
production of two new types of computers. Each type will require
assembly time, inspection time and storage space. The amounts of each
of these resources that can be devoted to the production of the computers
is limited. The manager of the firm likes to determine the quantity of
each computer to be produced in order to maximize the profit generated
by their scale. For this kind of study a mathematical approach is
preferred.

Examples of other similar situations may be given.

E4) a) Dynamic, continuous, deterministic.

b) Static, discrete, deterministic.
c) Dynamic, continuous, deterministic.
d) Dynamic, discrete, probabilistic.
e) Static, discrete, probabilistic.
 UNIT 2 MODEL FORMULATION
Structure Page No
I 2.1 Introduction
Objectives
2.2 Essentials of A Problem
2.3 Models from Finance
Markowitz Model
Sharpe Model
24 Summary
2.5 Solutions/Answers

2.1 INTRODUCTION
In Unit 1 we introduced you to the need of modelling a real life situation and
the role of mathematics in it. Various steps involved in modelling a real life
situation were discussed there. We classified the mathematical models into
various types and illustrated them through examples from population
dynamics, optimization problem, queueing system, etc. In this unit, we shall
proceed with the next step in modelling i.e., given a real world problem, how
do you convert it to model abstraction leading to a mathematical equation? We
shall draw your attention to the various modelling aspects which need to be
taken into account while formulating a mathematical model.

In Sec. 2.2, we shall discuss the method of identifying the essentials of a

problem which need to be incorporated into the model. In Sec. 2.3, we shall
focus on an example from the field of finance, formulate a model and estimate
the financial implications and risks associated with reality. Whenever you
decide to make some investment in the market, your concern is to choose an
investment strategy which maximizes your profit and minimizes the risk
involved. But how to select such a strategy? Solution to this problem as
provided by Markowitz will be discussed in Sec. 2.3. Sharpe's model which
simplifies the rigours involved in various computations using Markowitz's
model will also be discussed here.

After studying this unit, you should be able to:

identify the essentials of a given real world problem;
formulate a model incorporating all the essentials required for the

obtain a solution of the formulated problem and interpret the result.

In addition to the above objectives, using the discussed model from the field of
finance, you should also be able to
i) explain the meaning and calculation of expected return and risk measures
for an individual security;
ii) explain the portfolio selection problem of investments;
Introduction to iv) explain what the efficient frontier is and how important it is to inve
Mathematical Modelling analysis.

2.2 ESSENTIALS OF A PROBLEM

Mathematical modelling involves transforming some idealised form of the real-
world situation into mathematical terms. It is, therefore, an activity which
requires more than the ability to just solve complex sets of equations. Though
there is no hard and fast approach to develop a model, you still need to broadly
follow the following steps in the beginning.

1) Establish a main purpose: Models rarely replicate a system and also

they can mean different things to different people. For example, a
business man and a biologist have entirely different points of view of
looking at the camel as depicted in Fig. 1.

ANATOMY I

I MILK f

MQ.1: Mflerent conce~tualviews.

Their conceptual views of the same system or object are rather different
since they are both heavily influenced by their own environment,
background and objectives. For example, a biologist would be interested
in the anatomy and physiology of a camel whereas, a business man's
concern would be the profit he can e m h m it. The same is true when
we come to the mathematical modelling of any system or process.
Although the motivation for building the model is usually to find the
means to answering a particular question, the form of that question
influences the way in which we build the mathematical model.

In the case of the familiar problem of the motion of a simple pendulum,

what is our main purpose? It is to f h d the period of oscillation, or,
angular velocity of the bob, or, we may want to know the tension in the
string. Thus, before developing a model we must be clear about the
purpose of doing it.
I
 Model Formulation
ii) Sifting essentials from the non-essentials: Real situations are quite
complex. Developing a model which accounts for all the aspects of a real
world phenomenon is likely to be difficult, mathematically complex and
unmanageable. On the other hand, it may also be possible that a model
not accounting for all the aspects of the phenomenon under consideration,
is easier to develop and handle and it may still serve our purpose. It is,
therefore, necessary to keep a balance between the complexity of the
problem and the objectives of our study. For example, while studying the
motion of a simple pendulum, if you assume the oscillations to be small
and restrict the range of the values of the amplitude 0 , to say 1 8 1 S 308,
then you only need to study a linear model and hence solve a linear
d28 g
differential equation -7 + -6 = 0, where 4! is the length of the
dt e
pendulum and g is acceleration due to gravity. On the other hand, if you
have a problem in which you cannot assume the oscillations to be small
then you need to study a non-linear model resulting in a non-linear
d20
differential equation tT + g sin 6 = 0 (ref Unit 3, Block-1 of MTE- 14).
dt

Similarly, a realistic study for understanding the population dynamics of

an infectious disease is one which differentiates the population by age,
gender, geographical location, social stratification and takes into
consideration factors like immigration, emigration, birth, death,
incubation period of the infection and removal of infectives from
circulation. This study will be definitely superior but more complex. The
model developed would involve more dependent variables and hence
more number of differential equations to be solved as compared to the
model where, the population considered is assumed to be closed,
homogeneously mixed, with no incubation period of infection and no
removal of infectives kom circulation.

The search for essentials of the problem is related to the main purpose
of the model. For instance, the assumption of no removal of infectives in
general, is highly restrictive in the context of a human population.
However, this might be a reasonable approximation to the early stages of
some upper respiratory infections where a long time may elapse before an
infective is removed fi-om circulation. A mild cold infection in a
classroom may also be considered as an example of no removal in a
human population. On the other hand, the assumption of no removal is
fairly applicable to epidemics in insect and plant populations. In the case
of a common h g a l disease of flies, the diseased or dead fly remains
attached to a leaf or a blade of grass, creating a situation roughly
analogous to no removal. The dead or diseased plants in a forest are
rarely removed, and if dead plants are infectious, the assumption of no
removal is fulfilled. If you are interested in knowing more about
modelling the spread of infectious diseases, you may refer to Unit 10,
Block-3 of h4TE- 14.

iii) Significance of essentials: Once the essentials of a problem are

identified, they determine the number and type of the variables in the
formulation and hence the complexities involved in problem solving.
The variables involved may be discrete, continuous or piece-wise
25
Introduction to continuous, deterministic, stochastic, fuzzy or any combination of these.
Mathematical Modelling For example, biological population of a given species of large sized
animals is an example of deterministic and discrete quantity. Flow of
money in a market through organized financial institutions is
deterministic and continuous. The sale of long sized items like trucks,
car, aeroplanes per year is discrete and stochastic. Flow of water id a
canal may be piecewise continuous when @ up stream gates are opened
or closed according to rainfall in the catchment area. The response level
in a human neural network is a fuzzy quantity.

As we have already mentioned in Unit 1, the types of variables involved

in a formulation determine the type of model formulated and hence the
type of mathematics required to solve the problem. We shall be
illustraticg this point in the next section through an example. But, before
that, you may try the following exercises.

El) State the type of modelling you will use for the following problems
giving reasons for your answers. List the essentials and non-essentials in
the problems.
i) The economic viability of an insurance company depends critically
on its ability to assess risks and decide on the premium charged to
cover risks. If the premium are low, then payouts can exceed
revenue collected and the company can go bankrupt. On the other
hand, if they are high, the number of customers will go down, thus
affecting profitability. The problem is to develop a model to help
the insurance company decided the premium it should charge for
different risks to ensue economic viability and maximise its profits.
ii) A company manufacturing soft drinks is thinking of expanding its
plant capacity so as to meet future demand. The monthly sales for
the past 5 years are available. The problem is to develop a model to
obtain good estimates for future demand so as to help the company
make the right decisions.

E2) Give examples of at least two real life situations where mathematical
treatment of the problem is the only approach to find the solution of the
problem. Why do you think that there is no other scientific alternative for
the treatment of these problems? List the essentials and non-essentials in
these problems.

E3) Give at least two examples each of physical situations in which the
variables involved are i) continuous and stochastic; ii) piece-wise
continuous and stochastic; iii) hzzy.

Let us now illustrate the mathematical formulation of real world situations

h m the field of finance.

2.3 MODELS FROM FINANCE

All of us make some or the other kind of investments to secure our future, old
age, for our children's education, their marriage etc. People invest in shares,
stocks, bonds, mutual funds, etc. using an investment strategy (called security)
 related to the expeAed market scenario in the future. Our aim is to select a Model Formulation
security or a group of securities (i.e., a portfolio) which maximize the expected
return and minimize the uncertainty (i.e., risk). But then the problem is how to
choose such a security or a portfolio? Markowitz's model, named after Harry
Max Markowitz, an economist, who is best know for his pioneering work in
Mordern Portfolio theory, provides a solution to this problem. He studied the
effects of asset risk, correlation and diversification on expected investment
portfolio return. We shall discuss this model here, but before that let us define
some of the terms which are the essentials for the model under consideration.

Securities

When you borrow some money or take a loan fkom a broker then you have to
leave some item of values as security with the broker or sign a piece of paper
promising repayment with interest. Failure to repay the loan (plus interest)
means that the broker can sell your item to recover the amount of the loan (plus
interest) and perhaps make a profit. The terms of agreement are recorded when
the deal is made. This piece of paper serving as evidence is called a security.
Similarly, you may have some spare money which you would like to lend to
earn some profit out of it. You then think of investing your money in
Government Bonds, saving certificates, shares, mutual funds, etc. These
investment strategies are called the securities. Broadly speaking, a security
helps us to save our h d s in the event of default (and that is why the name). It
may be a simple promissory note, share of the common stock, a bond, etc.

Return

Once the investment has been made, you are interested in knowing how good
was your investment strategy. For this, you need to calculate the rate of return
on the investment strategy. What is the rate of return? It gives a relation
between the initial input and final output of an investment and is calculated as
follow.
End of period value - Beginning of period value
. . (1)
Return =
Beginning of period value
I
i Risk

[
Risk denotes the probability of specific eventualities which may have both
I beneficial and adverse consequences. However, in general usage the
convention is to focus only on potential negative impact of the investment
strategy. Often, it is described as a situation which would lead to negative
consequences.

Uncertainty

It is the lack of certainty. You may have limited knowledge and it is

impossible for you to exactly describe the existing state or future outcome of an
investment strategy. For example, there is always an uncertainty while
investing in the stock market or mutual funds.
Introduction to Portfolio
MatRematical Modelling
Suppose you want to invest an amount Woin n securities (say). Let wi be
the proportion of Wothat you invest in the i th security (1 5 i 5 n) . Then the

orderedn-tuple P = ( w , , w,, ..., w,), where z

n

i=l
wi =l,iscalledaportfolio

- n securities and wi is its ith portfolio value (or weight).

of

( : :)
For example, P = 0, -, - is a portfolio of three securities where no amount
1
is invested in the first security, -rd of the total funds are invested in the
3
2
skcond and -rd in the third security. By changing the value of wi in P ,
3
subjected to the condition that zn

i=l
w = 1, all the portfolios of given n

securities i.e., a feasible set can be obtained. Formally, we give the following
definition.

Definition: The set of all the possible portfolios which can be constructed from
a given set of securities is called the feasible set or the opportunity set.

, ,
Given a portfolio P = (w , w , ...,w ,) of n securities, our main purpose is
to see the,effect of portfolio values w on the terminal value of the return on
the portfolio P . But each w is a certain proportion of the initial funds that are
invested in ith security of the portfolio P . Thus, for quantifying the return and
risk of the portfolio P ,we have to calculate the return and risk of its
constituting n securities. It is therefore, important to select the proportions w
, ,
of the initial funds in such a way that the portfolio P = (w , w ,...,w ,) is
optimally good according to our investment objectives. Such a portfolio
which provides an investor the maximum level of satisfaction is called an
Optimal Portfolio and the problem of selecting such a portfolio is referred to
as portfolio selection problem. Thus, as a first step towards modelling we
state the formulated problem as follows:

Portfolio Selection Problem: From a set of feasible portfolios of risky

securities, how can an investor choose a portfolio that gives h i d e r the
maximum level of investment satisfaction (in terms of return and risk)?

Before we take up the problem you rnay try the following exercises.
- --

E4) At the end of year 2005, Mohm decided to invest Rs.30,000 in a portfolio
of stocks and bonds. Rs. 10,0'00were put into common stocks and
Rs.20,000 into corporate bonds. At the end of 2006, Mohan's stock and
bond holdings were worth R.s.13,000 and Rs.16,000, respectively.
During 2006, Rs.500 in cash dividends was received on the stocks and
Rs. 1000 interest payments was received on the bonds. What was the
percentage return on
 Mohan's stock portfolio during 2006? Model Formulation
i)
ii) Mohan's bond portfolio during 2006?
iii) Mohan's total portfolio during 2006?

E5) Given a set of eight securities with portfolio values

WilS = 0, - - .
- - - - - Find a feasible set of portfolio of these
3 ' 4' 2' 3 ' 2 ' 4 ' 4
securities.

Let us now discuss the Markowitz's approach to solving the portfolio selection

2.3.1 Markowitz Model

Markowitz's approach to portfolio selection begins by assuming that an
investor has a given sum of money to invest at the present time. This money
will be invested for a particular length of time, known as the investor's holding
period. Markowitz's approach is a single period approach where at the
beginning of the period say t = 0 , the investor must make decision on what
particular securities to purchase and hold until the end of the period say t = 1.
Since a portfolio is a collection of securities, this decision is equivalent to
selecting an optimal portfoli~from a set of possible portfolios. Typically it is
seen that an investor wants the returns to be high as well as, as certain as
possible. This means that the investor, looking forward to maximize expected
return and minimize risk has two conflicting objectives that must be balanced
against each other when making the purchase decision at t = 0 . The
Markowitz approach for how the investor should go about making this decision
gives full consideration to both these objectives. He used certain concepts
from probability theory to solve this problem. According to Markowitz, the
investor should view the rate of return associated with any of these portfolios to
be what is known in statistics as a random variable. Expected rate of return
can be viewed as a measure of the potential reward associated with any
portfolio and standard deviations can be viewed as a measure of risk associated
with any portfolio. Thus, once each portfolio has been examined in terms of its
potential rewards and risks, the investor is m a position to identify the portfolio
that appears most desirable to him or her.

Thus, as a second step in the model building process, we can say that the
Markowitz model is based on the following assumptions:
Investor invest money for a particular length of time, called the holding

Investors are rational and behave in a manner as to maximize their utllity

with a given level of income or money.
Investors are risk averse and try to minimize the risk and maximize

Investors prefer higher returns to lower returns for a given level of risk.
The evaluation of portfolios is canied out in terms of returns and the risk
associated with the constituting securities, over a given holding period.
Introduction to The obvious question which must be occurring in your mind is - How to
Mathematical Modelling quantify the expected return and risk of a security or a portfolio? Let us try to
do that as a next step in the modelling process.

Expected Return for a Security

Some investments, like bank certificates of deposit, have guaranteed rates of
return. Investments llke stocks are a bit more complicated. If you ask an
investor how much she expects to earn on the stock of a pharmaceutical
company or high-tech fm, she will answer in terms of a range like "I expect to
earn somewhere between 10 and 20 percent on these shares". Whenever an
investor describes fkture returns in terms of a range like this, you can be sure
that there is risk involved. In the investment context, risk is the uncertainty
that a given investment will earn its anticipated rate of return.

To calculate the expected rate of return, you have to first enumerate all the
possible rates of return that an investment could have. For simplicity's sake,
let's imagine an investment with four possible rates of return,
-lo%, - 5%, 10% and 20%.

The first two rates of return indicate a loss while the last two indicate a gain.
The next step is to assign probabilities to each rate of return. How do you
assign these probabilities? It requires to make some educated guesses based on
past performance of the investment itself, and the demonstrated performance of
similar investments. General market and economic factors should also be
aken into account, while assigning these probabilities. Let the probabilities
0.1, 0.1, 0.5 and 0.3 are assigned respectively to the above four rates of

en the expected rate of return or simply expected return E(R) of the

given investment (security) is calculated as follows:

E(R) =.x
n

i=l
(Possible Return x Probability) (2)
= [(-0.10) (0.10) + (-0.05) (0.10) + (0.10) (0.50) + (0.20) (0.30)]
= 0.095.

The expected return is 0.095. This means that the investor can expect a return
of 9.5% on her investment.

Thus, for any security, the expected return is the weighted average all
possible outcomes, where each outcome is weighted by its respective
probability of occurrence. It is calculated as
n
E(R) = C R j Pij
where
E(R) = the expected return on a security,
R = the jh possible return,

Pij - the probability of the i" return R j , and

n = the number of possible returns or outcomes.
 Model Formulation
You may now use the above result to do th.e following exercise.

1
E6) Let the return distribution on a security A be given as follows:
Possible rate of retum -8 -6 9 10 12 8
(in percent) (R j)
Associated probabilities 0.04 0.06 0.2 0.3 0.25 0.15
(pi j>
i Find the expected return of the security A .

! Risk for a Security

in Markowitz sense, the variability or risk of a security is measured by its A security with more
standard deviation. To calculate the risk associated with the expected return, standard deviation is
the variance or standard deviation is used. As you already know standard more risky than a
deviation is a measure of the spread or dispersion in the probability security with less
distribution, that is, it measures the dispersion of a random variable around its standard deviation.
mean. The larger this dispersion, the larger the variance or standard deviation
and hence larger is the risk for a security. Knowing the returns associated with
the securities the variance of the returns is calculated as

where
n2 = the variance of returns,
E(R) = the expected return on a security,
R j = the j" possible return,

Pij = the probability of the i ' retum R j, and

n = the number of possible returns.

o = (a2)'I2,the positive square root of the variance is the standard deviation

of the returns.

I
Let us consider the following example.

Example 1: Assume that the return distribution of a security is as given

follows:

Possible Return 0.01 0.07 0.08 0.1 0.15

Associated Probability 0.2 0.2 0.3 0.1 0.2

Find the standard deviation of the security.

Solution: The calculations for the standard deviation are as shown in Table 1.
Introduction to Table 1
Mathematical Modelling
Possible Associated R x pi,
[R,- E(R)]~ [R - E(R)]'
Return Probability
- pi1
0.01 0.2 0.002 0.0049 0.00098
0.07 0.2 0.014 0.0001 0.00002
0.08 0.3 0.024 0.0 0.0
0.10 0.1 0.010 0.0004 0.00004
0.15 0.2 0.030 0.0049 0.00098
Cpij=I C R j Pij *
C [ R j -E(R)I'P~~
j j
= 0.08 = 0.00202
= E(R) = o2

You may now try the following exercises.

E7) Calculate the expected retum and risk of a security given the following
information

-Probabilities 0.15 0.20 0.40 0.10 0.15

For the s&e of Possible returns 0.20 0.16 0.12 0.05 - 0.05
convenience, we assume
that all securities in a E8) C o m p p the risk of thZ two securities 1 and 2 whose return distributions
portfolio have equal
number of return
are as given in Table 2 below:
outcomes. Also, it is Table 2
assumed that in a
particular situation, Possible rate of return for security Associated probabilities
p '.J. = p , .J, V i and k . 1 2 P11 = P2j
0.19 0.09 0.13
0.17 0.16 0.15
0.11 0.18 0.42
0.10 0.1 1 0.30
- - -

When analysing the returns and risks on an investment made by an investor we

must be concerned with the total portfolio held by an investor. Individual
security returns and risks are important, but it is the return and risk to the
investor's total portfolio that ultimately matters, because investment
opportunities can be enhanced by packaging them together to form portfolios.

Portfolio Expected Return

The expected return on any portfolio is easily calculated as a weighted average
of the individual securities' expected returns. For a portfolio
,
P = (w ,w 2 , ...,w ,) of n securities, where w 's are the portfolio values (or
weights), the expected retum can be calculated as
n
E(R,)=C wj E(Rj) (5)
j=l
 Model Formulation
where
E(R,) = the expected return on the portfolio P ,
w = the portfolio weight for the jmsecurity,

E(R ,) = the expected return on the j" security, and

n = the number of different securities in a portfolio.

Consider for example a portfolio consisting of three stocks X, Y and Z with

expected returns of 12%, 20% and 17% respectively. Assume that 50% of
the funds is invested in security X, 30% in Y and 20% in 2. Then the
expected return on this portfolio is
E(Rp)=0.5x12%+0.3x20%+0.2x17% =15.4%.

It may be noted that regardless of the number of assets (securities) held in a

portfolio, or the proportion of total investable funds placed in each asset, the
expected return on the portfolio is always a weighted average of the expected
returns of individual asset in the portfolio.

Portfolio Risk
Portfolio risk is measured by the variance (or standard deviation) of the
portfolio's return, exactly as in the case of each individual security. Although
the expected return of a portfolio is a weighted average of its expected returns,
portfolio risk is not a weighted average of the risk of the individual
securities in a portfolio. Mathematically,

But, 0: +xw j 0;.

Portfolio risk is a unique characteristic and not simply the sum of individual
security risk. This is because a security may have a large risk if it is held by
itself but risk may be small when held in a portfolio of securities. But then the
question is how a portfolio of assets can reduce risk and how the risk is
measured? Let us analyze the portfolio risk.

Analyzing Risk of a Portfolio

Let us first assume that rates of return on individual securities are statistically
independent such that rate of return on any one security is unaffected by the
rate of return on another security. Jn this situation, the standard deviation of
the portfolio is given by

Thus, a, decreases as n increases i.e., the risk of the portfolio will be reduced
as more securities are added to the portfolio. You do not have to take decision
about which security to add, as all of them have identical properties. The only
concern is how many securities are to be added? However, in the real world,
the assumption of statistically independent returns on stocks is unrealistic.
Introduction to Most stocks are positively correlated with each other, that is, the movements in
Mathematical Modelung their returns are related. For example, a rise in interest rates will adversely
affect most of the firms, because most of the fixms borrow funds to finance part
of their operations. Therefore, there is a need for diversification i.e., putting
small fractions of the total funds in as many securities as are found suitable
after their evaluation is done.

Diversification

Diversification is the key to the management of portfolio risk. It allows the

investors to reduce portfolio risk without aecting returns. Thus,there is a
need to quantifjl diversification. Markowitz was the first one to quantify the
concept of diversification. He showed quantitatively why and how portfolio
diversification helps to reduce the risk of an investor's portfolio. Markowitz
showed that to measure and reduce portfolio risk to its minimum level for any
given level of return, we must measure the interrelationships among security
returns. The reason being that in any one time period, poor performance by
some securities may be offset by strong performance in other securities. Let us
now consider how to measure these interrelationships, or co-movements or
covariance, among security returns.

Covariance in Security Returns

Covariance is an absolute measure of the co-movements between security

returns used for calculating portfolio risk. Therefore, in order to calculate
portfolio variance or standard deviation we need to calculate covariance
between securities in a portfolio. The foxmula for calculating covariance
between two securities A and B is given as follows:

j=l
where
,rc = the covariance between securities A and B ,
RA = jUIpossible return on security A ,
,
E(R ) = the expected value of the return on security A , and
n = the number of likely outcomes for a security for the period.

Eqn. (8) gives the covariance as the expected value of the product of deviations
from the mean. Covariance can be positive, negative or zero.

Positive covariance indicates that the returns on the two securities tend to
move in the same direction at the same time. When one increases
(decreases), the other also do the same.
Negative covariance indicates that the returns on the two securities move
inversely. When one increases (decreases), the other tends to decrease
(increase).
Zero covariance indicates that the returns on two securities are
independent and do not move together in the same or opposite directions.
The covariance o, when divided by the standard deviations of securities A
and B gives
 Model Formulation

p, arc the
Possible rates of returns for security Associate probabilities
associated
X Y Pxj " P y j probabilities of the
.19 .18 0.33 security X and
.17 .16 . 0.25 p Y j t sarc the
.I 1 .I 1 0.22 associated
.10 .9 0.20 probabilities of the
L
secwity Y.
 lntrodnction to You can check that
Mathematie~lModelling
ox = (0.33x (0.0406)' + 0.25 x (0.0206)' + 0.22 x (-0.0394)' + 0.2(-0.0494)' )112

oy = (0.33 x (0.0384)' + 0.25 x (0.01 84)' + 0.22 x (-0.03 16)' + 0.2(-0.0516)' !I2
= (0.0013)"~= 0.036
then ox, can be calculated using Formula (1) as
om = 0.33 x 0.0406 x 0.0384 + 0.25 x 0.0206 x 0.0184
+ 0.22 x (-0.0394) x (-0.0316) + 0 . 2 0 ~(-0.0536) x (-0.0516)
= 0.0013.
Hence, by Formula (2), we have
~ X Y- 0.00 13
Pxy = oxay - 0.038 x 0.036 =0.95 cl.

I You may now try the following exercise.

E9) For the data given in E8), find the covariance 6,' and correlation
coefficient pI2 between two securities 1 and 2 .

Knowing the correlation and covariance that gives the comovement in security
returns, we are now ready to calculate portfolio risk. We start with the case of
two securities and then generalise it to n securities.

If P = ( w , , w,) beaportfoliooftwosecurities 1 and 2 with w, + w 2 =1,

then the risk of the portfolio, as measured by the standard deviations of
returns is given by

where
op = the standard deviation of the portfolio P ,,
o, = the standard deviation of security 1,
o2 = the standard deviation of security 2 ,
w, = the portfolio weight of security 1, and
,
w = the portfolio weight of security 2 .
p,, = the correlation coefficient between security 1 and 2 .

Portfolio risk is effected both by the correlation between assets and by the
percentage of funds invested in each asset. We shall now illustrate it through
an example.

Example 3: Consider a portfolio P of two securities X and Y whose return

distributions are as given in Example 2, Let 50% of investable funds is to be
placed in each security. Find the risk for the portfolio P .
Solution: We have the following data from Example 2. Model Formulation

Also we have w, = 0.5, w, = 0.5 .

Then the risk a, of the portfolio P is obtained as
o, = [(0.5)'(0.038)' + (0.5)~(0.036)~+ 2(0.5)(0.5) (0.038) (0.036) (0.95)]"'
+
= [0.0007 0.0006 + 0.0006 x 0.95]"~
= 0.0436
You may note that the risk of the portfolio depends heavily on the value of the
correlation coefficient between the returns of the two securities as is evident
from the following values
p = +l.O; op= .0447
p = +0.5; op= 0.04
p = +O. 15; o, = 0.0373
p = 0.0; o, = 0.036
p=-0.5; 0, =0.0316
p = -1.0; O, = 0.0265

The risk of the portfolio steadily decreases from 0.0447 to 0.0265 as the
correlation coefficient declines from + 1.0 to - 1.O. In the same way it can be
seen by holding the correlation coefficient constant that the size of the portfolio
weights assigned to each security has an effect on portfolio risk. You can
check it yourself in the above example by assigning different values to w, and
, , ,
w instead of assuming w = w = 0.5 .

The two-security case for calculating portfolio risk can be generalised to the
n-security case. Portfolio risk in the case of n securities can be calculated as
follows:
n n n

where
o$ = the variance of the return on the portfolio,
of = the variance of return for securityi,
o, = the covariance between the returns for securities i and j , and
wi = the portfolio weights or percentage of investable funds invested
in security i.
You may now try the following exercises.

E10) Four securities have the following expected returns:

A=15%,B=12%,C=30% and D = 2 2 % '
Calculate the expected returns for a portfolio consisting of all four
securities under the following conditions.
a) The portfolio weights are 25% each.
Introduction to b) The portfolio weights are 10% in A and the remaining is equally
Mathematical Modelling divided among the other three securities.
c) The portfolio weights are 10°h each in A and B and 40% each in
C and D.
El 1) Let the returns on three securities 1,2,3 be 25%, 28% and 14%
respectively with o, = 4, a, = 5, o3= 6, o,, = oI3= 15 and o,, = -10.
Find the standard deviation a, of the portfolio P = (0.5, 0.3, 0.2).

You have learnt to evaluate portfolios on the basis of their expected returns and
risk as measured by the standard deviation. The evaluation of the risk of a
portfolio involves the evaluation of the following three parameters:
i) Standard deviation of each security.
ii) Covariance between pair of securities
iii) Proportion of funds invested in each security.
Once the risk-return opportunities available to an investor is determined it is
seen that a large number of possible portfolios exist when the percentage of the
investors wealth to be invested in each security is varied.
Doest the investor need to evaluate all these portfolios? The answer to this
question is no. The reason being that an investor needs to look at only a subset
of the available portfolios meeting the following two conditions.
1. Portfolios that offer maximum expected returns for varying level of risk.
2. Portfolios that offer maximum risk for varying levels of expected returns.
The set of portfolios meeting these two conditions is known as the efficient set
or efficient frontier. As a next step in the modelling process, we now try to
get new information about the problem being studied. From a large number of
possible portfolios, we try to locate the efficient frontier. The efficient set can
be located from the feasible set, also know as the opportunity set. We shall
now see how this is done.

EMcient Frontier
Let us consider Fig. 2 illustrating the location of the feasible set.

Fig. 2: The feasible set of portfolios.

 Model Formulation
You know that the feasible set represents the set of all portfolios that could be
formed from a group of n-securities. That is, all possible portfolios that could
be formed from the n-securities lie either on or within the boundary of the
feasible set. The points denoted by A, B, C and D in Fig. 2 are examples of
such portfolios. In general this set will have an umbrella type shape similar to
the one shown in Fig. 2. The efficient set can then be located by applying the
two conditions of the efficient set. Let us h t , identify the set of portfolios
meeting the first condition.

If you look at Fig. 2, there is no portfolio offering less risk than that of
portfolio A . This is because if a vertical line is drawn through A , there will
no point in the feasible set to the left of the line. Also there is no portfolio
offering more risk than that of portfolio C . Thus,the set of portfolios offering
maximum expected return for varying levels of risk is the set of portfolios lying
on the 'northern' boundary of the feasible set between points A and C .

Let us now consider the second condition. The portfolio offering maximum

point in the feasible set that lies above this line. similarly, there is no portfolio
offering a lower expected return then portfolio B . Thus, the set of portfolios
offering minimum risk for varying levels of expected return is the set of
portfolios lying on the 'western' boundary of the feasible set between points B

1 and D. Now since both the conditions are to be fblfilled while identifying the
eficient set, only the portfolios lying on the northwest boundary between
points A and D need to be considered. Accordingly, these portfolios form the
efficient set, and the investor will have to find his or her optimal portfolio fkom
this set of efficient portfolios. All other portfolios are inefficient and can be
safely ignored.

Selecting an Optimal Portfolio

To select an optimal portfolio of assets using the Markowitz analysis, investor

should
1. Identify optimal risk-return combinations available from the set of risky
i assets being considered by using the Markowitz efficient frontier
analysis.
2. Choose the final portfolio from among those in the efficient set based on
an investor's preferences.

To select the expected return-risk combination that satisfy an individual

investor's personal preferences, indifference curves are used. Investors are
assume to know their indifferencecurve. These curves raresentinn an
I
investor's preferences for risk and return can be drawn on a two-dimensional
figure, where the horizontal axis indicates risk as measured by standard
deviation (a,) and vertical axis indicates reward as measured by expected
- return(E(R,)).
Introduction to Indifference curves for a hypothetical investor are shown in Fig. 3.
Mathematical Modelling

OP
Fig. 3: Indifference curves.

Each cruved line in Fig. 3 indicates one indifference curve for the investor and
represents all combinations of portfolios that the investor would find equally
desirable. For example, the investor with the indifference curves as shown i.1
Fig. 3, would find portfolios A and B equally desirable, since they lie on the
same indifference curve I,, even though they have different expected returns
and standard deviations. Portfolio B has higher standard deviation then
portfolio A so it is less desirable on that dimension but on the other hand, it is
preferred as it provides higher expected return. Thus, all portfolios that lie on a
given indifference curves are equally desirable to the investor. Also,
indifference curves cannot intersect, since they represent different levels of
desirability.

As an investor, we would always love to have portfolios with more return and
less risk, our indifference curves will always show some inclination towards
the return line. That is, we choose portfolios in such a way that the
corresponding curve heads towards the northwest direction. In other words,
farther an indifference curve is fiom the horizontal axis, the greater is the
utility. Let us now see how do we use indifference curves in conjunction with
the efficient frontier to find which feasible portfolio is optimal.

Fig. 4: Selecting an optimal portfolio on the efficient frontier.

To st& with, an investor should plot his or her indiffefence curves on the same
figure as the efficient set and then choose the portfolio that is on the
indifference curve and is furthest northwest. This portfolio will correspond to
the point where an indiffernce curve is just tangent to the efficient set. In
Fig. 4, this is portfolio P* on indifference curve 1,.
In Fig, 4, fie curve l3represents the best level of satisfaction but it is of no use
,
it does not meet the feasible region at all. In regard to are sevcral
portfolios that an investor could choose, say for example P but portfolio **
dominates f i s e pofifolios, since it is on an indifference that is further
,*hwest. A~SO,you may note that this point of intersection will be unique
because of the convexity of the efficient frontier. ~ e n c eP* is our optimal
pofifolio. With the selection of P' our problem of selecting an optimal
portfolio is solved completely. However, the model h a certain limitations
which we shall list now.

Limitations of Markowitz Model

Markowitz model presents a systematic procedure for the selection of the

optimal portfolio of risky securities but is cumbersome to work with. It
requires a full set of covariance between the returns of all securities to be
considered in order to calculate portfolio variance. This leads to gathering an
enormous amount of input data while dealing with the portfolios having large
number of securities. Evaluation of a single portfolio of n-securities requires
2n quantities for estimating the expected return and standard deviation of these
n(n - 1)
n-securities. There are covariances to be estimated for these
2
n-securities. Thus to evaluate a portfolio of n (risky) securities we need to
estimate in all a total of 2n + n(n-1) - - n(n+3) quantities. A lot of energy
2 2
and time is needed to carry out the computations for the selection of an optimal
portfolio. However, with the availability of computer packages, things can be
managed for large values of n .

You may now try the following exercises.

E12) How many portfolios are on an efficient frontier? What is the Markowitz
efficient set?

El31 How is an investor's risk aversion indicated in an *difference curve?

Are all indifference curves upward sloping?

E14) Why indiffaence curves of an investor cannot intersect?

The limitations of the Markowitz model raise two issues:

i, Are sLnpler methods for computing cBdent frontier?
ji) assets? M d ~be~~ i ~~toeoptimize
Cm Ihe MmkOwi"
individual d
ratherthan
reduction to As mentioned in Unit 1, the model-building process proceeds through several
,@ematicalModelling iterations, each a refmment of the preceding model. %Sharpe model
developed by Willism S h q e has simplified the ligours involved in computing
the efficient hntim using the Markowitz approach. We shall now discuss this
model in the next sub-section.

2.3.2 Sharpe Model

model COmonly known as the single-index model, provides an
expression for portfolio variance, which is easier to calculate than in
the c a e of the Mark0wit.Z analysis. This alternative approach can be to
solve the portfolio problem formulated by Markowit2 for determining the
set of podfoli0S. The single-index model, relates return on each
~ turns on a Common index. Generally a bmad market index of
s e c ~tot the
COmon stock rdums is used for this purpose.

The single-index model can be expressed by the following equation

Rj=a, +PjRM+ e j
where
R, = the return on j" security,
R M = the return on the market index,
a = part of j" security return independent of market performance,
p = a constant meas&ng the expected change in the dependent
variable, Ri ,given a change in the independent variable, R, ,and
e = the random residual error.

The single index model divides a security's return into two components a , and
a market rdated part PjRM. Given these values, the error term is the
difference between the return on jh security and the SUm of two components
of return, i.e.,
e j = R , - ( a j +QjR,)
For example, let us assume the return for the market index for period t is
lo%, a = 4y0 and p = 1.S . Then the estimate for stock j is

R~ =4%+1.5RM+e,
or, Rj=4%+(1.5)(10%)=19%

This shows that if the market index return is lo%, then tfie likely for
your stock is 19% , Further, if the actual retum On Jh stock for period is
16%,theerrortenmis 16%-19%=-3%.

~~t~ that R~ and ej are random variables and the P term, or bet% is
imponant as it m w u e s the msitivity of a stock to mmket movnncnts. To
use our model, we to h o w for each StoEk we consider3 the estimates of
m o u s l y available dab may be used to estimate future beta Or
huld also $ve some subjective estimate of beta.
 Model Formulation
The single-index model uses two simplifjmg assumptions. The first
assumption used in the single-index model is that the random error term and
the market index are uncorrected, meaning that the outcome of the market
index has no bearing on the outcome of the random error term. The second
assumptions is that the random error terms of any two securities are
uncorrelated, meaning that the outcome of the random error term8f i"
security has no bearing on the outcome of the random error term of j" (i # j)
security. This can be expressed as cov(ei,ej) = 0. In other words, the
returns of the two securities will be correlated (i.e., move together) only in their
common response to the return on the market. This mean that stocks covary
together only because of their common relationship to the market index. If
either of these two assumptions are invalid i.e., they are not good description of
reality, then model will be an approximation and alternately, more than one
index model may be useful in such cases.

In the single-index model, all the covariance terms can be accounted for stocks
being related only through common reactions to the market index, that is,
covariance depends only on the market risk. Therefore, the covariance
between any two securities can be written as

where ah is the variance of the market index. In the Markowitz model, we

need to consider all the covariance terms in the variance-covariance market. In
the single-index model, just as the security return is split into two components,
the risk of an individual security is also divided into two components. This
simplifies the covariance and also the calculations for $e total risk for a
security and for a portfolio are simplified to a large extent. The total risk of a
security, as measured by its variances, consists of two components, market risk,
which is common to all companies, and the company specific risk. Thus, we
can write

of =sf[o,] + ocj
2 2
(16)
(Market risk) + (company specific risk)

This simplification also holds for portfolios. It provides an alternative

expression for finding the minimum variance set of portfolios in the form

:;[ )
var iancc =
p:;olio]
Portfolio
+ [residual
variance]
Portfolio

The single-index model greatly simplifies the calculation of the portfolio

variance and therefore, the calculation of efficient portfolios. In the case of the
Markowitz analysis, 200 stock require 19,900 covariances and 200 variances.
Whereas, we would require 3n + 2 or 602 estimates for 200 securities while
using single index model.

I
As in the case of Markowitz analysis, in order to detennine the composition of
the tangency portfolio, the investor needs to estimate all the expected returns,
1 variances and covariances. In the case of single-index model, this can be done
/ by estimating a j, Pi and o, for each of the n risky securities. Also needed
Introduction to are the values of R M and its variance (T: . One way to estimate the
Mathematical Modelling
parameters is with time series regression. With these estimates Eqns. (13), (16)
and (17) can be used to calculate the returns, variances and covariances for the
securities. Using these values, the curved efficient set of portfolios can be
derived, h m which the tangency portfolio can be determined.

Limitations of the Sharpe Model

The single-index model is a valuable simplification of the full variance-

covariance matrix needed for the Markowitz model. It reduces by a large
amount the numb& of estimates needed for a portfolio of securities. However,
the model makes an assumption - the residuals for afferent securities are
uncorrelated. The accuracy of the estimate of the portfolio variance thus
depends on the accuracy of this assumption. For example, if the covariance
between the residuals for different securities is positive, not zero as assumed,
the true residual variance of the portfolio will be underestimated and the model
will be in accurate. In such cases models with more than one index are
available and can be used. We shall not be discussing such models here.

Before we end this unit you may try the following exercises.

El 5) How is the covariance between any two securities calculated with the
single-index model?

El 6) How would you compare the Markowitz model with the Sharpe model?

We now end this unit by giving a summary of what we have covered in it.

2.4 SUMMARY
In this unit, we have covered the following points:
1. Once the essential characteristics of the real world problem are identified
its conversion into a mathematical description in terms of equations can
be done in different ways according to the objectives of the study as
illustrated in the case of motion of a simple pendulum.
2. Portfolio selection problem: How can an investor choose an optimal
portfolio, fiom a feasible set of risky securities i.e., choose a portfolio that
provides h i d e r the maximum level of satisfaction in t m s of return and
risk.
3. Markowitz portfolio theory provides the way to select an optimal
portfolio based on using the full information about securities.
4. An efficient portfolio has the highest expected return for a given level of
risk or the lowest level of risk for a given level of expected return.
5. The Markowitz analysis determines the efficient set of portfolios, all of
which are equally desirable. The efficient set in an expected
return-standard'deviation space is a curve which is upward concave.
6. The efficient frontier gives the investment possibilities that exist from a
given set of securities. Indifference curves express investor preferences.
 Model FormuI.tion
I. The optimal portfolio for a risk-averse investor occurs at the point of
tangency between the investor's highest (northwest) indifference curve
and the efficient set of portfolios.
8. The Sharpe model relates retuns on each security to the returns on a
common index.
9. The Sharpe model provides an alternative expression for portfolio
variance, which is easier to calculate in comparison to the case of the
Markowitz analysis.

El) i) Dynamic, discrete, probabilistic

Essentials: risk, premium, number of customers, profit, etc.
P
Non-essentials: religion and gender of the people working in the
i company.
ii) Discrete, dynamic, probabilistic
Essentials: mice. advertising, competition, population changes,
seasonal variations, etc.
Non-essentials: name, religion and gender of the people working in
the company.

E2) State the problem giving reasons for why there is no other treatment of
the problem possible.

I E3) i) The meteorological quantities like rainfall, atmospheric

temperature, pollution are all continuous and stochastic. Similarly
give examples for ii) and iii).

1 E4) i) 20% ii) -7.5% iii) 1.67%

ES) P, = (0, -1
3 3
-1,
2
P = (0,
1
-,3
4 4
-1, P = ,, ,
4 2 4
) Similarly ~ n d

I others.

E8) a, = 0.0337, a, = 0.0362. Security 2 is more risky than security 1.

t E 12) There are large number of portfolios on an efficient hntier. Markowitz

I
eficient set is the set of portfolios having highest expected return for a
Introduction to given level of risk or the lowest level of risk for a given level of expected
Mathematical Modelling -. .

E13) Yes, all indifference c w e s are upward sloping. L

E14) Let three be a point X of intersection of two indifference c w e s I, and

I,. Show that all portfolios on I, must be as desirable as those on I2
and then reach a contradiction since I, and I, are two curves that are
supposed to represent different levels of desirability.

El 5) Explain Eqn. (15).

E16) Comparison between the two models can be made in terms of risk and
return evaluation, in terpls of number of estimates required, etc.
-x-
 UNIT 3 DATA ANA!

structure Page No
3.1 Introduction
Objectives
3.2 Data Visualization
3.3 Simple Linear Regression Models
F 3.4 Multiple Linear Regression Models
3.5 Summary
9
3 -6 Solutions/Answers

Most scientific disciplines are concerned with measuring items and collecting
data. One reason for this is the increasingly quantitative approach employed in
all the sciences, business and many other activities which directly affect our
lives. Volumes of data on various items like population, taxes, wealth, exports,

I research organisations or through direct field work. In order to draw any

rneaningfbl conclusion fiom the data, it is important to represent and analyse
the collected data in an effective way. With the help of statistics, data can be
used to

It analyse and summarise a situation;

model experimental outcomes;
I • quantify uncertainty;
1 make decisions;
I. predict future outcomes.
II Statisticians generally obtain data by taking a sample fiom some larger
1 population. We take samples because the entire population may be too large
to be examined in a timely and economic manner. Regression analysis is one
of the techniques used in statistics for modelling and analysing numerical data.
Regression models describe the variation in one dependent variable (also called
response variable or measurement) when one or more independent variables
(also known as explanatory variables or predictors) vary. Applications of

I random variation in the response variable. This relationship may be linear or

nonlinear combination of one or more model parameters and accordingly the
Introduction to model is classified as linear or nonlinear. In this unit, we shall discuss some of
Mathematical Modelling the linear regression models. For example, models relating to time and
distance, price and saIe, performance of a car related to engine displacement
and the hmepower of the engine, etc.

We shall start the unit by giving you in Sec. 3.2, the general idea of data
visualization i.e., visual representation of data. We shall discuss linear
regression modeIs with one predictor in Sec. 3.3 and with two or more
predictors in Sec. 3.4. Second order polynomial models in one variable are
also introduced in this section. You are also required to do computer practical
exercises on this unit which are given at the end of the unit.

Objectives
After studying this unit, you should be able to
use quantitative and graphical techniques for data visualization;
distinguish between simple and multiple linear regression models;
write down simple or multiple linear regression models appropriate for a
given set of data;
use the method of least squares for estimating the parameters of linear
regression model;
fit a line/curve/plane/surface,appropriate, for a given set of data.

3.2 DATA VISUALIZATION

Data visualization is the study of the visual representation of data. It is an
information which has been abstracted in some schematic form including
attributes or variables for the units of information. Data visualization is closely
related to information graphics, information visualization, scientific
visualization and statistical graphics. It refers to the more technologically
advanced techniques which allow visual interpretation of data through the
representation, modelling and display of solids, surfaces, properties and
animations, involving the use of graphics, image processing, computer vision
and user interfaces. It encompasses a much broader range of techniques than
specific techniques as solid modelling.

The origin of this field are in the early days of computer graphics in the 1950s,
when the first grwhs and figures were generated by computers. With the rapid
increase of computing power, larger and more complex numerical models were
developed, resulting in the generation of huge numerical data sets. Also, large
data sets were generated by data acquisition devices such as medical scanners
and microscopes, and data was collected in large databases containing text,
numerical information and multimedia information. Advanced computer
graphics techniques were needed to process and visualize these massive data
sets. Once the data are converted to a visual form, the trend and patterns are
often immediately apparent. Fig. 1 shows an example of large data set that has
been converted to colour-coded display. It shows the Indian map with the
population classified and presented using different colours.
 Data Analysis and
Fitting Models to Data

POPULATION MAP

4lABIM 51%
nA> 01 BEYGAL

NM C W *
Iwrtate
narc ltnm 9 cnn:
%YDAUA\ & IICOBAU I U A L U S

Fig.1: Population map of India-2001.

Visualization in scientific computing was used initially to refer to visualization

as a part of a process of scientific computing: the use of computer modelling
and simulation in scientific and engineering practice. Some of the most
popular examples of scientific visualizations are computer-generated images
that show real spacecraft in action, out in the void far beyond earth, or on other
planets. TV also offers scientific visualization when it shows computer drawn
and animated reconstructions of road or airplane accidents. More recently,
visualization is also increasingly concerned with data from other sources
including large and heterogeneous data collections found in business and
finance, administration, digital media, etc.

A new research area called Information Visualization was launched in the early
1990s to support analysis of abstract and heterogeneous data sets in many
application areas. Therefore, the phrase "Data Visualization" is gaining
acceptance to include both the scientific and information visualization fields.

Two main parts of data visualization mainly presumed are statistical graphics,
and thematic maps.

Statistical Graphics
Statistical graphics, also known as graphical techniques, are information
graphics in the field of statistics used to visualize quantitative data. Statistics
and data analysis procedures can broadly be split into two parts: quantitative
techniques and graphical techniques. Quantitative techniques are the set of
statistical procedures that yield numeric or tabular output. Examples of
quantitative techniques include hypothesis testing, analysis of variance, point
estimation, confidence intervals, and least squares regressicn. These and
similar techniques are all valuable and are mainstream in terms of classical
analysis.

On the other hand, there is a large collection of statistical tools that we

generally refer to as graphical techniques. YOUmust be familiar with most of
 Introduction to these techniques through your undergraduate statistics course. These include:
Mathematical Modelling scatter plots, bar diagrams, histograms, probability plots, residual plots, box
plots, block plots, and biplots. Exploratory data analysis (EDA) relies heavily
on these and similar graphical techniques. Fig. 2 gives the bar diagram for the
data showing the marks obtained (out of 10) by 100 students in an examination.
A
40- /T

36 35
I 35-
8
I:
30-
m
C1

8
a 25-
E
k 20-
0

15- 13 -
13

10-

5- 3

I I,
1 2 3 4 5 6 7 8 9 1 0
Marks of sGdents

Fig. 2: Bar diagram of marks obtained by students.

Graphical procedures are not just tools used in an EDA context; such graphical
tools are the shortest path to gaining insight into a data set in terms of testing
assumptions, model selection and statistical model validation, estimator
selection, relationship identification, factor effect determination, and outlier
detection. In addition, good statistical graphics can provide a convincing
means of communicating the underlying message, that is present in the data, to
others. To sum up we can say that the graphical statistical methods have the
following four objectives:

The exploration of the content of a data set;

The use to find structure in data;
Checking assumptions in statistical models;
Communicate the results of an analysis.

Thematic Maps
A thematic map displays spatial pattern of a theme or series of attributes. In
contrast to reference maps which show many geographic features (forests,
roads, political boundaries), thematic maps ernphasise spatial variation of one
or a small number of geographic distributions. These distributions may be
physical phenomena such as climate or human characteristics such as
population density and health issues. These types of maps are sometimes
referred to as graphic essays that portray spatial variations and
interrelationships of geographical distributions. Location, of course, is also
important to provide a reference base of where selected phenomena are
occuning. While general reference maps show where something is in space,
thematic maps tell a story about thqt place. Fig. 3 gives a Pie chart showing the
50
 Data Analysis and
proportion of people from different states of India living in a certain town of Fitting Models to Data
West Bengal.

Fig. 3: Pie chart showing the proportion of people state-wise.

You may now try the following exercises.

El) Explain at least four statistical graphical techniques you are familiar with.
lllustrate the advantages of each of them through examples.

E2) Give at least two examples of the thematic maps with which you are
familiar. Also explain the purpose of using each of them.

Once the dzta related to any study is collected, the next step is to make use of
this data to draw meaningfbl conclusions, i.e., analyse the data about the
subject under study. As we have already mentioned, regression models which
are statistical models are usekl in almost all the areas biological, physical,
social sciences, business, engineering, etc. in both the planning stages of
research and analysis of the resulting data. Care should be devoted to accurate
data collection because the conclusions fiom the analysis depend on the data.
A good data collection will result in better analysis and more applicable model.
If the data used in a regression model are not representative of the system
studied, then conclusions drawn fiom the model are likely to be in error.

Let us now discuss some linear regression models one by one

3.3 SIMPLE LINEAR REGRESSION MODELS

In simple linear regression, relationship between two variables, for example,
income and number of years of education; height and weight of people; length
and width of envelopes; temperature and output of an industrial process;
altitude and boiling point of water; or dose of a drug and response are
modelled. The linear relationship between any two such variables can be
represented by a straight line. Let us now illustrate through an example the
formulation of simple linear regression model.

Example 1 (Driving a car at a constant speed): Suppose you are driving a

car at 5.0km/h . The two variables of interest to you are time and distance. If
Introduction to you are driving at constant speed, then the theoretical relationship between
time and distance covered is given by the straight line as shown in Fig. 4(a).
Theoretical distance Observed distance

Time (min) Time (min)

(a> @)

(a)
Fig. 4: Scatter plot of'the time and distance data.

In a perfect world, where speed and distance could be measured without error,
all observations would lie exactly on this straight line. However, in reality it is
impossible to keep the speed exactly constant and to measure the precise
distance. Therefore, in a scatter plot of 'real7 data, the points would deviate
fkom the theoretical straight line. A realistic scatter plot might look like Fig.
4(b). For this example, we need a model that will describe the linear
relationship between the two variables, and, at the same time, take the variation
away from the line into account.

Formulation
If we let y represents the distance covered and x represents time then the
equation of the straight line in Fig. 4(a), relating these two variables is

where bo is the intercept and b, is the slope. Now the data points in Fig. 4(b)
do not fall exactly on a straight line so Eqn. (1) should be modified to account
for this. Let the difference between the observed value of y and the straight
line (bo + blx) be an error e . That is, it is a device that accounts for the failure
of the model to fit the data exactly.

Thus, a more appropriate model for the data in Fig. 400) is

Eqn. (2) is called a linear regression model. Variable x is called the

predictor or regressor variable and y the response variable. bo and b1 are
the parameters of the model and e is a random error component. The errors
are assumed to have mean zero and &own variance a*.We also assume
that the errors are uncorrelated. This means that the value of one error does not
depend on the value of any other error. Because Eqn. (2) involves only one
 regressor variable it is called a simple linear regression model. In gerenal, the Data Analysis and
,
response may be related to k regressors, x, , x, ,...,x , so that
Fitting ~ o d e l to
s Data

Eqn. (3) represents a multiple linear regression model, which we shall discuss
in the next section. A model is called linear because it is linear in the
parameters bo,b, , ...,b ,and not because y is a linear function of x's. You
will see that many models in which y is related to the x's in a nonlinear
fashion are treated as linear regression models as long as the equation is linear
in the b's . Once model (2) is formulated, we would like to use it to obtain
I information on the y's for specific x values. For that we have to estimate the
unknown parameters in the regression model by making the elrot minimum
and this process is called fitting the model to the data. There are several
parameter estimation techniques available. Here we shall be using the method
of least squares for parameter estimation.

Least squares estimation of the parameters

The parameters bo and b, are unknown and must be estimated using sample
data. Suppose we have n pairs of data, say (y, , x, ), (y, , x ,), ...,(yn, xn ) .
Then the estimation of bo and bl js done as follows:

Estimation of b, and b,

Let us use the method of least s q w e s to estimate bo and b, . That is, we will
estimate bo and b, so that the sum of the squares of the difference between the
observations yi and the straight line is a minimum. From Eqn. (2), we may
'write

Eqn. (2) may be viewed as a population regression model while Eqn. (4) is a
sample regression model, written in terns of the n pairs of data
(yi, xi), i = 1, 2, ..., n . Thus, the least squares criterion is that the error

should be minimum.

The least squares estimators or the 'best' estimates of bo and b,, say 6, and
61, are t1.e values that minimize e and, therefore, must satisfy

Simplifjmg Eqns. (6) &d (7), we get

Introduction to
Mathematical Modelling

Eqns. (8) and (9) are called the least squares normal equations. The solution
to the normal equations is obtained as

are the averages of yi and x i , respectively. Therefore, 6, and 6, in Eqns.

(10) and (11) are the least squares estimators of the intercept and slope. The
fitted simple linear regression model can then be written as
A A

y=bo +blx. (13)

This line is called the least squares line or the regression line. For a given set
of data, the least squares lines is the best-fitting line, in the sense that it
minimize e . The values jl are referred to as predicted values or fitted
values. The difference between the observed value yi and the corresponding
fitted value f i is a residual.

Mathematically, the ith residual is

Residuals play an important role in investigating the adequacy of the fitted

regression model and in detecting departures from the underlying assumptions.

Using simple algebra you can write the denominator and numerator of
Eqn. (11) in a more compact notation as

Thus, Eqn. (11) can be conveniently written as

Before proceeding further, you may try the following exercise.

 Data Analysis and
E3) ,VerifyEqns.(15)and(16). Fitting Models to Data

In addition to estimating b, and b, , we also need an estimate of aZ,the

variance of error. It gives the variance of the data points around the least
squares line.

Estimation of a2

The estimate of oz is obtained from the residual or error sum of squares

Substituting 3, = 6, + 6,xi in Eqn. (18) and simplifying, we get

a
Thus, SS, = S, - bi .,S

The residual sum of squares has n - 2 degrees of eeedom, because two

degrees of fkeedom are associated with the estimates 6, and 6, involved in
obtaining- 9- ;. . Thus, an estimate of oZ is given by
SS MS, .
62 =ex
n-2

The quantity MS, is called the error mean square or the residual mean
square. The square root of 6 ' is called the standard error of regression, and
has the same units as the response variable y . Because e2 depends on the
residual sum of squares, any violation of the assumptions on the model errors
or any misspecifiCation of the model form may damage the usefilness of 62as
.
an estimate of oz We now illustrate the estimation of the model parameters
through examples.

Example 2 (Demand for homes): Find a linear demand equation that best fits
the following data, and use it to predict annual sales of homes priced at
Rs. 14.00.000.

x = Price (lakhsof Rs.) 16 18 20 22 24 26 28

1'S~\CSof new homes this year 126 103 82 75 82 40 20

-
Solution: Calculations are shown in Table 1. 55
Introduction to Table 1
Mathematical Modelling

L I I

28 20 560 784
sums Xx = 154 Xy = 528 Zxy = 10,728 x x 2 = 3500

Substituting these values in the formula given by Eqns. (10) and (1 1), we ge
( ) - ( 1 ( 1 7(10, 728) - (154) (528) -7.929
slope=b, = - -
n(Zx ) - (Zx) 7(3500) - 1 5 4 ~

intercept = b, =
- Zy-m(Zx)
-
528-(-7.929)(154)m249,9.
n 7

Thus,our least squares line is 9 = 249.9 - 7.929~.

We can now use this equation to predict the annual sales of homes priced at
Rs.14,00,000. Remembering that x is the price in lakhs of rupees, we set
x = 14, and solve for y ,getting y = 139. Thus,our model predicts that
approximately 139 homes priced at Rs.14,00,000 will be sold.
***
Example 3: Consider the data shown in Table 2
Table 2

Use a best fit line to estimate the value of y for x = 6 and 8 . Also obtain the
estimate of the error variance of the best fit.
Solution: Calculations of the least squares line are given in Table 3.
Table 3
 - 89
Therefore, b, = -- 2.225
Data Analysls and
Flttlng Models to Data
40
and 6 , = 6 - 2.225(4) = 6 - 8.9 = -2.9.
P

The least squares line is jr = -2.9 + 2 . 2 2 5 ~. Thus, if x = 6 , we would predict

that
9 = -2.9 + 2.225(6) = 10.45.
Similarly, if x = 8 we would predict
9 = -2.9 + 2.225(8) = 14.9.
Calculations for SS, are done in Table 4.

Table 4

Thus, SS, = 1.975 , and it follows from Eqn. (21) that

6' = 1.975/(5- 2) = 0.6583.
***
Once the parameters of the model have been estimated, it is important to
evaluate the model adequacy. For this purpose, let us determine the coefficient
of determination.

Coefficient of Determination

The quantity R 2 defined by

is called the coefficient of determination where SS = , (fi - 7)' is the

i
regression sum of squares and S, = (y, - 7)' is the total sum of
1

squares. The total sum of squares can be partitioned into

S, = SS, + ss,
 Introduction to You may notice that S, determines the variability in y without considering
Mathematical Modelling
the effect of the regressor variable x ,and SS, is a measure of the variability in
y remaining after x has been considered. Thus, R~ in Eqn. (22) gives the
proportion of variation in y explained by the regressor x . Because
OSSSe.SS,,itfollowsthat 0 5 R 2 5 1 . ~ a l u e s o fR 2 thatarecloseto 1
imply that most of the variability in y is explained by the regression model.
For the data in Example 3, we have

3 that is, 99.34 percent of the variability in the data is accounted for by the
regression model.

The statistic R should be used with caution, since it is always possible to

make R~ large by adding enough terms to the model. For example, if there are
no repeat points (more than one y value at the same x value) a polynomial of
degree n - 1 will give a perfect fit (R = 1) to n data points. When there are
repeat points, R can never be exactly equal to 1, because the model cannot
explain the variability due to 'pure' error. Although R increases if we add a
regressor variable to the model, this does not necessarily mean the new model
is superior to the old one. Unless the mor sum of squares in the new model is
reduced by an amount equal to the original error mean square, the new model
will have a larger error mean square than the old one because of the loss of one
degree of freedom for error. Thus,the new model will actually be worse than
the old one.

We now list some of the limitations of the regression models. Limitations

which we are going to list are in fact the limitations of regression analysis in
general and apply to all the regression models.

Limitations
1. Regression models are intended as interpolation equations over the
range of the regressor variable(s) used to fit the model. They may not be
valid for extrapolation outside of this range. For example, see Fig. 5.
A
Y

/ j [ I /i L

I
I I I
I I I
I I I
I I I

XI xz
I

x, x
>
58 . Fig. 5: The risk of extrapolation la regression.
 Suppose that data on y and x were called in the interval x I x 5 x2. Data Analysis and
Fitting Models to Data
Over this interval, the linear regression equation shown in Fig. 5 is a
good approximation of the true relationship. However, if this equation is
used to predict values of y for values of the regressor variable in the
region x 2 5 x Ix 3 , then model is useless over this range of x because of
equation error.

2. The position of the x-values play an important role in the least squares
fit. While determining the height of the line, all points have equal
weight. Whereas, the slope of the line is strongly influenced by the
remote values of x . For example, consider the data in Fig. 6. The slope
in the least squares fit depends heavily on either or both of the points A
and B . The remaining data would give a very different estimate of the
slop if A and B were deleted. Fig. 6: Two extreme
observations.
3. Regression techniques indicate a strong relationship between two
variables, this does not imply that the variables are related in any causal
sense. Our expectations of discovering c a q e and effect relationship
from regression should be modest. For example, if you look at the data
given in E6), you will see that the linear trend between x a d y does not
establish cause and effect between homework and test results.

4. In some applications of regression the value of the regressor variable x

required to predict y is unknown. For example, consider predicting
maximum daily load on an electric power generation system from a
regression model relating the load to the maximum daily temperature. To
predict tomorrow's maximum load, we must first predict tomorrow's
maximum temperature. Thus, the prediction of maximum load is
conditional on the temperature foreeast. The accuracy of the maximum
load forecast depends on the accuracy of the temperature forecast. All
these considerations must be taken into account when evaluating model
perfon~lance.

I You may now try some exercises.

E4) In 1978, a company conducted a study of the amount of additional oil

that can be extracted from existing oil wells by "enhanced recovery
techniques" (such as injecting solvents or steam into an oil well to lower
the density of the oil). The following table gives the study's estimates of
recoverable oil based on the price per litre
-

Price per Litre (Rs.) 92 102 97 75

Recovery 21.2 29.4 41.6 49.2

Use a best fit line to estimate the additional amount of oil that can be
economically recovered.

E5) The percentage of new plant and equipment expenditure by companies on

pollution control is as shown.
Introduction to
Mathematical Modelling

Use a linear regression model to estimate the figure for 1985.

E6) Students in a statistics class claimed that doing the homework had not
helped prepare them for the midterm exam. The exam score y and
homework score x for the 18 students in the class were as follows:

Fit a simple linear regression model to the data and interpret the resua.
Calculate R for the data.
- - --- - -

Fitting Exponential c o k e s
So far, we'have seen how to fit a straight line to a set of bivariate data i.e., data
giving relationship between two variables. A straight line is the simplest model
for a set of bivariate data, and is not appropriate if the scatter plot of the data
shows curvature. One class of models that accounts for curvature is the class
of exponential curves. Some examples of exponential curves are

Models (24)-(26) have two important features. They are monotone (either
increasing or decreasing), and they are easy to fit. Monotonicity is important
because often, in practice, a monotone relationship exists between two
variables. For example, if x is the level of traffic in a communications
network and y is the number of packets lost, it only makes sense for y to
increase as x increases. Models (24)-(26) are easy to fit using the technique
for fitting lines. This is because, when we transform them to a different scale,
they are actually lines. Which, if any, of these models is appropriate for a
particular set of data is best determined by drawing scatter plots of the
transformed data and see which transformation makes the data look the most
linear. Let us see how these transformations are done.

T@ Model y = boxb1 .
Taking logarithms on both sides of Eqn. (24) and then adding an error term, we
obtain
 Data Analysis and
Fitting Models to Data

or, y*=b;+b,x*+e

where, = l n y , x* = l n x and b;=lnb,.

You may notice that the model (27) is a line in the variables y* and x* . In
order to estimate the parameters, we can simply calculate ln (y) and ln (x) for
all the data points, and then use least squares to fit a straight line. We now
illustrate the method through an example.

Example 4: Consider the set of data given in Table 5.

Table 5

Use a best fit line to estimate the value of y when x = 3.36. Also obtain the
residual for the fitted line.

Solution: A graph of these data in Fig. 7 shows that a line is not the
appropriate model and suggests that model (24) might be reasonable.

Fig. 7: A scatter plot of the data in Example 4.

Taking logarithms of all the data poinis we obtain the values as given in
Table 6.
Table 6

X* = ln (x) 2.197 0.693 1.609 2.079 1.386 0 1.099 1.946 1.792

5.54 1.96 4.30 5.11 3.47 0.65 2.90 4.91 4.61

Y* = ln (Y)
Introduction to A scatter plot of the log-transformed data in Fig. 8 looks quite linear.
Mathematical Modelling

0 I I I I I >
.5 1 1.5 2.0 2.5 ln(y)
Fig. 8: A scatter plot of the log transformed data in Example 4.

To obtain the estimates for the parameters in the model we calculate

-
x ( x * -x*)' = 4.138
- -
Z ( x * -x*) (y* -ye)=9.315.
This gives,

and our fitted model is

In (9) = 0.5 1+ 2.25 1 In (x)

or,
9 = 1.67 x 2.251 ,
Thus, when x = 3.36, we would predict

Residuals for this fitted line could be computed using Eqn. (14). For example,

Similarly, the transformations for the models (25) and (26) can also be
obtained.
 For the Model y = b,ebl', taking logarithms on both sides of Eqn. (25) and Data Analysis and
adding an error terms, we obtain, Fitting Models to Data

ln(y)=lnb,+b,x+e
Of,

Thus, we can estimate the parameters of model (25) by calculating ln(y) for
each data point and then using least squares with h(y) as the dependent
variable and x as the independent variable.

For the model e y = b,xbl ,once again, taking logarithms, on both sides of
Eqn. (26) and adding an error term, we get

To have a better understanding of models (25) and (26), you can solve the
following exercises.

E7) Find a linear regression equation that best fit the data given in Table 7.

Table 7

E8) Find a linear regression equation that best fit the data given in Table 8.
Table 8

Just as we can fit a line or a curve to two-dimensional data, we can also fit a
plane or curved surface to three-dimensional data and a hyper-plane or
hyper-surface to four- or higher-dimensional data.

We shall now discuss models that are suitable to fit three and higher
dimensional data.

3.4 MULTIPLE LINEAR REGRESSION MODELS

ersurface~is the SWe for
0 predictor

The pro
,
, ,f flitmg models for le
with an exam
of the sand
surfaces i wh?
t ~ o ninvolving M
fines. CC' US s f d
Introduction to FormulatSoo
Mathematical Modelling
Suppose we want to develop a model for estimating the effective life of a
cutting tool based on two variables namely, the cutting speed and the depth of
cut. A regression model that might describe this relationship is

which is the equation for a plane in three dimensions where y denotes the
effective tool life, xl denotes the cutting speed, and x denotes the depth of
cut. b,, b, , b, are the unknown parameters of the model and e is the random
error. Eqn. (30) is a multiple linear regression model with two predictor
variables. The term linear is used because Eqn. (30) is a linear fbction of the
,
unknown parameters b ,b, and b .

In general, the response y may be related to k predictor variables. The model

y=b, +b,x, + b2xz + . . a + b k x k+ e (31)
is called a multiple linear regression model with k predictors. The
parameters b j, j = 0, 1, ..., k are called the regression coefficients and it
represents the expected change in the response y per unit change in x when
all the remaining predictor variables xi (i # j) are held constant. For this
reason the parameters b j, j = 1,2, ..., k are often called partial regression
coefficients.

Least Squares Estimation of the Parameters

Suppose that n > k observations are available, and let yi denotes the iU
observed response and xij denote the iU observation or level of regressor x j .
We assume that the error term e has the mean zero and variance oZ,and that
the errors are uncorrelated.

Estimation of Regression Coefficients

Let us use the method of least squares for estimating the regression coefficients
in model (3 1).

We write the sample model corresponding to Eqn. (3 1) as

yi = b, + blxil + b2xi, + - a . + bkxik+ e,

The least squares function is

The function S is to be minimized with respect to b,, b, , ...,b, . The least Data Anrlysls and
FlfflngModels to Data
squares estimators 6,. 6,, 6,, ...,6, of b, , b, , b, , ..., b, must satisfy

and

Simplifying Eqns. (34) and ( 3 9 , we obtain (k + I) least squares normal

equations, one for each of the unknown regression coefficients. The solution to
the normal equations will be the least squares estimators, 6,, 6,, ...G1,.

It is more convenient to deal with multiple regression models in matrix

notation. Using matrix notation Eqn. (32) can be written as

1 XI, x,, .-.

where y =

Yn 1 xnl xn2 ...

Now we want to find the vector b, which minimizes

= y'y - b x'y - y' xb + b'x'xb

= y 'y - 2b x'y +b'x' xb (37)
since b'x'y is a (1x 1) matrix, or a scalar, and its transpose (b'x'y)' = y'xb is
the same scalar.

The least squares estimators then must satisfy

which gives x'x b = x'y (38)

Eqns. (38) are the least squares normal equations. To solve the normal
equations multiply both sides of Eqns. (38) by the inverse of x'x . Thus, the
lest squares estimator of b is
Introduction to provided (xlx)-' exists. The (x'x)-' matrix will always exist if the
Mathemtied Modelling
regressors are linearly independent. Using the value of 6 given in Eqn. (39),
the fitted value of y can be written as

= x (xlx)-' x'y
=Hy.
The n x n matrix H = x (x' x)-' x' is called the 'hat' matrix. The residual
ermr vector can be written as
e=y-9 (41)

We shall now illustrate this concept through an example.

Example 5: Find a linear regression equation that best fit the data given in
Table 9.
Table 9

Solution: Computations for 6 are as follows:

 Data Analysis and
Fitting Models to Data

Thus the least squares fit is

9 = 5.375 + 3.012x, - 1 . 2 8 6 ~ ~ .

In addition to estimating the regression coefficients we now obtain an estimate

of the variance of error for the multiple regression model.

Estimation of a2

As in simple linear regression, we obtain an estimator of a2 from the residual

sum of squares

= e'e .
Substituting e = y - x b in Eqn. (43), we get
SS, =(y - x i ) ' ( y - x 6)

=y1Y-2b'x'y +G1x'xb.
Since x' x 6 = x' y ,Eqn. (44) reduces to
1

SS, = y l y - b ' x ' y . (45)

The residual sum of squares has n - (k + 1) degrees of freedom associated with
it since (k + 1) parameters are already estimated in the regression model.
Thus, an estimate of a2 is given by

&2 = SS' =MS., the residual mean square. (46)

n - (k + 1)

Example 6: Estimate the error variance oZ for the multiple regression model
fit to the data in Example 5.
Solution: For the data in Example 5, we have
SS, = y ' y -&'my
Introduction to Just as in the case of simple linear regressions model adequacy can be
evaluated in the case of multiple regression models also.

Coefficient of Multiple Determinations

The coefficient of multiple determination R is defined exactly the same way

as in Eqn. (22) i.e,

R in this case is a measure of the reduction in the variability of y obtained

,
by using the regressor variables x, , x , ..., x, . As in the simple linear
regression case, we must have 0 S R 2 1 . However, a large value of R does
not necessarily imply that the regression model is a good one. Adding a
regressor to the model will always increase R ', regardless of whether or not
the additional regressor contributes to the model. Thus, it is possible for
models that have large values of R 2 to perform poorly in prediction or
estimation.

Limitations
All the limitations mentioned in Sec. 3.3 apply to multiple linear regression
models also.

The linear regression model y = xb + e is a general model for fitting any

relationship that is linear in the unknown parameters b . This includes the
important class of polynomial regression models. We now introduce you to
polynomial regression models in one variable.

Polynomial Regression Models in One Variable

Polynomial models in one variable are models of the form
y = bo + b,x + b2x2+ e (47)

or, y=bo+b,x+b2~2+b3~3+e (48)

and so on. Model $47) is a quadratic model and model (48) is a cubic curve. In
general the k* order polynomial model in one variable is

Unlike exponential models, polynomial models are non-monotonic. A

quadratic curve has one inflection point and a cubic curve has two inflection
points. Thus, a quadratic model is useful for cases where increasing the
explanatory variable has one effect up to a point, and the opposite effect
thereafter. For example, in a chemical experiment, increasing the reaction
temperature might increase the yield up to some optimal temperature, after
,which a M e r increase in temperature would cause the yield to drop off. In
an agricultural experiment, increasing fertilizer will probably increase plant
growth up to a point, after which more fertilizer will cause a chemical
imbalance in the soil and may hinder plant growth. There are examples where
quadratic and cubic models can be useful. However, it is usually difficult to Data Analysis and
Fltting Models to Data
justify the use of a high-order polynomial because, generally, no physical
explanation justifies the large number of inflection points. Thus, such models
are seldom used. Moreover, there are several problems that arise when fitting a
high-order polynomial in one variable. We shall not be going into those details
here. Here we shall only concentrate on quadratic polynomial and illustrate its
best fit for a given data through an example.

If we let x, = x and x, = x in the model given by Eqn. (47) then it reduces

to
y = b, + blxl + bzx2 + e (50)
which is a multiple linear regression model with two regressor variables and
can be treated by the method discussed in previous section. Let us consider an
example to understand the procedure.

Example 7: The data given in Table 13 is obtained by a process engineer while

studying the relationship between a vacuum setting and particle size
distribution for a granular product.
Table 13

Vacuum 18 18 20 20 22 22 24 24 26 26
setting = x
Particle 4.0 4.2 5.6 6.1 6.5 6.8 5.4 5.6 3.3 3.6
size = y

Fit a linear regression model to the data.

Solution: A scatter plot of the given data is shown in Fig. 9. It appears that the
data has a somewhat parabolic curvature

Vacuum setting
Flg. 9: A Matter plot of the data in Example 7.

We fit the model

, ,
to the given data. Using x = x and x = x Z,we can transform the above
model to
Introduction to y = b , +b,x, +b,x, + e .
Mathematlcd Modelling
We now use the method used in Example 5 to estimate the regression
coefficients.

x'y = 1117.60
40):;24[:

Thus, the least squares fit is

The above quadratic curve has a maximum, which can be obtained as

Corresponding to x = 21.82 the predicted value of y is

Thus, to maximize the particle size in the product the best setting for the
vacuum is at 21.82.
***
You may now try the following exercises.
 Data Aarlysls and
Fitting Models to Data
E9) The yearly fluctuations in the groundwater table is believed to be
dependent on the annual rainfall and the volume of water pumped out
from the basin. The data collected on these variables for a period of 10
years is given in Table 10.
Table 10

Establish the multiple linear regression equation among the variables.

E10) The mileage performance of a car is considered to be dependent on the

engine displacement and the horsepower of the engine. The data
collected on these variables for 17 cars is given in Table 11. Find a linear
regression equation that best fit the data. Also obtain the error variance
of the best fit.
Table 11
Introduction to El I) Table 12 presents data concerning the strength of kraft paper and the
Mathematical Modelling percentage of hardwood in the batch of pulp from which the paper was
produced.
Table 12
-
Hardwood concentration % = (x) Tensile strength = y
1 6.3
1.5 11.1
2 20.0
3 24.0
4 26.1
4.5 30.0
5 33.8
5.5 34.0
6 38.1
6.5 39.9
7 42.0
8 46.1
9 53.1
10 52.0
11 ' 52.5
12 48.0
13 42.8
14 27.8
15 21.9

Find a linear regression model that best fit the data.

El 2) The sale price of a holiday cottage depends on the age and livable area of
the cottage. Find a linear regression model that best fit the data given in
Table 13. Also find the residual and the residual mean square for the
data.
Table 13

Price in thousand rupees Age (years) Area (mZ)

745 36 66
895 37 68
442 47 64
440 32 53
' 1598 1 101

We now end this unit by giving a summary of what we have covered in it.

3.5 SUMMARY
In this unit, we have covered the following points.

1. Data visualization is the study of the visual representation of data. It

provides information abstracted in some schematic form.
2. In statistics, regression analysis is a technique in which regression ~ a t AnaIysls
a and
Flttlng Models to Data
models are formulated to analyse numerical data consisting of values of a
dependent variable (also called response variable) and of one or more
independent variables (also known as explanatory variables or
predictors).

3. Regression models involving only one predictor are called simple

regression models and those involving two or more predictors are called
multiple regression models.

4. Regression models are linear if they are linear in the parameters involved
in the model.

5. Method of least squares can be used to fit a line or a curve to

two-dimensional data and a plane or curved surface to three-dimensional
data.
6. Second order polynomial regression models in one-variable fit a
quadratic curve to a given set of data which can be converted to multiple
regression model by using the transformation of the independent
variables.

El) You can discuss scatter plot, histogram, bar diagram, box plot, block plot,
etc. You can illustrate them by considering a type of data for which each
of them is most suited. For example, in case of continuous data
histogram is preferred whereas, discrete data is commonly represented by
a bar diagram.

E2) Cartographs in weather bulletins are used to compare data on a

geographical basis. Think of other similar examples.

b) Do as in a) above.

E4) 6 , = 93.727,6, = -0.638, 3 = 93.727 - 0.638x, where x is price per

litre and y is recovery.

E5) 3=538.155-0.268 x when x=1985, y=6.2

Introduction to Plotting the regression equation and 18 points, it is apparent that the
Mathematical Modelling
slope 6, is the rate of change of 9 as x varies and the intercept 6 , is
the value of 9 at r = 0. But the linear trend does not establish cause and
effect between homework and test results.

E7) Draw a scatter plot of the data and see that a line is not an appropriate
model. Transform the data by taking log of y's and obtain

Check that the scatter plot of the above data looks linear.
The best fit linear regression line is
9, = 0.73 + 4.209 x
or, jl=e 0.73+4.209~= 2.075 e4.209~

E8) Taking log of x's the data transforms to

The scatter plot of the above data indicates that linear model is
appropriate for the data. The best fit regression line is
9=-4.1+4.62 xl
=-4.1+4.62 hx.

E9) 9 = 47.5 + 0.56 x, - 15.3 x , ,where xl is annual rainfall, x, is ground

water volume and y is water table in cms.

E10) 9=46.429-0.109~~ - 0 . 0 5 8 ~ ~Errorvariance

. =-s s e ,where x, is
14
horsepower, x, is engine displacement and y is miles per gallon.

El 1) Draw the scatter plot of the given data and check that the model to be
fitted is
y=b, +b,x+b,x 2
,
Put x, = x and x = x * in the above model and proceed exactly as in
Example 7 to obtain 9 = -6.6959 + 11.7703~- 0.635x2.
 Data Anllydr and
Fifflng Modcb to Data
Introduction to
Mathematical Modelling PRACTICAL EXERCISES

Sessions 1 and 2
1. An electric utility is interested in developing a model relating peak hour
demand (y) to total energy usage during the month (x) . Data for 53
residential customers for the month of August, 2005 are shown in
Table 1.
Table 1

Customer x(KWH) y(KW) Customer x(KWH) y(KW)

1 679 .79 27 837 4.20
2 292 .44 28 1748 4.88
3 1012 .56 29 1381 3.48
4 493 .79 30 1428 7.58
5 582 2.70 31 1255 2.63
6 1156 3.64 32 1777 4.99
7 997 4.73 33 370 .59
8 2189 9.50 34 2316 8.19
9 1097 5.34 35 1130 4.79
10 2078 6.85 36 463 .5 1
11 1818 5.84 37 770 1.74
12 1700 5.21 38 724 4.10
13 747 3.25 39 808 3.94
14 2030 4.43 40 790 .96
15 1643 3.16 41 783 3.29
16 414 .50 42 406 .44
17 354 .17 43 1242 3.24
18 1276 1.88 44 658 2.14
19 745 .77 45 1746 5.71
20 435 1.39 46 468 .64
21 540 .56 47 1114 1.90
22 874 1.56 48 413 .5 1
23 1543 5.28 49 1787 8.33
24 1029 .64 50 3560 14.94
25 710 4.00 51 1495 5.1 1
26 1434 .3 1 52 222 1 3.85
53 1526 3.93
I a

Write a program in C-language to

a) find a linear model that best fit the data using the least squares
estimates and estimate the peak hour demand for a total of 2050
KWH energy usage.
b) find R for the model.

A manufacturer of particle boards is interested in the strength of particle

boards as a function of the baking temperature. The data from an
experiment designed to study this relation are given in Table 2.
 Table 2 Data Analysis and
Fitting Models to Data
Strength Temp. Strength Temp.
66.30 40 75.78 . 55
64.84 40 72.57 55
64.36 40 76.64 55
69.70 45 78.87 60
66.26 4.5 77.37 60
72.06 45 '15.94 60
73.23 50 78.82 65
71.40 50 77.13 65
68.85 50 77.09 65

Write a programme in C-language to

a) obtain a best fit line using the least squares estimates,

b) fit a quadratic model using the least squares estimates.

c) compare the models in a) and b) above by finding R' for each of
them.

3. When gasoline is pumped into the tank of a car, vapors are vented into
the atmosphere. An experiment was conducted to determine whether y ,
the mount of vapour, can be predicted using the following four variables
based on initial conditions of the tank and the dispensed gasoline:
x, = tank temperature
x, = gasoline temperature
x, = vapour pressure in tank
,
x = vapour pressure of gasoline
The data are given in Table 3.
Table 3

Write a program in C-language to

77
Introduction to a) fit a linear regression model to the data using the least squares
Mathematical Modelling estimates.

b) find the 'hat' matrix.

c) find the residual error.

d) find R for the model.