ANALYSIS OF

SIMULATION OUTPUT
Durgaraj katuwal
Why analysis of simulation
output ?
■ Many simulation includes some sort of randomness, which can arise in
a variety of ways e.g. in a simulation of manufacturing system, the
processing times required at a station may have random variations or
the arrival times of new jobs may not be known in advance.
■ In a bank, customers arrive at random times & amount of time spent
at a teller is not known beforehand. Because of the randomness of the
components driving a simulation, the o/p from simulation is also
random. So, statistical techniques must be used to analyze the
results.
■ To test different ideas, to learn about the system behavior in new
situation, to learn about simulation model and the corresponding
simulation system.
Nature of the problem
■ Once a stochastic variable has been introduced into simulation model,
almost all the system variables describing the system behavior also
become stochastic.
■ Hence it needs some statistical method to analyze the simulation
output. •
■ A large body of statistical methods has been developed over the years
to analyze results in science, engineering and other fields.
■ It seem natural to attempt applying these methods to analyze the
simulation output but most of them pre-suppose that the results are
mutually independent (IID) and the simulation process almost never
produce raw output that is IID. For example: customer waiting times
from queuing system are not IID. Thus it is difficult to apply classical
statistical techniques to analysis of simulation
Nature of the problem
■ Once a stochastic variable has been introduced into simulation model, almost all the
system variables describing the system behavior also become stochastic.
■ Hence it needs some statistical method to analyze the simulation output.
■ A large body of statistical methods has been developed over the years to analyze
results in science, engineering and other fields.
■ It seem natural to attempt applying these methods to analyze the simulation output
but most of them pre-suppose that the results are mutually independent (IID) and
the simulation process almost never produce raw output that is IID. For example:
customer waiting times from queuing system are not IID. Thus it is difficult to apply
classical statistical techniques to analysis of simulation model.
■ Making a variable stochastic changes the properties of remaining variables to
stochastic since the endogenous events make one variable depend upon another
■ While characterizing the property of a variable, in every time unit the value
changed is compared to predefined intervals known as confidence interval to find
similarity between estimated and simulated value
■ Simulation results are not mutually independent definitions
Estimation Methods
probability distribution with a finite mean μ and finite variance 𝜎 2 .
■ A random variable is drawn from an infinite population that has a stationary

■ Random numbers that meet all these conditions are said to be IID (Independently
and Identically Distributed) variable for which the central limit theorem can be
applied.
The theorem states that the sum of 𝑛 IID variable drawn from a population that
has mean μ and a variance of 𝜎 ^2 is approximately distributed as a normal
■

variable with mean 𝑛𝜇 and variance 𝑛𝜎^2

■ . Any normal distribution can be transform into a standard distribution that has a
mean of 0 and variance of 1.
■ Let xi (i=1,2,3,……….n) be n IID random variables. Using central limit theorem, we
have normal Variate
Estimation Methods

■ In terms of sample mean ̅𝑥, ( Dividing by n in both denominator and

numerator, we get
Estimation Methods
with ordinary mean 𝜃, then point estimator
■ a) Point estimation (For discrete data): Let y1, y2 ...yn be discrete time data

■ Is unbiased if its expected value is 𝜃 i.e. E( 𝜃` )= 𝜃 and is biased if E( 𝜃` )≠ 𝜃

and the difference E(𝜃`) - 𝜃 is called bias of 𝜃`
■ As an example of a point estimate, assume you wanted to estimate the mean
time it takes 12-year-olds to run 100 yards. The mean running time of a
random sample of 12- year-olds would be an estimate of the mean running
time for all 12-year-olds. Thus, the sample mean M, would be a point estimate
of the Population mean, μ.
Estimation Methods
■ Example 5.1 Q.N.->Following are the random sample of height of
people of the town. If the population mean is 6.1 ft., find the bias of
the point estimator.

Solution : 𝜃` = (5.5 + 6.1 + 5.7 + 6.6 + 5.2 + 6.0 + 5.6 + 6.3 + 5.9 +
5.8)/10 = 5.87 Now, bias of estimator = 5.87 – 6.1 = - 0.23
Estimation Methods
Estimation Methods
b) Interval Estimation/Confidence Interval Estimation
Confidence interval is a measure used to analyze the correctness of the
point estimator. In the above example we got point estimation of height
of the people of a city but we don't know whether to accept or reject it.
Confidence intervals are based on the premise that the data being

Suppose the model is the normally distributed with mean 𝑥 , Variance 𝜎

produced by the simulation is represented well by a probability model.

2 and we have a sample of n size then the confidence interval is given

by:
Estimation Methods
■ In practice, the population variance is usually not known; in this case
variance is replaced by the estimate calculated by the formula

terms of estimated variance s 2 , the confidence interval for 𝑥 is

■ This has a student t-distribution, with n – 1 degree of freedom. In

defined by
Estimation Methods

Here, the quantity tn-1, α/2 is found on the student distribution table.
Example 5.2 Q.N.->
The daily production time of a product in a factory for
120 days is 5.8 hours and sample standard deviation
(S) is 1.6. Calculate confidence interval for 95%
confidence level.
Example 5.3
Q.N.->Following are the random sample of marks of
student of the orchid college. If the population mean
is 51.1 , find the bias of the point estimator.
51, 52, 56 ,50, 48 ,49, 59.8, 45.9, 58, 47
Simulation Run statistics
■ In the estimation method, it is assumed that the observations are
mutually independent and the distribution from which they are draws is
stationary.
■ Unfortunately many statistics of interest in simulation do not meet these
conditions.
■ For example: consider a single server system in which the arrival occurs
with poison distribution and service time has an exponential and queue
discipline is FIFO.
■ Where, Poisson distribution means a discrete frequency distribution which
gives the probability of a number of independent events occurring in a
fixed time.
■ Suppose the study objective is to measure the mean waiting time. In
simulation run, the simplest approach to estimate the mean
waiting time by accumulating the waiting time of n successive
entities and dividing by n. this is the sample mean, denoted by
Waiting time measured in this way is not independent.
Simulation Run statistics
■ Whenever a waiting line forms, the waiting time of each entity on the
line clearly depends upon the waiting time of its predecessors. Such
data are called auto-correlated. Another problem that must be faced
is that distribution is not stationary. TE early arrivals get the service
quickly, so a sample mean that include early arrivals is biased. The
following figure show the mean waiting time for different sample
sizes.
Simulation Run statistics
■ Consider a single server system in which the arrival occurs with a
Poisson distribution and the service time has an Exponential
distribution. Suppose the study objective is to measure the mean
waiting time defined as the time entities spend waiting to receive

𝑀/𝑀/1.
service (excluding service time). This system is commonly denoted by

■ Here 𝑀/𝑀/1 indicates,

■ 1st 𝑀 = interarrival time is distributed exponentially
■ 2nd 𝑀 = service time is distributed exponentially 1 = one server
Simulation Run statistics
waiting time by accumulating the waiting time of 𝑛 successive entities
■ In simulation run the simplest approach is to estimate the mean

and divide by 𝑛. This measure the sample mean denoted by 𝑥̅𝑛(). If

𝑥𝑖(𝑖 = 1,2, … … , 𝑛) are the individual waiting time then,
Simulation Run statistics
■ Whenever a waiting line forms the waiting time of each entity on the
line depends upon the waiting time of its predecessor. Any series of
data that has this property of having one value affect the other values
is said to be auto correlated. The sample mean of auto correlated data
can be summed to approximate a normal distribution as the sample
size increases
Simulation Run statistics
■ A simulation run is started with the system in some initial state,
frequently the idle state in which no service is being given and none
entities are waiting. The early arrivals that have a more than normal
probability of obtaining service quickly so a sample mean that
includes the early arrivals will be biased.
■ For a given sample size starting from a given initial condition the
sample mean distribution is stationery, but if the distribution would
be compared for different sample sizes the distribution will be slightly
different.
■ The following figure is based on theoretical results which shows how

for 𝑀/𝑀/1 system, starting from an initial empty state with a server
the expected value of sample mean depends upon the sample length

utilization of 0.9.
Replications of Runs
■ One way of obtaining independent result is to repeat simulation.
Repeating the experiment with different random numbers for the

mean 𝑥 . Suppose the experiment is repeated p times with

sample size n gives a set of independent determination of sample

independent random values of n sample sizes.

the variance for the j th run is denoted by 𝑥j(n) and 𝑠𝑗 2 (𝑛)

■ Let xij be the ith observation in jth run and let the sample mean and

respectively. Then for jth run, the estimates are

Replications of Runs

following estimate for the mean 𝑥 and variance s2 of the populations as

■ Combining the result of p independent measurement gives the

experiment results for the 𝑀/𝑀/1 system.

■ The following figure shows the result of applying the procedure to
Elimination of initial bias

The initial bias as shown in figure below needs to be removed. Two

general approaches can be taken into remove the bias.
1. The system can be started in a more representative then an empty
state.
2. The first part of the simulation can be removed.
Elimination of initial bias
■ The ideal situation is to know the steady state distribution for the
system and select the initial condition for the distribution.
■ The more common approach to know the initial bias is to eliminate on
initial selection of runs.
■ The run is started from an idle state and stop after the certain period of
time. The run is then restarted with statistics being gathered form the
point of restart.
Elimination of initial bias

■ It is usual to program the simulation so that statistics are gathered

from the beginning and simply wipe out the statistics gathered up to
the point of restart.
■ No sample rules can be given beside how long an interval should be
eliminated.

■ The disadvantage of eliminating the first

part of the simulation run is that the
estimate of the variance needed to
estimate a confidence limit, mostly based
on less information.
Title Lorem Ipsum dolor

LOREM IPSUM LOREM IPSUM LOREM IPSUM

DOLOR DOLOR DOLOR