Scribd
Upload
5 views

INDU6311_Lecture7a

Uploaded by

aditya thakkar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
5 views

INDU6311_Lecture7a

Uploaded by

aditya thakkar
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 20

Lecture 7(a): Ensuring

“Correctness” in Simulation

1
Outline

 Verification
 Validation
 Credibility

2
Readings

 Law – Chapter 5
 Sargent, "Verification and Validation of
Simulation Models” (paper posted on
Moodle)

3
Verification
System description and assumptions

Is the
translation
correct?

Simulation program

4
Verification Techniques
 Technique 1: start from smaller/simpler
models or sub-models
Why?

Example: for a multi-teller bank with balking,


write a simulation without balking first

Balking occurs when potential customers arriving at a queueing system


choose not to enter it.
5
Verification Techniques
 Technique 2: second opinion
Why?

Example: divide your team into sub-teams


working on different parts of the model, then
give to each other to determine and/or fix bugs

6
Verification Techniques
 Technique 3: test the model under a
variety of conditions and check if the
output is reasonable
Why?

Example: check that utilization is reasonable


under a variety of arrival and processing rates

7
Verification Techniques Why?

 Technique 4: trace – display the values


of variables, attributes, counters after
every event Arena: step
 Compare to hand calculations

Example: check that utilization is reasonable


under a variety of arrival and processing rates

8
Verification Techniques Why?

 Technique 5: run the model under


simplifying assumptions for which its
characteristics are known or easily
computed

Example: for a job shop model with multiple


stations, run a simpler model with one station
(for each station) and compare to analytical
results from queueing theory
9
Verification Techniques Why?

 Technique 6: observe animation of the


simulation output

Example: queue build-up at a station, more


entities going on a particular branch of the
decide module

10
Verification Techniques Why?

 Technique 7: double-check your input


(distributions, parameters, time units)

Example: Expo(mean) – did you use the mean


or the rate?

rate

11
Validation
Objectives of the study
Accurate
representation
for the
objectives?

Simulation program

12
Validation Techniques
 Validation Technique 1: comparison
with existing system
 Sample statistics
 Confidence intervals
 Formal tests

Compare current state performance with


current actual performance! (Q: what
performance metrics would you use?)
13
Validation Techniques
 Validation Technique 2: comparison
with expert opinion
 Stakeholders validate that the output of
the model is representative of reality
 Stakeholders cannot tell difference
between output from model and reality
(i.e., model should produce output as
“good” as the real system)

14
Validation Techniques
 Validation Technique 3: comparison
with another model
 If the other model exists and has already
been validated 😏

15
Validation Techniques
 Validation Technique 4: comparison via
animation
 Determine if the system behaves “as
expected” via animation
 E.g., are all potential paths of the entities
represented? Are the queues building up
where we expected?

16
Validation Techniques
 Validation Technique 1: numerical
comparison with existing system
 Sample statistics
 Confidence intervals
 Formal tests

Focus on this one if possible!

17
Establishing Credibility
Stakeholders
Do
stakeholders
accept the
model?

Simulation program

18
Techniques for Establishing
Credibility
 Technique 1: Maintain contact with the
stakeholders (show them the model,
with animation!)

 Technique 2: Maintain a list of


assumptions/specifications (refer to it
while interacting with the stakeholders!)

19
Questions?

20

You might also like