Scribd
Upload
41 views

Towards Credibible Simulation Results

This document discusses techniques for improving the credibility of simulation results. It covers: 1) The simulation life cycle and importance of automation tools to guide construction, execution, and analysis of simulations. 2) Issues that can introduce bias like initial conditions, correlated data, and unknown required data amounts. It presents techniques like warm-up periods and independent replications to address these issues. 3) The importance of relative precision in determining stopping criteria for simulations to balance accuracy and cost. Various statistical tests are presented for analyzing correlated replications to improve confidence in results.

Uploaded by

Edjair Mota
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as ODP, PDF, TXT or read online on Scribd
41 views

Towards Credibible Simulation Results

This document discusses techniques for improving the credibility of simulation results. It covers: 1) The simulation life cycle and importance of automation tools to guide construction, execution, and analysis of simulations. 2) Issues that can introduce bias like initial conditions, correlated data, and unknown required data amounts. It presents techniques like warm-up periods and independent replications to address these issues. 3) The importance of relative precision in determining stopping criteria for simulations to balance accuracy and cost. Various statistical tests are presented for analyzing correlated replications to improve confidence in results.

Uploaded by

Edjair Mota
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as ODP, PDF, TXT or read online on Scribd
You are on page 1/ 21

06/10/14 UPV/DISCA/GRC 2014 1

Towards Credibible
Simulatio Results
Edjair Mota
[email protected]
UPV/DISCA/GRC 2014 2 06/10/14
!utlie

T"e simulatio li#e $%$le

Automatio tools

Pat" to Automatio

Iitialisatio &ias

Correlated Data

'ial Relati(e Pre$isio


UPV/DISCA/GRC 2014 ) 06/10/14
T"e *i#e Ci$le o# a Simulatio Stud%
06/10/14 UPV/DISCA/GRC 2014 4
So+"isti$ated
+a$,a-es
UPV/DISCA/GRC 2014 . 06/10/14
Automatio Tools
/T"e use o# automatio tools to -uide
t"e $ostru$tio0 t"e e1e$utio0 ad
t"e aal%sis o# results i simulatio
e1+erimetatio $a im+ro(e t"e
$redibilit% o# simulatio studies o#
wireless etwor,s23
Enhancing the Credibility of Wireless Network imulations with E!periment "utomation# $%errone et al.&
UPV/DISCA/GRC 2014 6 06/10/14
T%+es o# Simulatio

Terminating

S+e$i#i$ starti-0 sto++i- $oditios

Ru le-t" well4de#ied 5ad #iite6

Non-terminating7

*o-4ru 5te$"i$all% #ore(er6

T"eoreti$all%0 iitial $oditios do8t matter 5but


+ra$ti$all% t"e% usuall% do6

9ot $lear "ow to termiate a simulatio ru


UPV/DISCA/GRC 2014 : 06/10/14
Pat" to Automatio
Initial Bias
'nitial settings of
the simulation
model.
Correlation
Precision
" common way to
deal with this
issue( warm up.
UPV/DISCA/GRC 2014 ; 06/10/14
T"e Iitialisatio Problem
UPV/DISCA/GRC 2014 < 06/10/14
T"e Iitialisatio Problem

=et"ods #oud i t"e literature7

Ver% lar-e ru le-t" to ma,e t"ose e##e$ts e-li-ible>

Set t"e iitial $oditio $lose to t"e /most li,el%3


be"a(ior o# t"e s%stem i stead%4state o+eratio to
dimiis" t"e trasiet be"a(ior>

Determie a o+timal tru$atio +oit 5statisti$s4


$leari- time6>

*et t"e model warm u+ to t"at +oit i simulated


time> ad t"e delete t"e warm4u+ data ad base t"e
aal%sis o t"e remaii- tru$ated data2
UPV/DISCA/GRC 2014 10 06/10/14
T"e Iitialisatio Problem

?arm u+ ad tru$atio =et"od7

Re@uires a reliable met"od #or estimatio o# t"e


e##e$ti(e le-t" o# t"e warm4u+ +eriod2

Remo(al o# a% obser(atios de$reases t"e sam+le


siAe ad t"us i$reases t"e (aria$e o# t"e rele(at
sam+le statisti$s2
UPV/DISCA/GRC 2014 11 06/10/14
=SBR4. 5=ar-ial Stadard Brror Rule6
)
*
)
+
)
,
)
-
)
.
)
/
)
0
)
1
)
2
)
k
...
Simulation
Model
ME3$*&
ME3$+&
ME3$,&
ME3$-&
ME3$.&
y
*
y
+
y
,
y
-
y
.
y
/
y
0
y
1
y
2
y
*4
y
**
y
*+
... ... ... ... y
n

X
1
=
1
5

i=1
5
y
i
X( k, d)=
1
kd

j=d+1
k
X
j
S
(k , d)
2
=
1
kd

j=d+1
k
(X
j
X(k , d))
2
X( k, d) z
1/ 2
S
( k, d)

kd
d54 d5* d5+ d5, d5- d5.
UPV/DISCA/GRC 2014 12 06/10/14
=SBR4.

Ad(ata-es7

Base o# uderstadi- ad im+lemetatio>

Ie1+esi(e $om+utatio>

B##i$ie$% i +reser(i- re+resetati(e simulatio


data>

B##e$ti(eess i miti-ati- t"e iitial bias2


UPV/DISCA/GRC 2014 1) 06/10/14
Pat" to Automatio
Initial Bias
'nitial settings of
the simulation
model.
6bser7ations are
strongly correlated.
Correlation
Precision
" common way to
deal with this
issue( warm up.
" common way to
deal with this issue
independent
replications.
UPV/DISCA/GRC 2014 14 06/10/14
Correlated Data
Simulation
Model
)
*
)
+
... )
n
X(n) z
1/ 2

S
2
(n)
n
S
2
(n) =
1
n1

i=1
B
(X
i
X(n))
2
IID
X(n) =
1
n
X
i
UPV/DISCA/GRC 2014 1. 06/10/14
&at$"i- =et"ods
Simulation
Model
)
*
)
+
... )
m
)
m8*
)
m8+
... )
+m
... ... ... ... )
n
X
1
(m) X
2
(m) X
b
(m)
10B30
UPV/DISCA/GRC 2014 16 06/10/14
&artelsC Test
Simulation
Model
)
*
)
+
... )
m
)
m8*
)
m8+
... )
+m
... ... ... ... )
n
X
1
(m) X
2
(m) X
b
(m)
R
1
R
2
R
b
RVN =

1=1
n1
( R
i
R
i +1
)
2

i =1
n
( R
i
R)
2
X =
1
b

1
b
X
i
H =
t
b1, 1/ 2
b(b1)

i =1
b
(X
i
X)
2
UPV/DISCA/GRC 2014 1: 06/10/14
!(erla++i- &at$" =eas
)
*
)
+
)
,
...)
m
)
m8*
)
m8+
... ... )n

)
+
)
,
...)
m
)
m8*

)
,
...)
m
)
m8*
)
m8,

X =
1
b

1
b
X
i
H =
t
1.5( b1), 1/ 2
b(b1)

i=1
b
(X
i
X)
2
=
H
X
0.05
b = n m + 1
X =
1
b

1
b
X
i
X
1
(m)
X
2
(m)
X
3
(m)
UPV/DISCA/GRC 2014 1; 06/10/14
Co(era-e o# t"e Results
ystem load
C
o
n
f
'
d
e
n
c
e
$9&
UPV/DISCA/GRC 2014 1< 06/10/14
Pat" to Automatio
Initial Bias
'nitial settings of
the simulation
model.
6bser7ations are
strongly correlated.
:he total amount
of re;uired data is
unkonwn a priori.
Correlation
Precision
" common way to
deal with this
issue( warm up.
" common way to
deal with this issue
independent
replications.
" common way to
deal with this
issue( look at the
data se;uentially.
UPV/DISCA/GRC 2014 20 06/10/14
Relati(e Pre$isio
Simulation
Model
X =
1
b

1
b
X
i
H =
t
1.5( b1), 1/ 2
b(b1)

i =1
b
(X
i
X)
2
=
H
X
0.05
X =
1
b

1
b
X
i
)
*
)
+
)
,
...)
m
)
m8*
)
m8+
... ... )n

)
+
)
,
...)
m
)
m8*

)
,
...)
m
)
m8*
)
m8,

X
1
(m)
X
2
(m)
X
3
(m)
06/10/14 UPV/DISCA/GRC 2014 21
T"a, DouE

You might also like