Uncertainty Analysis

for
Flow Measurements and Techniques
using
Standardized Methodology
Overview

 Background
 Uncertainty Analysis (UA) Frameworks
 AIAA (1995)
 UA Implementation Example
 Methodology
 Assessment of Elemental Uncertainties
 Customized GUI for UA Implementation
 Conclusions
 Outlook
 Background
SAMPLE REQUEST
regarding uncertainty analysis originated from a Hydrologic Service

…. Has anybody out there had to defend the validity of an ADCP flow
measurement against a legal challenge from a third party?
……When current meters were used to undertake these measurements we could
claim that the flow measurement was undertaken in conformance with British
and International standards for current meter gauging and that the current meter
had a valid calibration certificate…
In the case where flow measurements are now taken using ADCPs we feel more
vulnerable to legal challenges. This is for two reasons:
1. There is no ISO document in place. The Agency has to rely on its own internal
document on gauging procedures which is based on the draft ISO document.
2. ADCPs do not have "certificates of calibration" . The only checks on the
performance that can be made are against other ADCPs or other types of flow
monitoring equipment.

(posted on the USGS’ Hydro-Acoustics Work Group webpage by

R. Iredale, The Environment Agency of England and Wales, 2005)
 UA Frameworks
Over the last 50 years, considerable efforts have been made by professional societies
to develop and implement uncertainty analysis (UA).
One of the rigorous UA methodology (based on sound statistical and engineering
concepts):
Guide to the Expression of Uncertainty Measurement (ISO, 1993)
- adopted widely by various scientific & research communities, e.g., NIST (1994), NF
ENV 13005 (1999)
- the guide recognizes the need for further adaptation for specific areas

Specific adaptations for engineering:

- Assessment of Wind Tunnel Data Uncertainty (AIAA, 1995)
- Test Uncertainty (ASME, 1998)

Key assumptions/concepts for ISO (1993)-based standards

- Gaussian pdf-s for the error sources
- 2 sample standard deviations for 95% confidence level
- for large samples (N ≥ 10), special procedures for handling small samples
- RSS used for combining uncertainties
- Taylor-series expansion for propagation of uncertainties
- total uncertainties expressed using confidence intervals
 UA Frameworks
Terminology for ISO (1993) - based standards
ISO (1993) AIAA-S-071-1995 ASME PTC 19.1-1998
Uncertainty of a measurement
 input quantity  individual variable  independent parameter
 type A standard uncertainty  bias limit  systematic uncertainty
 type B standard uncertainty  precision limit (differently estimated  random uncertainty (differently
 combined standard uncertainty for single and multiple tests) estimated single and multiple tests)
 total uncertainty  measurement uncertainty
Uncertainty of a result
 functional relationship  data reduction equation derived result
 bias limit  systematic uncertainty
 precision limit (differently estimated random standard deviation
for single test with single readings (differently estimated for single
and averaged readings, multiple tests) and multiple tests)
 sensitivity coefficients  sensitivity coefficients  sensitivity coefficients
 combined standard uncertainty  combined standard uncertainty  uncertainty of the result (accounts
(accounts for correlated errors) (accounts for correlated errors) for correlated errors and the level
 expanded uncertainty (accounts  uncertainty at specified confidence of confidence)
for the level of confidence) level
 coverage factor f( degrees of  coverage factor f( degrees of freedom  coverage factor f(degrees of
freedom and t Distribution) and t Distribution) freedom and t Distribution)

The 3 standards provide the same total measurement uncertainty

 AIAA (1995)
Engineering approach, simple, clear, widely applied

 = to ta l e r r o r
  = b ia s e r r o r
 = p r e c is io n e r r o r
 k+ 1

k k  k+ 1
X
 Bias error (b): fixed, systematic X t ru e X k X k+ 1

( a ) tw o r e a d in g s
 Bias limit (B): estimate of b

 Precision error (e): random  

FR EQ U E N C Y O F O C C U R R E N C E
 Precision limit (P): estimate of e
 Total error: d = b + e

X
X t ru e 
M A G N IT U D E O F X
(b ) in fin ite n u m b e r o f r e a d in g s
AIAA (1995)

Implementation Sequence
Determine the data reduction equation
Key feature: data-reduction equation r  r X1, X2 ,, X J 
r = r(X1, X2, X3,…, Xj)

Identify sources of uncertainty for each X i

E LE M EN TA L
ERRO R SO UR CES

IN D IV ID U A L
1 2 J M EASUREM ENT
Assess relative significance of uncertainty
SYSTEM S
sources (order of magnitude estimates)

X X X M EASUREM ENT
1 2 J O F IN D IV ID U A L
B , P B , P B , P V A R IA B L E S
1 1 2 2 J J

Considering the significant sources, estimate the

precision and bias limits for each X i
D A T A R E D U C T IO N
r = r ( X , X ,......, X ) E Q U A T IO N
1 2 J

r E X P E R IM E N T A L For the experimental result r, determine the

B , P R E S U LT
r r precision and bias limits and overall uncertainty
 AIAA (1995)
Implementation Aspects
 Measurement systems for each individual variable Xi : instrument, data acquisition and
reduction procedures, operational environment (laboratory, in situ), the flow and its
interaction with the instrument and the environment
 Estimates of errors are meaningful only when considered in the context of the process
leading to the value of the quantity under consideration
 Uncertainties estimated following the signal propagation from sensor to the
final result
 Uncertainties estimated with a pre-established confidence level (95% for most
engineering areas)
 UA differently conducted dependent on the type of experiment:
 Single test (for complex or expensive experiments): one set of
measurements (X1, X2, …, Xj) for r

 Multiple tests (ideal situations): many sets of measurements (X1, X2, …,

Xj) for r at a fixed test condition with the same measurement system
 AIAA (1995)
MULTIPLE TESTS (recommended)

Given a data reduction equation for a measurement r  r ( X 1 , X 2 ,, X J )

M
1
The result and its uncertainty is r Ur with r
M
r
k 1
k

The uncertainty in the final result U r =  B 2r + P 2r  1/ 2

where the bias limit of the result is

J J 1 J r
B    B  2
2 2 2
  Bik i 
r
i 1
i i
i 1 k i 1
i k
X i

and the precision limit of the result is

 M  rk  r  2 
1/ 2
tS r
Pr  ; S r    ; t  2 for M  10
M  k 1 M  1 
 AIAA (1995)
SINGLE TEST

Given a data reduction equation for a measurement r  r ( X 1 , X 2 , X 3 , X j , )

The result and its uncertainty is r Ur

The uncertainty in the final result U r =  B 2r + P 2r  1/ 2

where the bias limit of the result is

J J 1 J
r
B    B  2
2
r i
2
i
2
  i k Bik i 
i 1 i 1 k i 1 X i

and the precision limit of the result is

 r  r   2 1/ 2
M

Pr  tS r S r   k   Based on prior information

 k 1 M  1 
 AIAA (1995)

Implementation Aspects

 sound engineering judgment to optimize the output with minimum costs, e.g.:
 use of end-to-end uncertainty estimation approach
 uncertainty sources < 1/4 or 1/5 of the largest sources are usually considered
negligible

 specific procedures for single and multiple measurements

 specific procedures for dealing with small statistical samples

 methodology for assessment of calibration uncertainties

 methodology for data validation

 D E F IN E P U R P O S E O F T E S T A N D
R E S U L T S U N C E R T A IN T Y R E Q U IR E M E N T S

AIAA (1995) S E L E C T U N C E R T A IN T Y M E T H O D

D E S IG N T H E T E S T
- D E S IR E D P A R A M E T E R S (C D , C R ,....)
- M O D E L C O N F IG U R A T IO N S (S )
- T E S T T E C H N IQ U E (S )

Implementation -
-
-
M E A S U R E M E N T S R E Q U IR E D
S P E C IF IC IN S T R U M E N T A T IO N
C O R R E C T IO N S T O B E A P P L IE D
Aspects
D E T E R M IN E E R R O R S O U R C E S
A F F E C T IN G R E S U L T S

 Integration of UA in all YES

E S T IM A T E E F F E C T O F
phases of the T H E E R R O R S O N R E S U LTS

measurement
N O U N C E R T A IN T Y N O
IM P R O V E M E N T
A C C E P TA BLE ?
P O S S IB L E ?

N O YES

N O TEST IM P L E M E N T T E S T

S TA R T T E S T

M EASU R E-
R E S U LT S N O M EN T
A C C E P TA B LE ? SYSTEM
PRO BLEM ?

YES YES

N O C O N T IN U E T E S T S O LV E P R O B LE M

D O CU M EN T R E S U LT S
YES E S T IM A T E - R EFER ENC E C O N D IT IO N
PU RP O SE
A C TU A L D ATA - P R E C IS IO N L IM IT
A C H IE V E D ?
U N C E R T A IN T Y - B IA S L IM IT
- TO TA L U N C E R T A IN T Y
 AIAA (1995)

IMPLEMENTATION EXAMPLE
 Extensively used in laboratory measurements and field conditions,
from simple (Pitot tube) to complex (LDV) instruments

 Widely applied for teaching and research purposes

 Successful implementation to discharge measurements:

 conventional instruments (Muste et al. 2007)
 contemporary, nonintrusive techniques:
 Large-Scale Particle Image Velocimetry (Y-S. Kim et al, 2007)
 Acoustic-Doppler Current Profilers (Gonzalez-Castro & Muste, 2007)
 ADCP UA: Implementation
 Currently, ADCPs are the most efficient instrument for riverine
environment characterization (monitoring and research needs)

 If properly operated, the instrument can accurately document

discharges, mean velocities, and selected turbulence characteristics

 Despite their extensive use, there are aspects regarding their

capabilities, operation, and uncertainty analysis not documented yet
 ADCP UA: Implementation

ADCP Uncertainty Analysis (UA) status

Past efforts (non-standardized methodologies)

 Discharge: Simpson & Oltman (1992), Gordon (1993),
Lipscomb (1995), Morlock (1996), Simpson (2001), Gartner
(2002), Muller (2002), Yorke & Oberg (2002), USGS-RDI (2005)
 Turbulence measurements: Droz (1998), Stacey (1999),
Nystrom (2002), Schemper & Admiraal (2002)

On-going efforts (standardized methodology)

 UA formulated within the framework of authoritative
engineering standards
ADCP UA: Implementation
Discharge Measurement with ADCP mounted on a boat

Unmeasurable Near-bank Areas

Unmeasurable Top Area Qet

Qel Qer

Qm
Measurable Area

Qeb
Unmeasurable Bottom Area

Qt  Qm  Qem  Qet  Qeb  Qel  Qer

where Qel  KVl Ll Z l ; Qer  KVr Lr Z r

 ADCP UA: Elemental Uncertainty Assessment
Error identification
Accounted in Reduction
Source Biases Estimation of Depends upon Can be estimated from
Equations through 1
e1: Spatial resolution Water and boat velocities, depths † ADCP, mode, settings, boat speed End-to-end calibration 2
ADCP frequency, mode, settings, UA of signal processing algorithms,
e2: Doppler noise Water and boat velocities Bva, Bvb
speed of sound, gating time instrument intercomparison
e3 : Velocity ambiguity Water and boat velocities † Mode, settings End-to-end calibration
e4 : Side-lobe interference Discharge through unmeasured areas * Beam angle, settings, bathymetry End-to-end calibration
e5: Temporal resolution High frequency velocity components † Settings End-to-end calibration
UA of C(Salinity, Temperature) with data
e6: Sound speed Water and boat velocities, depths BC Water properties
from reference meter
e7 : Beam angle Water and boat velocities, depths B ADCP Manufacturer’s specifications
e8 : Boat speed Water and boat velocities, depths † Site, flow, boat operation End-to-end calibration
e9: Sampling time Frequency of large-scale flow Instrument intercomparison based on long
ADCPV9f Long-term means †
structures 3 data records under steady conditions
ADCP, draft, settings, velocity, flow Experimental Measurements and CFD
e10 : Near-transducer Velocities near the ADCP Bnt
depth Modeling
e11: Reference boat velocity Water and boat velocities, depths Bvb Sediment concentration, flow 4 Manufacturer’s Specifications
UA of depths as f(C and gating time) and
ADCP, settings, draft, bathymetry,
e12: Depth Discharge through unmeasured areas BDa, BDp, BDp, BDo, BDavg 5 BC, Bt and BDADCP and concurrent depth
water properties, time gating
range measurements
e13: Cell positioning Measured and unmeasured discharge Bt , BDa, BDp, BDo, BDavg ADCP, setting, water properties 
Water and boat velocities, depths
e14: Rotation Bp , Br , Bh ADCP, setup, site Manufacturer’s Specifications
and geographic orientation
e15: Timing Distances by gating and discharge Bt ADCP, speed of sound, gating time Manufacturer’s Specifications
ADCP settings, bathymetry, cross
e16: Edge Discharges through channel edges B ,BL Manufacturer’s Specifications
section, edge distances
Discharge through unmeasured top Velocity distribution model, turbulence Field and Laboratory Experiments with
e17: Vertical profile model BQ1 6
and bottom areas intensity reliable CFD-LES Modeling
Highly resolved data / End-to-end
e18: Discharge model Discharge through measured area BQ2 6 Discharge model
calibration
e19: Finite summation Discharge through measured area BQ3 6 ADCP settings, boat velocity 
e20: Site conditions & operation Total discharge † Site, boat operation Concurrently measured data
ADCP UA: Implementation
Data Reduction Equations (Teledyne/RDI’s ADCP)

V f  u f , v f 



Vb   u b , vb 
y

T zU ( t )
Qm     V
0 zL (t )
f  
 V b  k dz dt
 ADCP UA: Implementation

Exact approach – discharge in the direct measured area

Qi , j

i 1 i i 1 i  Ns

j
j 1
unit area for Qi , j

 Using BT

N s 1 n
z  z j t i 1  t i 
Qm  
i 1 j 1
j 1

2 sin 
vb 4  vb3 v1  v 2  vb1  vb 2 v4  v3 i 1, j
 ADCP UA: Implementation
Exact approach: in-bin discharge
 Using BT

Qmi , j   u a vb  ub va  i 1, j  z j 1  z j   ti 1  ti 

WATER VELOCITY WITH RESPECT TO ADCP

1
ua   2 cos R csc   v1  v2  + sin R  cos P sec   v1  v2  v3  v4   2 sin P csc   v4  v3 
4
1
va   2 cos P csc   v4  v3  - sin P sec   v1  v2  v3  v4 
4
BOAT VELOCITY WITH RESPECT TO CHANNEL BED
1
ub    2 cos R csc   v b1  vb 2  + sin R  cos P sec   v b1  vb 2  vb 3  vb 4   2 sin P csc   vb 4  vb 3 
4
1
vb    2 cos P csc   vb 4  vb 3  - sin P sec   v b1  vb 2  vb 3  vb 4 
4
ACTUAL WATER VELOCITY
V f  V a  V b  ua  ub , va  vb 
DEPTHS
Dtop  D AD CP  DB ;

D p
 D0  Da D p
 Da
DB  Db  f or Mode 1; DB  Db  f or Mode 5
2 2

D p  D0
DLG max  D avg cos   D ADCP  
2
f or Mode 1; DLG ma x  Dav g co s  D A DCP  
Dp
2
for Mode s 5, 8 & 11

Dtota l  Dav g c os  D ADCP ; Z1  Dto t a l  DLG 

Da ; Z2  Dt ot al  Dt op 
Da ; Z3  Dt o tal
2 2

PARTIAL DISCHARGES

Qm i , j
 z j 1  z j  t i 1  ti   u a vb  u b va  i 1, j  z j 1  z j  t i 1  ti   V a V b k i 1, j

Z  t
m m
Da Z b 1
1
 t i 1  ti    u a vb  u b va  j
Da b 1
3
 Z b 1
2 i 1  ti    u a vb  u b va  j
j 1 j 1
Qt  Qb 
i
Z b 1
2
 Z b 1
1
 i
Z 2
b 1
 Z b 1
1

Q el  CVl Ll Z l ; Q er  C Vr L r Z r

TOTAL DISCHARGE
N s 1 N s 1

 Q 
m
Qto tal  Q i 1 j 1
mi , j

i 1
ti  Qb i
 Qe m  Qe l  Qe r

C
If v  FD , the discharge is a functional relationship of the form:
2 FS


Qmi , j  f FD1 , FD 2 , FD 3 , FD 4 , FS , C , FD b1 , FD b1 , FD b1 , FD b1 ,  , r , p, h,  t a ,  z 
 ADCP UA: Implementation
Exact approach – top and bottom discharges (extrapolation)
DIS CHARGE (m 3/s )
Z3
D ADCP
TOP LAYER 3-POINT SLOP E
TOP Q (ES TIMATED) POWER
DB CONS TANT
Z2
D top ADCP
TRANSDUCER FACE ACTUAL
SCALAR P ROFILE

D TRIP LE (m 2/s 2)
PRODUCT
DEPTH D total
CELL Da
MID Q ADCP
D avg MEASURED ADCP
DISCHARGE VELOCITIES

Z P OWER FIT

D LG
Z1 P OWER
BOTTOM LAYER
BTM Q (ES TIMATED) P OWER IN LOW 0.2 D total

CONS TANT

  t
m

i 1  t i    u a vb  ub va  j
m
ti 1  t i  u a vb  ub va j
b 1 b 1
Da Z 3  Z2 Da Z b 1
1
j 1
Qbi  j 1

Z b 1
2
 Z1b 1
 Qt i
Z b 1
2
 Z b 1 1

 ADCP UA: Implementation
Uncertainty Propagation to Final Result: Bias Limit
BQ2  BQ2  BQ2  BQ2  BQ2  BQ2  BQ2
t m em et eb el er

   Q  2  Qmi , j  2  Qmi , j  2 4   Qmi , j

2 2 2
 2  Qmi , j  2   
2

  
 n   p  p  r  r     
 mi , j
 B 2
 B   B    Bv    Bv

r 1  va r
 a r  v  br   
  br 
    
   2 2 2 2  
N s 1 j 1
  Qmi , j  2  Qmi , j  2  Qmi , j  2  Qmi , j  2  
B2
      B  B 
 zi1  z  zi  t  ti1  t
B   Bt
 i  
Qm    z
i 1
  
j 1   j   i 1   i  j
 
 n 1  4  Qmi , j Qmi , j 
 
 
 0 .15
va rj var ( j 1)
B B
v a rj v a r ( j 1)
 
 j 1  r 1   j  i

  Q  2 2
4   Q 
2 2
 2 
  eti  B 2   Qeti  B 2    eti  B 2   Qeti  B 2    Qeti  B 2 
n

  p  p  r  r  
j 1 t 1  va t 
 vat  v  vbt      
      bt  j   
 2 2 2 2 2

N s 1
  Qeti  2  Qeti  2  Qeti  2  Qeti  2  Qeti  2 
BQ2et       BDa    BD  
  BDo    BD  
  BDb 
i 1   Da 
 D   D 
 Do   Db 
p avg
 p   avg  
 
   Qeti  B 2   Qeti  B 2    0.15 Qeti Qeti B B 
2 2
n 1 4
 
  t  ti 1  t  ti   
  va rj var ( j 1)
v a rj v a r ( j 1 )
 
  i1   i  j 1 r 1 
  j 
 i
2 2 2 2
 2  Qel  2 2
 Q   Q   2
B 2
  el  BV2   el  BL    BD   Qel  BD
 Q 
  el  B2
 D   D
  
Qet
 Vel   Le   ADCP 
el e avg ADCP

 avg 
2 2 2 2
 2  Qer  2 2
 Q   Q  Qer  2  Q 
B 2
  er  BV2   er  BL    BD    BD   er  B2
Qer
 Davg    
 Ver   Ler   D ADCP 
er er avg r ADCP

 r 
ADCP UA: Implementation

Uncertainty Propagation to Final Result: Precision Limit

tS Qt
PQt 
M

Uncertainty Propagation to Final Result: Total Uncertainty

U Qt  BQ2t  PQ2t
 ADCP UA: Implementation

Practical approach
(pitch and roll neglected in DRE; errors accounted through end-to-end calibrations)

 velocity (instrument coordinates neglecting the pitch and roll angle) 1

 (V  V ) 2  (V  V ) 2 (V  V ) 2
 (V  V ) 2 2
V f i 1 , j   3 4 1 2
 Vb  2 
2 3 4 1 2
 Vb  cos(   )
 4 sin 2
 2 sin  

q = beam angle,
where
a = angle of the flow to instrument
β = angle of the boat velocity
V1 ~ V4 = in beam water velocities
Vb = boat velocity

 total discharge

n
N s 1

Qt    V f  i 1, j  Vb , i 1 sin( i 1, j 
  i 1 )   z j 1  z j    t i 1  t i    Qel  Qer
i 1  j 1 
 ADCP UA Software - architecture
• Developing tools - Borland C++ Builder (v.6) & Microsoft Access
• Software Configuration
ADCP Uncertainty Analysis & GUI Flow Chart

New Measurements Prior Information ( u1 ~ u19 )

Processing Database
ADCP ADCP Configuration File - Calibrations
- Vertical, Horizontal Velocity Profile
Output Output - operational and - Discharge (WinRiver & WinADCP - Manufacturer specifications
Binary ASCII environmental - Literature compilation
Homologuous)
Files Files specifications - Visualizations - prior measurements
- User UA archive

Statistical Analysis for UA

Error Propagation to Final Results

(Embedded in Uncertainty Analysis GUI)
- Velocity Uncertainty Analysis
- Discharge Uncertain Analysis

Uncertainty Analysis Graphical User Interfaces

Numerical Display
Graphical Display
Uncertainty Analysis Output

5
4
3
2
1
 ADCP UA Software - GUIs
Archive database
- Elemental uncertainties are archived in categories based on river characteristics.
- Users with limited level of preparedness can estimate uncertainties
using default values obtained in similar environment and operating conditions.
- The stored information is updated as soon as new measurements are processed.
- User can also create new archives using new classification categories
ADCP UA Software - GUIs
Information for archiving

Discharge

Channel
Profile

Vessel Moving Path

Flow Direction
River
Characteristics
 ADCP UA Software - GUIs
Assessment of bias limit
Individual Error Source Input
Default Values based
on Archive Database

River Characteristics

Relevant Literature
for error sources
 ADCP UA Software - GUIs
Assessment of precision limit
Variation of Discharge
Measurements

Repeated Measurements
for a transect

Result
 ADCP UA Software & GUIs
Assessment of total uncertainty

Calculated Uncertainty in
Discharge Measurement
 Conclusions

 Feasibility of UA engineering standards for implementation to

ADCP measurements
 The methodology is comprehensive, simple to implement
 Easily upgradeable as new info occur
 UA allows tracing of the measurement accuracy to primary
standards  withstand legal and strict QA/QC requirements
 Finalization of UA – an extensive and expensive effort
Collaboration between manufacturers and users in a coordinated
effort = key to complete UA for the variety of measurement
situations and operating conditions encountered in monitoring
practice
 The framework was adopted by ASCE’s HME Task Committee and
the UNESCO group on Data Requirements for Integrated Urban
Water Management (Fletcher et al., 2007)
 Currently evaluated by the ISO committee (Herschy)
 Conclusions

 The UA customized software for ADCP velocity and

discharge measurements requires minimum user
preparation

 Autoarchiving uncertainties for specific environments and

operating conditions can provide information about
dominant sources of uncertainties at various sites.

 By continuously increasing the sample size through

archiving, the UA output is progressively enhanced.
 Outlook

 Work closely with manufacturers and users to assess

elemental error sources (manufacturer, operator, environment,
or combinations) and integrate them in the AIAA (1995)
uncertainty assessment framework for rigorous documenting
velocity and discharge measurement accuracy

 Conduct sensitivity analysis and field tests for compiling

uncertainty minimizations guidelines

 Develop operational guidelines for conducting accurate

measurements in various flow regimes
 Outlook

 Need for coordination and extensive collaboration among

ADCP manufacturer, operators, data users, and third-party
evaluators

 Need for evaluation of the status of current developments

and to strategize for integrative efforts to assess
methodologies for operation and accuracy assessment of
the ADCP as well as other flow measurement techniques
over an extend the range of flow conditions (present WMO
effort)

 IIHR is willing to be actively involved in the WMO initiative

Thank you!

Questions?