Statistics - Lying without sinning?

• "Lies, damned lies, and statistics"

1954
 Statistics - Lying without sinning?
In North Dakota, 54 Million Beer Bottles by the side of the Road
April 01 2002

South Dakota's Pierre Capital Journal reports (Mar. 1) that "an average of 650
beer cans and bottles are tossed per mile of road annually." The statistic is
attributed to Dennis W. Brezina, an activist against drunk-driving.

But how did he come up with his data? According to the Journal, Brezina traveled
"highways across the nation to determine whether the problem he perceived was For more
widespread. He made two trips to South Dakota, one in 1998 and another in Check out
2000." He counted "cans and bottles in ditches in May of both years" and claimed
to have found an average of "one beer can or bottle every 16 feet when walking www.STATS.org
randomly selected stretches of ditch."

But the math appears a little blurry. The web site of the South Dakota Department
of Transportation claims that the state "has 83,472 miles of highways, roads and
streets." Assuming Brezina's estimate is correct, South Dakotans appear to be
world-class litterbugs, tossing aside approximately 54,256,800 bottles or cans
every year. According to the Census Bureau there are 754,844 people in South
Dakota. So, according to Brezina, the average resident throws at least 71 beer
bottles or cans on the side of the road every year.
 Statistics for Quantitative Analysis

• Statistics: Set of mathematical tools used to describe

and make judgments about data
• Type of statistics we will talk about in this class has
important assumption associated with it:

Experimental variation in the population from which samples

are drawn has a normal (Gaussian, bell-shaped) distribution.
Normal distribution
• Infinite members of group:
population
• Characterize population by taking
samples
• The larger the number of samples,
the closer the distribution becomes to
normal
• Equation of normal distribution:
1  ( x   ) 2 / 2 2
y e
 2
 Normal distribution

• Estimate of mean value

of population = 
• Estimate of mean value
of samples = x

x i
Mean = x  i

n
 Normal distribution
• Degree of scatter (measure of central tendency)
of population is quantified by calculating the
standard deviation

• Std. dev. of population = 

• Std. dev. of sample = s

 ( xi  x ) 2
s i

n 1
• Characterize sample by calculating xs
 Standard deviation and the
normal distribution
• Standard deviation defines
the shape of the normal
distribution (particularly
width)

• Larger std. dev. more

scatter about the mean,
worse precision.

• Smaller std. dev. means

less scatter about the
mean, better precision.
 Standard deviation and the
normal distribution
• There is a well-defined relationship between the std. dev. of a population
and the normal distribution of the population.

• (May also consider these percentages of area under the curve)

Amount of Data

Standard deviations
68 %
95 %
99.7 %
Total % of the data covered by distribution
 Example of mean and standard
deviation calculation

Consider Cu data: 5.23, 5.79, 6.21, 5.88, 6.02 nM

x = 5.826 nM  5.82 nM

s = 0.368 nM  0.36 nM

Answer: 5.82 ± 0.36 nM or 5.8 ± 0.4 nM

Learn how to use the statistical functions on your
calculator. Do this example by longhand calculation
once, and also by calculator to verify that you’ll get
exactly the same answer. Then use your calculator for
all future calculations.
 Learn to use your calculator’s statistical functions to
calculate
mean and standard deviation. You’ll save yourself a lot of
work.

http://www.willamette.edu/~mjaneba/help/TI-85-stats.htm

http://www2.ohlone.edu/people2/joconnell/ti/
 Relative standard deviation (rsd)
or coefficient of variation (CV)

s
rsd or CV =  100
x

From previous example,

rsd = (0.36 nM/5.82 nM) 100 = 6.1% or 6%

 Standard error
• Tells us that standard deviation of set of samples should decrease if we take more
measurements

• Standard error =
s
sx 
•
n
Take twice as many measurements, s decreases by

• Take 4x as many measurements, s decreases by

2  1.4
• There are several quantitative ways to determine the sample size required to achieve a
4 2
desired precision for various statistical applications. Can consult statistics textbooks for further
information; e.g. J.H. Zar, Biostatistical Analysis
 Variance

Used in many other statistical calculations and tests

Variance = s2

From previous example, s = 0.36

s2 = (0.36)2 = 0. 129 (not rounded because it is usually
used in further calculations)
 Average deviation
• Another way to express
degree of scatter or
uncertainty in data. Not as ( x  x )
i
statistically meaningful as d i

standard deviation, but n

useful for small samples.

Using previous data:

5.23  5.82  5.79  5.82  6.21 5.82  5.88  5.82  6.02  5.82
d
5
d  0.25  0.25 or 0.2 nM

Answer : 5.82  0.25 nM or 5.8  0.2 nM

 Relative average deviation (RAD)
d 
RAD   100 (as percentage )
x
d 
RAD   1000 (as parts per thousand , ppt )
x
Using previous data,

RAD = (0. 25/5.82) 100 = 4.2 or 4%

RAD = (0. 25/5.82) 1000 = 42 ppt

 4.2 x 101 or 4 x 101 ppt (0/00)
 Some useful statistical tests

• To characterize or make judgments about data

• Tests that use the Student’s t distribution
– Confidence intervals
– Comparing a measured result with a “known” value
– Comparing replicate measurements (comparison of
means of two sets of data)
From D.C. Harris (2003) Quantitative Chemical Analysis, 6th Ed.
 Confidence intervals

• Quantifies how far the true mean () lies from the
measured mean, x. Uses the mean and standard
deviation of the sample.

ts
x
n
where t is from the t-table and n = number of
measurements.
Degrees of freedom (df) = n - 1 for the CI.
 Example of calculating a
confidence interval
Consider measurement of dissolved Ti
in a standard seawater (NASS-3):
Data: 1.34, 1.15, 1.28, 1.18, 1.33,
1.65, 1.48 nM
DF = n – 1 = 7 – 1 = 6
x = 1.34 nM or 1.3 nM ts
s = 0.17 or 0.2 nM x
95% confidence interval n
t(df=6,95%) = 2.447
CI95 = 1.3 ± 0.16 or 1.3 ± 0.2 nM
50% confidence interval
t(df=6,50%) = 0.718
CI50 = 1.3 ± 0.05 nM
 Interpreting the confidence interval
• For a 95% CI, there is a 95% probability that the true
mean () lies between the range 1.3 ± 0.2 nM, or
between 1.1 and 1.5 nM

• For a 50% CI, there is a 50% probability that the true

mean lies between the range 1.3 ± 0.05 nM, or between
1.25 and 1.35 nM

• Note that CI will decrease as n is increased

• Useful for characterizing data that are regularly obtained;

e.g., quality assurance, quality control
Nitrate Concentrations (g/mL)
Trial 1 Trial 2 Trial 3 Trial 4 Trial 5 Trial 6 Trial 7 Trial 8 Trial 9 Trial 10
0.51 0.51 0.51 0.5 0.51 0.49 0.52 0.53 0.5 0.47
0.51 0.52 0.53 0.48 0.49 0.5 0.52 0.49 0.49 0.5
0.49 0.48 0.46 0.49 0.49 0.48 0.49 0.49 0.51 0.47
0.51 0.51 0.51 0.48 0.5 0.47 0.5 0.51 0.49 0.48
0.51 0.5 0.5 0.53 0.52 0.52 0.5 0.5 0.51 0.51
0.506 0.504 0.502 0.496 0.502 0.492 0.506 0.504 0.5 0.486 mean

average 0.4998
mg/mL frequency
stdev 0.01647
0.53 3
0.52 5
0.51 13
0.5 10
0.49 10 Let’s Graph the Data!
0.48 5
0.47 3
0.46 1
 nitrate concentration

14
outlier
12
10
frequency

8
6 ± 1

4
± 2
2
0
0.44 0.46 0.48 0.5 0.52 0.54
 g/mL
 Confidence Interval Exercise
s
x    t  sm    t 
n
Calculate the 95, 98 and 99 % confidence intervals

For the nitrate concentration data

95 % 0.500 ± 0.005
98 % 0.500 ± 0.006
99 % 0.500 ± 0.006
50 % 0.500 ± 0.002
 0.500 ± 0.006 0.500± 0.006

0.500 ± 0.005 0.500 ± 0.002

 Testing a Hypothesis (Significance Tests)

Carry out measurements on an accurately known standard.

Experimental value is different from the true value.

Is the difference due to a systematic error (bias) in the method - or simply to random error?

Assume that there is no bias

(NULL HYPOTHESIS),
and calculate the probability
that the experimental error
is due to random errors.

Figure shows (A) the curve for

the true value (A = t) and
(B) the experimental curve (B)
 Comparing a measured result
with a “known” value

• “Known” value would typically be a certified value

from a standard reference material (SRM)
• Another application of the t statistic

known value  x
t calc  n
s
Will compare tcalc to tabulated value of t at appropriate
df and CL.

df = n -1 for this test

 Comparing a measured result
with a “known” value--example
Dissolved Fe analysis verified using NASS-3 seawater SRM
Certified value = 5.85 nM
Experimental results: 5.76 ± 0.17 nM (n = 10)

known value  x 5.85  5.7 6

tcalc  n  10  1.674
s 0.17
(Keep 3 decimal places for comparison to table.)
Compare to ttable; df = 10 - 1 = 9, 95% CL
ttable(df=9,95% CL) = 2.262

If |tcalc| < ttable, results are not significantly different at the 95% CL.
If |tcalc|  ttable, results are significantly different at the 95% CL.
For this example, tcalc < ttest, so experimental results are not significantly
different at the 95% CL. THE NULL HYPOTHESIS IS MAINTAINED and no BIAS
at the 95 % confidence level.
Comparing replicate measurements or
comparing means of two sets of data
• Another application of the t statistic
• Example: Given the same sample analyzed by two
different methods, do the two methods give the “same”
result?
x1  x 2 n1 n2
t calc 
s pooled n1  n2

s12 (n1 1)  s 22 (n2 1)

s pooled 
n1  n2  2
Will compare tcalc to tabulated value of t at appropriate df
and CL.
df = n1 + n2 – 2 for this test
 Comparing replicate measurements or
comparing means of two sets of data—
example
Ewww!

Determination of nickel in sewage sludge

using two different methods
Method 1: Atomic absorption Method 2: Spectrophotometry
spectroscopy
Data: 3.91, 4.02, 3.86, 3.99 mg/g Data: 3.52, 3.77, 3.49, 3.59 mg/g

x1 = 3.945 mg/g x2 = 3.59 mg/g

s1 = 0.073 mg/g = 0.12 mg/g

s2
n1 =4 n2 =4
 Comparing replicate measurements or
comparing means of two sets of data—example

s12 (n1 1)  s22 (n2 1) (0.07 3 ) 2 (4 1)  (0.12 ) 2 (4 1)
s pooled    0.0993
n1  n2  2 442

x1  x2 n1 n2 3.945  3.59 (4)(4)

tcalc    5.056
s pooled n1  n2 0.0993 44

Note: Keep 3 decimal places to compare to ttable.

Compare to ttable at df = 4 + 4 – 2 = 6 and 95% CL.

ttable(df=6,95% CL) = 2.447

If |tcalc|  ttable, results are not significantly different at the 95%. CL.
If |tcalc|  ttable, results are significantly different at the 95% CL.

Since |tcalc| (5.056)  ttable (2.447), results from the two methods are
significantly different at the 95% CL.
 Comparing replicate measurements or
comparing means of two sets of data

Wait a minute! There is an important assumption

associated with this t-test:

It is assumed that the standard deviations (i.e., the

precision) of the two sets of data being compared are not
significantly different.

• How do you test to see if the two std. devs. are

different?

• How do you compare two sets of data whose std. devs.

are significantly different?
 t-tests and the Law
 
Clearly, the meanings of 1.083 ± 0.007 and 1.0 ± 0.4 are very different. As a
person who will either derive or use analytical results, you should be aware of
this warning published in a report entitled “Principles of Environmental Analysis”:
 
 
Analytical chemists must always emphasize to the public that the single most
important characteristic of any result obtained from one or more analytical
measurements is an adequate statement of its uncertainty interval. Lawyers
usually attempt to dispense with uncertainty and try to obtain unequivocal
statements: therefore, an uncertainty interval must be defined in cases
involving litigation and or enforcement proceedings. Otherwise, a value of
1.001 without a specified uncertainty, for example may be views as legally
exceeding a permissible level of 1.
 
L. K. Keith, W. Crummett, J. Deegan Jr., R. A. Libby, J. K. Taylor, and G. Wentler,
Analytical Chemistry, 55, 2210 (1983).
F-test to compare standard deviations

• Used to determine if std. devs. are significantly

different before application of t-test to compare
replicate measurements or compare means of two
sets of data

• Also used as a simple general test to compare the

precision (as measured by the std. devs.) of two sets
of data

• Uses F distribution
F-test to compare standard deviations

Will compute Fcalc and compare to Ftable.

s12
Fcalc  where s1  s2
s22

DF = n1 - 1 and n2 - 1 for this test.

Choose confidence level (95% is a typical CL).

From D.C. Harris (2003) Quantitative Chemical Analysis, 6th Ed .
F-test to compare standard deviations
From previous example:
Let s1 = 0.12 and s2 = 0.073

s12 (0.12 ) 2
Fcalc    2.70
s22 (0.07 3 ) 2
Note: Keep 2 or 3 decimal places to compare with F table.

Compare Fcalc to Ftable at df = (n1 -1, n2 -1) = 3,3 and 95% CL.
If Fcalc  Ftable, std. devs. are not significantly different at 95% CL.
If Fcalc  Ftable, std. devs. are significantly different at 95% CL.
Ftable(df=3,3;95% CL) = 9.28
Since Fcalc (2.70) < Ftable (9.28), std. devs. of the two sets of data are
not significantly different at the 95% CL. (Precisions are similar.)
Comparing replicate measurements or
comparing means of two sets of data-
revisited

The use of the t-test for comparing means was

justified for the previous example because we
showed that standard deviations of the two sets of
data were not significantly different.

If the F-test shows that std. devs. of two sets of data

are significantly different and you need to compare
the means, use a different version of the t-test 
 Comparing replicate measurements or
comparing means from two sets of data when
std. devs. are significantly different

x1  x2
tcalc 
s12 / n1  s22 / n2

 
 
 ( s1 / n1  s2 / n2 )
2 2 2

DF   2   2

 1 1( s / n ) 2
( s 2
/ n ) 2

 2 2

  n1  1 n2  1  
Flowchart for comparing means of two
sets of data or replicate measurements
Use F-test to see if std.
devs. of the 2 sets of
data are significantly
different or not

Std. devs. are Std. devs. are not

significantly different significantly different

Use the 2nd version Use the 1st version of the

of the t-test () t-test (see previous, fully
worked-out example)
 One last comment on the F-test

Note that the F-test can be used to simply test whether

or not two sets of data have statistically similar
precisions or not.

Can use to answer a question such as: Do method one

and method two provide similar precisions for the
analysis of the same analyte?
 Statistics in the News
Outliers Disrupt the Mean
January 01 1999

In 1984, according to Larry Gonick and Woollcott Smith, the University of

Virginia announced that the mean starting salary of its graduates from the
Department of Rhetoric and Communications was a very hefty $55,000 per
year. But before you abandon your computer science training for speech
classes, you should know that the graduating class contained a significant
"outlier," or extreme data point not typical of the rest of the data set - Ralph
Sampson, future NBA All-Star, who majored in speech. It would have been
better to learn the median salary, the data point in the middle of the set.
 Evaluating questionable data points
using the Q-test
• Need a way to test questionable data points (outliers) in an
unbiased way.
• Q-test is a common method to do this.
• Requires 4 or more data points to apply.

Calculate Qcalc and compare to Qtable

Qcalc = gap/range

Gap = (difference between questionable data pt. and its

nearest neighbor)

Range = (largest data point – smallest data point)

 Evaluating questionable data points
using the Q-test--example
Consider set of data; Cu values in sewage sample:
9.52, 10.7, 13.1, 9.71, 10.3, 9.99 mg/L

Arrange data in increasing or decreasing order:

9.52, 9.71, 9.99, 10.3, 10.7, 13.1

The questionable data point (outlier) is 13.1

gap (13.1  10.7)
Calculate Qcalc    0.670
range (13.1  9.52)
Compare Qcalc to Qtable for n observations and desired CL (90% or
95% is typical). It is desirable to keep 2-3 decimal places in
Qcalc so judgment from table can be made.

Qtable (n=6,90% CL) = 0.56

From G.D. Christian (1994) Analytical Chemistry, 5th Ed.
 Evaluating questionable data points
using the Q-test--example
If Qcalc < Qtable, do not reject questionable data point at stated CL.

If Qcalc  Qtable, reject questionable data point at stated CL.

From previous example,

Qcalc (0.670) > Qtable (0.56), so reject data point at 90% CL.

Subsequent calculations (e.g., mean and standard deviation)

should then exclude the rejected point.

Mean and std. dev. of remaining data: 10.04  0.47 mg/L

 Q or G outlier test?
G (95 % confidence) Number of Observations
1.463 4
1.672 5

questionable _ value  x
1.822 6
1.938 7

G calc  2.032 8

s
2.11 9
2.176 10
2.234 11
2.285 12
2.409 15
2.557 20
reject if Gcalc > G table

Q (90 % confidence) Number of Observations

gap 0.76 4
Q calc  0.64 5

range 0.56
0.51
6
7
0.47 8
0.44 9
0.41 10

reject if Qcalc > Q table

 No. of observations 90% 95% 99% confidencelevel

3 0.941 0.970 0.994

4 0.765 0.829 0.926
5 0.642 0.710 0.821
6 0.560 0.625 0.740
7 0.507 0.568 0.680
8 0.468 0.526 0.634
9 0.437 0.493 0.598
10 0.412 0.466 0.568
Rejection of outlier recommended if Qcalc> Qtable for the desired confidence level.

Note:1. The higher the confidence level, the less likely is

rejection to be recommended.
2. Rejection of outliers can have a marked effect on mean
and standard deviation, esp. when there are only a few
data points. Always try to obtain more data.
 The following values were obtained for
Q Test for Rejection the concentration of nitrite ions in a sample
of Outliers of river water: 0.403, 0.410, 0.401, 0.380 mg/l.
Should the last reading be rejected?
Qcalc  0.380  0.401 (0.410  0.380)  0.7
But Qtable = 0.829 (at 95% level) for 4 values
Therefore, Qcalc < Qtable, and we cannot reject the suspect value.
Suppose 3 further measurements taken, giving total values of:
0.403, 0.410, 0.401, 0.380, 0.400, 0.413, 0.411 mg/l. Should
0.380 still be retained?

Qcalc  0.380  0.400 (0.413  0.380)  0.606

But Qtable = 0.568 (at 95% level) for 7 values
Therefore, Qcalc > Qtable, and rejection of 0.380 is recommended.

But note that 5 times in 100 it will be wrong to reject this suspect value!
Also note that if 0.380 is retained, s = 0.011 mg/l, but if it is rejected,
s = 0.0056 mg/l, i.e. precision appears to be twice as good, just by
rejecting one value.