Type I and Type II Errors

Type I error
Reject H0 when H0 is true

Type II error
Fail to reject H0 when H0 is false

Double F = Type II
Type I & II Errors
• If we reject when is true, we have committed a Type I
error.
• If we fail to
reject when
is true, we
have committed
a Type II error.
Type I & II
Errors
American Justice System Example:
• Ho: innocent
• Ha: guilty

• Type I error: punish an innocent person

• Type II error: let a not innocent (guilty)
person go free
Type I & II
Errors
Type I and II Errors

Quality Control Example:

• Ho: the product is acceptable to the customer
• Ha: the product is unacceptable to the
customer
• Type I error: reject acceptable product and
don't ship it.
• Type II error: ship unacceptable product to the
customer
Example: Perfect Potatoes
A potato chip producer and its main supplier agree that each shipment
of potatoes must meet certain quality standards. If the producer
determines that more than 8% of the potatoes in the shipment have
“blemishes,” the truck will be sent away to get another load of
potatoes from the supplier. Otherwise, the entire truckload will be
used to make potato chips. To make the decision, a supervisor will
inspect a random sample of potatoes from the shipment. The
producer will then perform a significance test using the hypotheses
H0 : p = 0.08
Ha : p > 0.08
where p is the actual proportion of potatoes with blemishes in a given
truckload.
 H0 : p = 0.08
Ha : p > 0.08
where p is the actual proportion of potatoes with
blemishes in a given truckload.
Describe a Type I and a Type II error in this
setting, and explain the consequences of each.
A Type I error would occur if the
producer concludes that the proportion
of potatoes with blemishes is greater
than 0.08 when the actual proportion is
0.08 (or less).
Consequence: The potato-chip
producer sends the truckload of
acceptable potatoes away, which may
result in lost revenue for the supplier.
A Type II error would occur if the
producer does not send the truck away
when more than 8% of the potatoes in
the shipment have blemishes.
Consequence: The producer uses the
truckload of potatoes to make potato
chips. More chips will be made with
blemished potatoes, which may upset
consumers.
Probability of a Type I Error
•When we want to make a decision about a
claim, we choose a significance level, , in
advance.
• If P-value reject the null hypothesis.
• If P-value fail to reject
the null hypothesis.
Probability of a Type I Error
• The probability of a Type I error is the probability of
rejecting the null hypothesis when it is true. This
probability is exactly the significance level of the test.

So,
Probability of a Type II Error
To calculate the probability of a type II error, we must
know the actual parameter, or p, in order to compare
to the null hypothesis value, which is really false in a
type II error. Since we do not know the true parameter
value, we use a theoretical value to discuss probability
of a type II error.
Probability of a Type II Error

𝑃(𝑡𝑦𝑝𝑒 𝐼 𝑒𝑟𝑟𝑜𝑟) =
Power of a Test
• The probability that a test correctly rejects a
false .

• In a court of law, this is like the probability that

the criminal justice system convicts people who
are guilty.
Power of a Test
• Probabilities of type I and type II errors and
Power:

• (NOTE: we must know the true value of or p)

• Power =
• When power is high, we can be confident in the
strength of the test.
How to decrease the probability of errors
• To reduce the likelihood of a type I error:
1)Decrease (Tradeoff: Then the
probability of type II error increases.
Power also decreases.)
2)Increase distance between and . (But
we don’t know the true parameter.)
How to decrease the probability of errors
• To reduce the likelihood of a type I error:
3) Increase sample size. (i.e. increase the
amount of data/ evidence.)
Standard deviation (and standard error)
decreases. With fixed BOTH type I and type II
errors decrease and power increases!
• GO WITH OPTION #3
 Power
• The probability of NOT making a Type II error.
• The higher the power, the less likely the
mistake is.
Factors that Increase Power

• Sample Size
• The larger the sample size, the higher the
power.
• Alpha Significance Level
• Change alpha (from 0.01 to 0.05) increases the
power, because a less conservative alpha
increases the chance of (correctly) rejecting
the null.
Factors that Increase Power

• Value of the Alternative Parameter

• The greater the difference between the
Hypothesized and True Mean the more
obvious the result and therefore the greater
the power.

More:
https://onlinecourses.science.psu.edu/stat414/
book/export/html/245
Increase Power, Decrease Type II
 Blue = Power
Increase Power, Decrease Type II Red = Type II error
What’s Worse? Type I or II
• It depends.
• It’s impossible to
minimize both error
types completely.
Error Probabilities
We can assess the performance of a significance test by
looking at the probabilities of the two types of error.
That’s because statistical inference is based on asking,
“What would happen if I did this many times?”
Reminder: H0 : p = 0.08
Ha : p > 0.08
where p is the actual proportion of potatoes with
blemishes. Suppose that the potato-chip producer
decides to carry out this test based on a random sample
of 500 potatoes using a 5% significance level (α = 0.05).
Error Probabilities
Assuming is true, the sampling distribution of will have:
Shape: Approximately Normal because 500(0.08) = 40
and 500(0.92) = 460 are both at least 10.
Center:
Spread:
 The shaded area in the right tail is 5%. Sample proportion
Error Probabilities values to the right of the green line at 0.0999 will cause us
to reject H0 even though H0 is true. This will happen in 5% of
all possible samples. That is, P(making a Type I error) = 0.05.
Type 2 Errors Investigation WS
www.rossmanchance.com/applets

Select: Improved Batting Averages (Power)

Or direct link:
http://statweb.calpoly.edu/chance/applets/power/power.
html
Error Probabilities
The potato-chip producer wonders whether the significance test of H 0: p = 0.08 versus H a:
p > 0.08 based on a random sample of 500 potatoes has enough power to detect a
shipment with, say, 11% blemished potatoes. In this case, a particular Type II error is to fail
to reject H 0: p = 0.08 when p = 0.11.
What if p = 0.11?
Error Probabilities
Error Probabilities
Earlier, we decided to reject H0 at
α = 0.05 if our sample yielded a
sample proportion to the right of
the green line.
Since we reject H0 at α= 0.05 if
our sample yields a proportion
> 0.0999, we’d correctly reject
the shipment about 75% of
the time.
The power of a test against any alternative is 1 minus the
probability of a Type II error for that alternative; that
is, power = 1 - β.
Extra Example:
Slow response times by paramedics, firefighters, and
policemen can have serious consequences for accident
victims. In the case of life-threatening injuries, victims
generally need medical attention within 8 minutes of the
accident. Several cities have begun to monitor emergency
response times. In one such city, the mean response times
to all accidents involving life-threatening injuries last year
was minutes. The city manager shares this information and
encourages these first responders to “do better.” At the end
of the year, the city manager selects an SRS of 400 calls
involving life-threatening injuries and examines the
response times.
a) State hypotheses for a significance test to determine
whether the average response times has decreased.
b) Describe a Type I error in this setting, and explain the
consequence.
c) Describe a Type II error in this setting, and explain the
consequence.
d) Which error is more serious in this setting? Justify
your answer.
Section 9.3
Tests About a Population Mean
Lack of significance does not imply that H0 is true. Even a large difference
can fail to be significant when a test is based on a small sample.
Significance tests are not always valid. Faulty data collection, outliers in the
data, and other practical problems can invalidate a test. Many tests run at
once will probably produce some significant results by chance alone, even
if all the null hypotheses are true.
A Type I error occurs if we reject H0 when it is in fact true. A Type II error
occurs if we fail to reject H0 when it is actually false. In a fixed level α
significance test, the probability of a Type I error is the significance level α.
The power of a significance test against a specific alternative is the
probability that the test will reject H0 when the alternative is true. Power
measures the ability of the test to detect an alternative value of the
parameter. For a specific alternative, P(Type II error) = 1 - power.