STATISTICAL ERRORS (TYPE I, TYPE II, POWER)

Data, data everywhere, but not a thought to think.

-Jesse Shera

The most recent Advanced Placement Statistics Outline of Topics includes the concepts of type
I and type II errors, and power. The purpose of this paper is to provide simple examples of
these topics.

Assume that two samples of people have the indicated ethnic distributions. The sample sizes are
25 and 20, respectively.

African-American Native-American Caucasian Oriental

3 1 15 6
SAMPLE #1

10 4 3 3
SAMPLE #2

Sometimes a visual display of data is helpful. Here are dot plots for each sample.

SAMPLE #1

African-
***
American

Native-
*
American

**************
Caucasian
*

Oriental ******

SAMPLE #2

African-
**********
American
Native-
****
American

Caucasian ***

Oriental ***

Here is our challenge: One of the two samples is randomly chosen, and then one individual is
randomly picked from the chosen sample. Based on our observation of the individual, we must
make a conjecture as to which sample the chosen individual belongs.

Here is a null hypothesis, Ho.

Ho: The individual came from SAMPLE #1

Based on our observation of the randomly chosen individual, we must decide to either accept or
reject Ho. A rejection of Ho is, of course, equivalent to asserting that the individual came from
SAMPLE #2.

There are two types of errors that can be made during this process.

Type I error: Ho is rejected when it is true.

Type II error: Ho is accepted when it is false.

Here are four possible tests relating to Ho. A test consists of randomly choosing an individual
and then accepting or rejecting Ho based on an observation of the individual. As defined, each
test can produce a correct conclusion or an incorrect conclusion.

Test #1: Accept Ho if the randomly chosen individual is Caucasian.

Test #2: Accept Ho is the randomly chosen individual is Caucasian or Oriental.

Test #3: Accept Ho if the randomly chosen individual is not Native-American.

Test #4: Accept Ho if the randomly chosen individual is not African-American.

Let's examine Test #1. A Type I error can only occur when Ho is true. Hence, the probability of
a Type I error with Test #1 is 10/25 = 40%. A Type II error can only occur when Ho is false. In
this situation, if Ho is false, then the selected individual came from Sample #2. The probability
of a Type II error is 3/20 = 15%. Here is a probability summary for Test #1.
 Sample is #1 Sample is #2

Accept Ho 60% (Correct decision) 15% (Type II error)

Reject Ho 40% (Type I error) 85% (Correct Decision)

The power of a test is the probability that an incorrect null hypothesis is rejected. In this
situation, the power is 85%, which is calculated by the formula

1 - probability(Type II error).

In this situation, if the randomly chosen individual is from SAMPLE #2, one can expect this test
to correctly conclude that the individual is not from SAMPLE #1 in 85 out of every 100 trials.

Here are corresponding probabilities for all of the indicated tests. (You can easily check these
out!)

Prob. (Type I Prob. (Type II Power of

error) error) the test

40% 15% 85%

Test #1

16% 30% 70%

Test #2

4% 80% 20%
Test #3

12% 50% 59%

Test #4

There are, of course, other tests that could be used. Of the four tests examined, Test #3 produces
the smallest Type I error, but yields a whopping 80% Type II error. Strategy #1 has the smallest
Type II error, but also the largest Type I error.

As suggested by the examples above, decreasing the chance of one type of error frequently
increases the chance for the other error type. In real-life situations, one can decrease the
probability of both error types by collecting more data or having more information available.
However, one must frequently decide which error type should be minimized. Here are two
simple examples:

Example #1:
In the legal world, a null hypothesis might be "This person is innocent." A Type I error would
be judging the person guilty when he is innocent. A Type II error would involve declaring the
person innocent when he is guilty. If one accepts the thought that it is better to release a guilty
person than to convict an innocent one, then it would be important to minimize the chances of a
Type I error.

Example #2:
In the world of medicine, a null hypothesis might be "This drug will cure an illness." A Type I
error would be concluding that the drug does not work when it actually does. A Type II error
would conclude that the drug does work when it actually doesn't. One could argue that a Type II
error should be minimized here if one agrees that spending time and money on a useless drug
would replace what might be some other effective treatment.

Here are three important facts that can help minimize the confusion that sometimes results when
working with these error types:

1. A Type I error can only occur if a null hypothesis,Ho, is true.

2. A Type II error can only occur if a null hypothesis,Ho, is false.

3. The power of a test is 1 - probability(Type II error).

It is usually a practical impossibility to work with an entire population. A statistician takes

samples and "generalizes" his/her results to reach a conclusion about a population. This does
involve producing a statement (null hypothesis) which may or may not be true. As noted above,
there are two types of statistical errors that can be made. The main purpose of this paper is
simply to introduce the two error types and, provide some simple examples illustrating them. It
is hoped that these examples will be helpful to teachers and students as they prepare for the
Advanced Placement Statistics Examination.