Scribd
Upload
391 views

Chapter 21 More About Tests: Zero in On The Null

This document discusses key concepts related to statistical hypothesis testing including: - The null hypothesis is a statement about a parameter value used to compute the probability of observing sample statistics; it comes from the problem context, not the data. - A p-value tells the probability of observing results at least as extreme as the sample statistic, given that the null hypothesis is true. It is not the probability that the null hypothesis is true. - An alpha level defines a threshold for statistical significance; common levels are 0.1, 0.05, and 0.01. Type I and Type II errors depend on the alpha level and effect size. - Power is the probability of correctly rejecting a false null hypothesis; it

Uploaded by

Ikequan Scott
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
391 views

Chapter 21 More About Tests: Zero in On The Null

This document discusses key concepts related to statistical hypothesis testing including: - The null hypothesis is a statement about a parameter value used to compute the probability of observing sample statistics; it comes from the problem context, not the data. - A p-value tells the probability of observing results at least as extreme as the sample statistic, given that the null hypothesis is true. It is not the probability that the null hypothesis is true. - An alpha level defines a threshold for statistical significance; common levels are 0.1, 0.05, and 0.01. Type I and Type II errors depend on the alpha level and effect size. - Power is the probability of correctly rejecting a false null hypothesis; it

Uploaded by

Ikequan Scott
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 13

Chapter 21 More About Tests

Zero In on the Null


the null hypothesis must be a statement about the
value of a parameter for a model
we use this value to compute the probability that the
observed sample statistic (or something farther from
the null) would occur
the null hypothesis comes from the context of the
problem or situation NOT the data

How to Think About P-values


A P-value is a conditional probability. It tells us the
probability of getting results at least as unusual as the
observed statistic given that the null hypothesis is
true.
P( data | H0 )
The P-value is NOT the probability that the null
hypothesis is true!
The lower the P-value, the more comfortable we feel
about our decision to reject the null hypothesis, BUT
the null hypothesis does not get anymore false.

Alpha Levels
We can define rare event arbitrarily by setting
a threshold for our P-value.
If our P-value falls below that point, we will
reject the null hypothesis.
We call such results statistically significant.
Symbol:
Common Levels: 0.1, 0.05, 0.01

Traditional Critical Values from the Normal Model

0.1
0.05
0.01
0.001

1-sided
1.282
1.645
2.33
3.09

2-sided
1.645
1.96
2.576
3.29

We find these values on the t-table. Use tail


probability and the df = line.

If one-sided, is all on one side.

If two-sided, is split equally into the tails.

Making Errors
Type I Error the null hypothesis is TRUE, but we
mistakenly reject it
Type II Error the null hypothesis is FALSE, but we
fail to reject it
** When you choose an alpha level, you are setting the
the probability of a Type I Error.

Power
we can never prove a null hypothesis is true; we only
fail to reject it
when we fail to reject a null hypothesis, it is natural
to wonder whether we looked hard enough
(Was our test too weak to tell?)
Power is a tests ability to detect a false null
hypothesis or the probability that a test correctly rejects
a false null hypothesis
When the power is high, we can be confident that weve
looked hard enough at the situation.

The value of the power depends on how far the truth lies
from the null hypothesis value.
The distance between the null hypothesis value, p0,
and the truth, p, is called the effect size.
Power depends directly on effect size.
Power = 1
where is the probability of a Type II Error

If we reduce the probability of a Type I Error ( ),


we automatically increase the probability of a Type II
Error ( ).
We can reduce the probability of BOTH Type I and
Type II Errors by reducing the standard deviation
(spread). The power is also increased.
Take a larger sample!

What Can Go Wrong?


Dont interpret the P-value as the probability that H0
is true.
The P-value is about the data, not the hypothesis.
Its the probability of the data given that H0 is true,
not the other way around.
Dont believe too strongly in arbitrary alpha levels.
Its better to report your P-value and a confidence
interval so that the reader can make her/his own
decision.

Dont confuse practical and statistical significance.


Just because a test is statistically significant
doesnt mean that it is significant in practice.
And, sample size can impact your decision about a
null hypothesis, making you miss an important
difference or find an insignificant difference.
Dont forget that in spite of all your care, you might
make a wrong decision.

You might also like