Error And Power
- 1. Week 8: Type I Error and Type II Error Statistical Power 1
- 2. Type I and Type II error: When we conduct hypothesis testing, we either reject or fail to reject the null hypothesis. Our decision usually causes four outcomes: 1. Reject the null hypothesis when it is false. 2. Keep the null hypothesis when it is true. 3. Reject the null hypothesis when it is true. (Type I error: alpha α) 4. Fail to reject the null hypothesis when it is false. (Type II error: beta β ) 2
- 3. Type I and Type II error: State of Nature Decision Null true Null false Made Reject Null Type I error Correct decision (α) (1 – b) POWER! Fail to reject Correct Type II error (b) null decision 3
- 4. Another example: TRUTH Innocent Guilty Reject Incorrect decision Correct decision (1 – null: Find Type 1 error (α) b) or Power Guilty! Fail to Correct decision Incorrect decision reject Type II error (b) null: Find Innocent 4
- 5. Scenario 1: Null: Reality: Conclusion: Decision: There is no Null is true: Fail to reject the Correct difference There is no null: Conclude decision. or difference or there is no relationship. relationship. difference or relationship. We failed to reject the null hypothesis when it is in fact true. In reality, there is no significant difference or relationship to find in this case. 5
- 6. Scenario 2: Null: Reality: Conclusion: Decision: There is no Null is false: Reject the null: Correct difference There is a Conclude there is decision. or difference or a difference or POWER! (1 – relationship. relationship. relationship. b) We reject the null hypothesis when it is in fact false. In reality, there is a significant difference or relationship to find in this case. 6
- 7. Scenario 3: Null: Reality: Conclusion: Decision: There is no Null is false: Fail to reject the Incorrect difference There is a null: Conclude Decision: Type or difference or that there is no II error (b) relationship. relationship. difference or relationship. We failed to reject the null hypothesis when it is false. In reality, there was a significant difference or relationship to find in this case and we didn’t find it. 7
- 8. Scenario 4: Null: Reality: Conclusion: Decision: There is no Null is true: Reject the null: Incorrect difference or There is no Conclude there is Decision: Type relationship. difference or a difference or I error (α) relationship. relationship. We reject the null hypothesis when it was in fact true. In reality, there was not a significant difference or relationship to find in this case and we found a difference or relationship. 8
- 9. Type I and Type II error: A Type I Error is the false rejection of a true null. It has a probability equal to alpha ( α ). A Type II Error is the false retention of a false null. It has a probability equal to beta ( β ). 9
- 10. Statistical power: Power is the probability of correctly rejecting a false null hypothesis. It’s represented by 1- β . Factors affecting statistical power: 3. Sample size 4. Effect size 5. Alpha level 6. Directionality 10
- 11. Sample Size Sample size affects power. All else being equal, a larger sample size yields more power than a smaller sample size. 11
- 12. Effect Size Power is influenced by effect size. Effect size is the difference between the value of the null and the value of the alternative, or the desired difference to be detected. All else being equal, a test is more powerful with a larger effect size. 12
- 13. Alpha Level The level of significance, or alpha, influences power. All else being equal, a higher alpha yields higher power. Does this mean we necessarily want to set a high alpha level? 13
- 14. Directionality Power is affected by directionality of the test. All else being equal, a one-tailed test is more powerful than a two-tailed test. Why? 14
- 15. In-Class Activity: Group 1: Influence of alpha level on power Group 2: Influence of sample size on power Group 3: Influence of effect size on power Group 4: Influence of directionality on power 15