Welcome to Analysis of

Behavioural Data

CHAPTER 8
INSTRUCTOR: NICOLE JENNI
Announcements
Assignment 2: MAR 12
Mar 5/7: Ch 8 part 2, Ch 9
Mar 12 Ch 10 + Demo 3 posted
Mar 14: Start Ch 11, Midterm Review
Mar 19: MIDTERM 2 *CH 7-10* (Ch 11 moved to final)
Mar 26: Assignment 3 due

Class Mate’s survey: looking at ways to reduce single use cups on campus. Please respond!
https://ubc.ca1.qualtrics.com/jfe/form/SV_79g74hhWXarLzFA
 Cohen’s d
One of many “standardized” indicators of effect size.
Estimates the population parameter δ (delta).
How many standard deviations away from the [comparison value] is our
sample group mean*?
𝑀−𝜇 How is it different?
𝑑= 𝑧𝑜𝑏𝑡𝑎𝑖𝑛𝑒𝑑 =
𝑀−𝜇𝑀
𝜎 𝜎𝑀

This is the standard deviation (NOT SEM)

Cohen’s d for comparing a sample
against a known population mean
How many standard deviations away from the known
population mean on happiness is our class mean?

𝑀−𝜇
𝑑=
𝜎
Effect sizes look at difference between the
two groups, independent of sample size

H0 H1

𝑟𝑎𝑤 𝑔𝑟𝑜𝑢𝑝 𝑚𝑒𝑎𝑛 𝑠𝑐𝑜𝑟𝑒𝑠 68 71

 e.g., Sample mean
Our best
estimate of
the true
population
parameter

subtract “Margin of Error” add “Margin of Error” to

to find Lower Bound find Upper Bound

Critical Standard Critical Standard

X X
value Error value Error

ALL CIS FOLLOW THE SAME PAT TERN… AND WE’RE

GOING TO SEE THIS REPEATEDLY FOR THE REST OF
THE TERM.
Start with a point estimate of the mean of
whatever population our sample belongs to.
Our class sample statistics:
n = 74, M= 2.101, s = 0.684

1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0

“How happy are you?” (Response range 1-4)

 Add a Confidence Interval to identify the range of
plausible values for the mean of whatever population
our sample belongs to.
𝑀 sample ± zcritical(𝝈𝑴 )

Look up Zcrit using your

Z-table. 95CI would Lower Boundary Upper Boundary
have 2.5% in each tail
𝑀lower = 𝑀sample - zcritical(𝝈𝑴 ) Mupper = 𝑀sample + zcritical(𝝈𝑴 )

1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0

“How happy are you?” (Response range 1-4)

Adjusted the width of the bar to reflect
our upper and lower .95CI boundaries

Lower Boundary Upper Boundary

𝑀 lower = Msample - zcritical(𝝈𝑴 ) 𝑀 upper = Msample + zcritical(𝝈𝑴 )
𝑀 lower = 2.101 – 0.1683 𝑀 upper = 2.101 + 0.1683
𝑀 lower = 1.9326 𝑀 upper = 2.2694

Interval estimate

1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0
 Recall that µWorldValuesSurvey = 1.86 and
.95CI for μclass = [1.96, 2.24]
How likely is it that our class is a random sample from
this general population? Or do we likely come from a
different population?
Is our sample statistically significantly
different from the general world
population? .95CI for μclass

A) Yes
B) No 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6
C) I don’t know
Is our class just as happy as the general population
represented in the World Values Survey?
One way of expressing this result…
H1: µclass ≠ µWVS
H0: µclass = µWVs Our class is significantly less happy
(𝑀= 2.10, s = 0.68, n = 74, .95CI for
μ = [1.96, 2.24]) than the general
How do we write this up population represented in the
in words? World Values Survey (μ = 1.86, σ =
0.74, Cohen’s d = .33.
Stop and Think
𝑀sample + zcritical(𝝈𝑴 )
95% CI
How would things change if
Z=1.96 I asked you to compute a
90% CI???
45% 45%
95% CI: 𝑴sample + 1.96(𝝈𝑴 )
5%
90% CI 5%

Z=1.65
Stop and Think
𝑀sample + zcritical(𝝈𝑴 )
95% CI
How would things change if
Z=1.96 I asked you to compute a
90% CI???
45% 45%
95% CI: 𝑴sample + 1.96(𝝈𝑴 )
5%
90% CI 5%
90% CI: 𝑴sample + 1.65(𝝈𝑴 )
Z=1.65
 Headline from CBC Calgary:
Alberta workers have among the highest stress levels in Canada,
says survey
http://www.cbc.ca/news/canada/calgary/alberta-workers-stress-morneau-shepell-1.4513104

“The margins of error are plus or minus three per

cent, 19 times out of 20.”
19/20 = .95
Q6. Is Alberta’s workplace stress significantly
different from Ontario’s?
A) Yes B) No
Point
Estimate of
μstress based
on sample
B Province data
BC 29
A
Alberta 36
S
Saskatchewan 38
M
Manitoba 38
O
Ontario 41
Q
Quebec 29
At
Atlantic 31
T
Territories 29
26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

“The margins of error are plus or minus three per cent, 19 times out of 20.”
Is Alberta’s workplace stress significantly
different from BC’s?
A) Yes B) No
Point
Estimate of
μstress based
on sample
Province data
B
BC 29
A
Alberta 36
S
Saskatchewan 38
M
Manitoba 38
O
Ontario 41
Q
Quebec 29
At
A
Atlantic 31
T
Territories 29
26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

“The margins of error are plus or minus three per cent, 19 times out of 20.”
Statistical
Power
THE ABILITY TO DETECT AN EFFECT ( IF IT EXISTS).
THE ABILITY TO REJECT A NULL HYPOTHESIS ( THAT
SHOULD BE REJECTED).
Which technique would be better able to
detect a tumour?
A. CT Scan B. MRI Scan

MRI is a more powerful technique than CT.

It’s better able to detect brain structures that exist.
If we don’t have enough statistical power,
what error will we make?
A. If the null is not true, a Type I error
B. If the null is not true, a Type II error
C. If the null is true, a Type I error
D. If the null is true, a Type II error
Assuming there really is “an effect”…

Retain the null

- No tumour (but there
Not enough Type II is!)
Power? Error
- Sample came from
null population (but it
didn’t!)
We make a decision…
Truth
(if we had access to the full population from which
we drew the sample, this is what we’d find)

H0 is True H0 is False
Researcher’s Reject H0 Type I Error Correct Decision
Decision PUBLISH! (Rate = α) Power (1-β)
(based on
Retain H0 Correct Decision Type II Error
sample)
(1-α) (Rate = β)
(1-α) + (α) = 1.00 (1-β) + (β) = 1.00

If the null is really false (effect exists), β% of the time we’re going to make a
mistake and say there is NO EFFECT!
 Statistical Power
The measure of the likelihood we will reject
the null hypothesis, given that the null
hypothesis is false
◦ Probability that we will reject the null when we Power (1- β)
should

◦ Probability that we will avoid a Type II error

◦ Ranges from probability of 0.00 to probability of 1.00

Type II error (β) Type I alpha (α)
◦ 80% historically used as minimum for study
conduction
You can compute Power by calculating
the area under the curve

Explore this “Power Analysis” visualization: https://rpsychologist.com/d3/NHST/

Optional Activity & Study tool: Exploring
conceptual relationships using the visualization
•What happens to power when d = 0? What error might we make in that
case?
•What happens to power when the effect size is medium?
• What can we do in our study to achieve at least 90% power?

•If we’re stuck with a tiny effect size, what’s the maximum power we can
achieve?
•If we only have 10 people, what effect size is needed to give us power =
80%? (Is that number a realistic expectation?)
•What happens to power if we switch from two-tailed alpha to a one-
tailed alpha?

Explore this “Power Analysis” visualization: https://rpsychologist.com/d3/NHST/

Power is a good thing… so
how do we get MORE?
 Five factors that affect statistical power
1. Increasing alpha Conceptually, these
increase power by
2. Turning a two-tailed hypothesis into a one- moving the rejection
tailed hypothesis line inwards
3. Increasing N
Conceptually, anything
4. Exaggerating mean difference between levels that increases your test
of the independent variable (M −𝜇) statistic will increase
power
5. Decreasing standard deviation (𝜎) 𝑀−𝜇 z=𝑀−𝜇
z= 𝜎
𝜎𝑋ത
𝑁
1. Increasing Alpha
2. Two-Tailed Versus One-Tailed Tests
3&4. Increasing Sample Size or Decreasing
Standard Deviation (both reduce SEM)
5. Increasing the Difference Between the
Means (Effect Size)
PRACTICE PROBLEM
COHEN’S D
 You are interested in understanding the mental health
repercussions of the COVID19 pandemic on university
students. You sample 100 students from UBC and have them
complete the Beck’s Depression Inventory (BDI).
The population of university students score a mean of 17.2 with a standard deviation
of 6. In your sample, you measure a mean BDI score of 18.3 with a standard deviation
of 8. A priori you decide to run a one-tailed test, with alpha= 0.05. your research
hypothesis is that COVID19 increased BDI scores. Conduct a null hypothesis test and
draw a conclusion
 You are interested in understanding the mental health
repercussions of the COVID19 pandemic on university
students. You sample 100 students from UBC and have them
complete the Beck’s Depression Inventory (BDI)
The population of university students score a mean of 17.2 with a standard deviation of 6. In your
sample, you measure a mean BDI score of 18.3 with a standard deviation of 8. A priori you decide to run
a one-tailed test, with alpha= 0.05. your research hypothesis is that COVID19 increased BDI scores.
Conduct a null hypothesis test and draw a conclusion
SEM: Zcrit: ??

5%

𝑧 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐 𝑠𝑐𝑜𝑟𝑒𝑠 −2 −1 0 1 2
?? ?? 17.2 ?? ??
𝑟𝑎𝑤 𝑔𝑟𝑜𝑢𝑝 𝑚𝑒𝑎𝑛 𝑠𝑐𝑜𝑟𝑒𝑠
 You are interested in understanding the mental health
repercussions of the COVID19 pandemic on university
students. You sample 100 students from UBC and have them
complete the Beck’s Depression Inventory (BDI)
The population of university students score a mean of 17.2 with a standard deviation of 6. In your
sample, you measure a mean BDI score of 18.3 with a standard deviation of 8. A priori you decide to run
a one-tailed test, with alpha= 0.05. Conduct a null hypothesis test and draw a conclusion

SEM: 0.6 Zcrit: 1.65

5%

𝑧 𝑠𝑡𝑎𝑡𝑖𝑠𝑡𝑖𝑐 𝑠𝑐𝑜𝑟𝑒𝑠 −2 −1 0 1 2
16.0 16.6 17.2 17.8 18.4
𝑟𝑎𝑤 𝑔𝑟𝑜𝑢𝑝 𝑚𝑒𝑎𝑛 𝑠𝑐𝑜𝑟𝑒𝑠
PRACTICE: CIs
Practice: You are interested in whether students who attend private school receive a
better education that the general population of highschool students. The provincial
grade 12 (standardized) test population mean is 77% standard deviation 15%. You
sampled 65 BC private school students and measured a mean of 81% standard
deviation 13%

A. Set a 95% confidence interval for your sample mean, and use this to 𝑀sample ± zcritical(𝜎𝑀 )
decide whether you should reject or retain your null hypothesis.
𝑀−𝜇
𝑑=
𝜎
B. Now set a 90%CI and a 99% CI. How does this change your interval? 𝑀−𝜇
How does this change your decision? z=
𝜎𝑀
𝜎
95% CI → alpha .05 → zcrit 1.96 𝜎𝑀 =
𝑁
90% → alpha .10 → zcrit 1.65
99% CI → alpha .01 → zcrit 2.59
Practice: You are interested in whether students who attend private school receive a
better education that the general population of highschool students. The provincial
grade 12 (standardized) test population mean is 77% standard deviation 15%. You
sampled 65 BC private school students and measured a mean of 81% standard
deviation 13%

C. What is the effect size? (Cohen’s D) 𝑀sample ± zcritical(𝜎𝑀 )

𝑀−𝜇
𝑑=
𝜎
𝑀−𝜇
z=
𝜎𝑀
𝜎
𝜎𝑀 =
𝑁
8. A treatment is administered to a sample of N = 12 individuals selected from a
population with a mean of µ= 75 and a standard deviation of σ = 10. After treatment,
the effect size is measured by computing Cohen’s d, and a value of d = 0.60 is
obtained. Based on this information what is the mean for the treated sample?

A. 75
B. 81
C. 87
D. It cannot be determined based on the information provided.
8. A treatment is administered to a sample of N = 12 individuals
selected from a population with a mean of µ= 75 and a standard
deviation of σ = 10. After treatment, the effect size is measured by
computing Cohen’s d, and a value of d = 0.60 is obtained. Based
on this information what is the mean for the treated sample?

A. 75
B. 81 (correct answer)
C. 87
D. It cannot be determined based on the information provided.