Quantitative Reasoning

Design of studies

Sudarshan Narasimhan
(Dash)

[email protected]
Tutor
Provost office, QR team

1
Definitions
 In experiments and observational studies, there are typically 2 sets of variables,
namely independent variables and dependent variables.
 An independent variable is a variable that represents a quantity/feature that is subject to
variation in an experiment/observational study
 A dependent variable is a variable which potentially changes depending on how the
independent variable is manipulated in an experiment/observational study.

2
Other forms that they go by
Controlled experiment Observational study
Independent variable - Mainly consists of Independent variable - Also known as the
treatments and controls exposure variable

Dependent variable - Also known as Dependent variable - Also known as

outcome/response/disease variable outcome/response/disease variable

3
Controlled experiments
 Statisticians use the method of Comparison to see the effect of a treatment(s) on an
outcome(s).
 Comparison of outcomes between a treatment group and a control group.

4
Effectiveness of vaccine
 Scenario : You want to test the effect of a vaccine against COVID-19 infections in Singapore.
You inject all Singaporeans with the vaccine and compare infection rates to before your vaccine
was implemented.

 You discover that the infection rate is SIGNIFICANTLY lower compared to before your vaccine
came into the market. Can we say the vaccine works?

 Why/why not?

5
 Impt
Ideal features of both groups Point!

How similar must the treatment group and the control group be?

How different must the treatment group and control group be?

 What if the control group and treatment group are different in other factors?
 It becomes difficult to ascertain if it’s really the treatment that is responsible for observed differences
between both groups.

6
How can I ensure similarity?
We should try to put subjects in control and treatment groups randomly.
Impt
Point!
 Law of large numbers ensures that all other variables are almost equally present in both groups.

Random allocation

7
But then…..
 What if the people KNOW whether they’re in the control
or treatment group?

 Placebo

8
Blinding (eliminating human
bias)
In a single-blind experiment, either the participants OR evaluators (only one), can’t distinguish
between the treatment and control group.
In a double-blind experiment, BOTH participants AND evaluators (both) çan’t distinguish
between treatment and control groups.
The blinding generally is done to guard against human bias. Impt
Point!

9
Double blind random controlled

Single blind random controlled

Random controlled

Treatment and control

(w/o randomization)

No Control

10
Observational studies
In an observational study, the investigators do not assign the subjects to either group.

For example, in a study on smoking and heart disease, the smokers form the treatment
exposure group and the non-smokers form the control group. It’s a self-selecting process.

It is NOT TRUE that observational studies have NO CONTROL group. There IS a control group

Impt
Point!

11
Association and causation
Observational studies can ONLY establish association:
One thing is linked to another.

But association does not prove causation.

Impt How to examine association? By comparing rates.

Assume there are 2 characteristics A and B. compute these two:
Point!
Rate(A|B) & Rate (A|not B). If these two are different, A and B are associated.

12
Is smoking ASSOCIATED with heart
disease?
HD No HD Row Total
Smokers 38 14 962 15 000
Non-smokers 44 84 956 85 000
Column Total 82 99 918 100 000

13
 Impt
Some notes on rates Point!

Symmetry
 If Rate(A|B) ≠ Rate (A|not B) then Rate(B|A) ≠ Rate (B|not A) all the time.

Direction of an association:
 A and B are positively associated if Rate(A|B) > Rate (A|not B)
 A and B are negatively associated if Rate(A|B) < Rate (A|not B)

Combined rates.
 r(A | B) = x and r(A | C) = y. Then overall r(A | B + C) is between x and y.

14
 Exam
Definition Point!

 A confounder is a third variable, associated with dependent and independent variables.

 It MUST be necessarily a different variable from the dependent and independent variables.

15
The problem with observational
studies

16
 Let me
think

Controlling the confounder.

deeper

17
Simpson’s paradox
Relationship between rates in subgroups are reversed when subgroups are combined.

SURE SIGN of confounding variable when you encounter Simpson’s paradox.

Impt
Point!

Counter by slicing.

18
Example of how Simpson’s
paradox occurs
Major Males Females

A 2000 applied. 40% selected 10 applied 20% selected

B 10 applied 80% selected 1000 applied 60% selected

C 10 applied 70% selected 1000 applied 50% selected

19