Unit 5 - Experimental Design 2022
Unit 5 - Experimental Design 2022
Designing
Experiments
1
Definitions:
Observational Study - observe
outcomes without imposing any
treatment
Experiment - actively impose some
treatment in order to observe the
response
Survey - simply asking respondents for
data, no observation or treatments 2
I’ve developed a new rabbit food, Hippity
Hop, to improve the overall health of
rabbits. Makes fur
soft & shiny!
Increases
energy!
Rabbit Food
100% of daily
vitamins &
essential oils! 3
Can I just make these claims?
NO!
What must I do to make these
claims? Do an experiment
Rabbit Food
9
Rabbit Food
14
Definition:
• Confounding Variable – A variable whose
effect on the response cannot be
separated or distinguished from the
Becauseofobservational
effects the factor onstudies may contain
the response.
confounding variables, their results can NOT
be used to show cause-effect relationships.
• It is a variable that is related to both
group membership and the response
variable of interest in a research study
15
Principles of Experimental Design
• Control - the effects of extraneous
variables on the response
• Randomization – the use of chance to
assign subjects to treatments
• Replication - of the experiment on
many subjects to quantify the natural
variation in the experiment
• Comparison - the experiment must
have two or more treatments
(one of which may be a control or placebo) 16
The ONLY way to show
cause & effect is with a
well-designed,
well-controlled
experiment!!!
17
Example 1: A farm-product manufacturer wants to
determine if the yield of a crop is different when the
soil is treated with three different types of fertilizers.
Sixteen similar plots of land are planted with the same
type of seed but are fertilized differently. At the end of
the growing season, the mean yield from the sample
plots is compared.
A A B B
NO! The amount of
water or soil type is A A B B
confounded with the
type of fertilizer! A A B B
A A B B19
Example 2: A consumer group wants to test
cake pans to see which works the best (bakes
evenly). It will test aluminum, glass, and
plastic pans in both gas and electric ovens.
Experiment units? Cake batter
Factors? Two factors - type of pan & type of oven
Measure
Treatment
Random Assignment
Response
A for A
Experimental Compare
Units Treatments
Treatment Measure
Response
B
for B
22
Suppose the researcher decides to randomly
Suppose that the sixteen plots of land are not similar.
assign fertilizer A to 8 plots and the remaining
Some are close to a river with trees along the bank,
plots with B. He has a greater yield with
while others are farther away from the river. See the
fertilizer A. Does this mean that A is the
diagram below.
better fertilizer?
A B B A23
Experimental Designs Continued . . .
Randomized block – units are blocked into
Units should be blocked on a variable that
groups (homogeneous) and then randomly
affects the response!!!
assigned to treatments
Measure
Random Assignment
Treatment A
response
for A
Block Compare
treatments
1 for block 1
Treatment B Measure
response
Experimental
Measure
Random Assignment
Treatment A
response
for A
Block Compare
treatments
2 for block 2
Treatment B Measure
response 24
for B
Then thethe
Suppose researcher
researcher randomly
decides assigns
to blockfertilizer
the 16 Aplots
to 4
Suppose
plots in that
the pinkthe sixteen
block and toplots
4 of inland
plots the are
blue not similar.
into two blocks based on their proximity to the river.
Some are remaining
block. The close to aplots
riverreceive
with fertilizer
trees along B. Hethehasbank,
while others
a greater yield are
withfarther
fertilizeraway
A. Doesfrom
thisthe
mean river.
that See
A the
is the better
diagram fertilizer?
below.
A B B A
Yes! Blocking is a way
to “control” the
A A B B
effects of extraneous
variables. B B A A
B A B A25
Matched pairs - a special type of
block design
– match up experimental units according
to similar characteristics & randomly
assign one to one treatment & the
other automatically gets the 2nd
treatment
– have each unit do both treatments in
random order
– the assignment of treatments is
dependent
26
Treatments vs.
Treatment A
Measure
Treatment Groups
Random Assignment
response for A
Experimental Compare
Units treatments
Measure
Treatment B response for B
Completely Randomized
Block Compare
treatments
1 for block 1
Treatment B Measure
Measure
Random Assignment
Treatment A
response
for A
Block Compare
treatments
2 for block 2
Treatment B Measure
response 27
for B
Treatment A Treatment B
Pair
Next, experimental
randomly assignunits
according
one unit from atopair
specific
to
Treatmentcharacteristics.
A. The other
unit gets Treatment B.
29
Spanking and IQ Continued . . .
Confounding
In this variables
study, two groups mustwere
of children be associated
followed forto both 806
4 years;
children
to theages 2 to 4 and 704
explanatory children
(factor) ages
and 5 to 9. IQvariables.
response was measured
at the beginning of the study and again four years later.
Be sure found
Researchers to describe
that thethose relationships
average IQ of children,when
ages 2asked
to 4, who
to identify
were not spanked was 5 to confounding
points higher thanvariable!
those who were
spanked and 2.8 points higher for children, ages 5 to 9.
These are called
Does spanking “CAUSE”confounding
a decrease in IQ?
Why or why not? variables.
Treatment A Treatment B
Confounding does NOT
occur in a completely
randomized design!
High bias & high variability High bias & low variability
Low bias & high variability Low bias & low variability
38