Experimental

Study Design

Dr. Muhammad Asif Syed

MBBS, MBA, MPH, MPhil, PhD
PGC Field Epidemiology (CDC-Atlanta)
PGC One Health (USA)
PGC Biosafety and Biosecurity (Germany)
Epidemiology
• The study of the distribution and determinants of health
related states or events in specified populations and
application of this study to the control of health problem.
( John M Last, 1988 )
Study Design Hierarchy
 Experimental Study (Also Known as Intervention Studies )
• Best study design to prove causation.

• Investigator decides who will get the exposure and who will not.
• So under direct control of the investigator unlike other type prospective
study where exposure is not dictated by the investigator.

• Epidemiologist takes some action, intervention or manipulation

in contrast to descriptive studies where no action is taken but
observation is done.
Aims of Experimental Studies
1. To provide scientific proof of etiological factors.

2. To provide a method of measuring the effectiveness and

efficiency of health services.
Types of Experimental Studies

• Clinical Trials Field Trial

• Non Randomized Trial • Risk Factor Trial
• Animal Studies
• Randomized Control • Community Intervention Trial
• Human Studies
Trial • Health Services Evaluation
Trial
• Cessation Experiment
Animal Studies
Animal Studies
• Example

• Research in cows helped create the world's first

vaccine, which in turn helped end smallpox.

• Studies with monkeys, dogs, and mice led to the

polio vaccine.

• Drugs used to combat cancer, HIV/AIDS,

Alzheimer's, hepatitis, and malaria would not
have been possible without research with
primates.
Animal Studies
• Have Played important role in men’s quest for knowledge.
• Important applications of animal experiments have been in:
• Experimental reproduction of Human diseases in animals
• Testing the efficacy of preventive and therapeutic measures.
• Completing the natural history of disease.
Advantages Limitations
• Can be bred in labs and can be • Not all human diseases can be
manipulated easily. reproduced in animals.
• Rapid multiplication reduces the • Conclusions may not be strictly
time of research. applicable to human beings
Human Studies
Planned trial on scurvy by James Lind
in 1747:
• James Lind, a Scottish surgeon took 12 sailors suffering from scurvy and
divided them into 6 pairs.

• He than assigned each pair to 6 different treatments for scurvy.

• The two patients who were given lemons and oranges recovered most
quickly, suggesting a beneficial effect of citrus.
 Planned trial • Edward Jenner’s experiment with
cowpox in 1796.
‒ James Lind, a Scottish surgeon • Finlay and reeds experiments (1881-
took 12 sailors suffering from 1900)
scurvy and divided them into 6
• Mosquito borne nature of Yellow fever
pairs.
was elucidated.

‒ He than assigned each pair to 6 • Goldberger’s Pellagra experiments,

different treatments for scurvy. 1915
• Inducing pellagra by diets deficient in
‒ The two patients who were given nicotinic acid.
lemons and oranges recovered • Providing evidence that pellagra is a
most quickly, suggesting a nutritional deficiency disease and not
beneficial effect of citrus. an infectious disease.
 Human Studies
• Since, not all the diseases can be reproduced in animals, Human studies will
always be needed. With Human Studies:
‒ Disease etiology can be investigated.
‒ Evaluation of preventive and therapeutic measures can be done.

• Problems in application of experimental method for studying diseases in

humans:
‒ Ethical Considerations
‒ Logistics

• So, always weigh benefits of the experiment against risks involved.

 What to use?

OR
 Ask yourself following questions:
• Are you studying the efficacy of a Therapeutic Procedure / Preventive
Procedure / Health Care System ?
‒ If Answer is Yes, go to the next question.

• Can you Randomly divide the subjects into two groups, one getting the
intervention and other not to get it?
‒ If Answer is Yes, go ahead with RCT.

• But, If Answer of any question is No, explore possibility of using an

Observational study or a Quasi experimental design (Non-Randomized Trials).
 NON

CONTROL TRIAL
Non-Randomized Trials
(Quasi-Experimental Designs)
• A clinical trial in which the participants are not assigned by
chance to different treatment groups

• As here randomization is not done.

‒ So, low comparability than RCT and chances of spurious results are
high than RCT.
 Non-Randomized Trials
(Quasi-Experimental Designs)
• In this design the investigator does not assign individual subjects randomly
to the exposed & non- exposed groups.

• These are designed as:

‒ It is always not possible for ethical, administrative and other reasons to choice a RCT.
‒ Some preventive measures apply only to groups or community-wide basis.
‒ When disease RCT require follow-up of thousands of people for a decade or more.
Types of Non-Randomized Trials

Before and
Uncontrolled Natural After
Trials Experiment comparison
(With or without
Control)
Uncontrolled Trial
• A study in which all the participants are given a treatment and
simply followed for a period of time to see if they improve, with no
comparison against another group (control group)

• Uncontrolled trials are often used in the early phases of drug

research, phases I and II, to determine pharmacokinetic
properties or to investigate tolerated dose ranges.

• They can also be useful to study side effects, biochemical

changes in long-term therapies, tolerance, interactions or
efficacy of drugs, appropriate dose.
 Uncontrolled Trial
• Use of implied “historical controls” i.e., the experience of earlier
untreated patients affected by the same disease.
• (When the disease is uniformly fatal and a new drug becomes available a decline
in case fatality that parallels the use of the drug would suggest that the drug is
having effect.)

Example: indirect epidemiological evidence from well over a

dozen uncontrolled studies of cervical cancer screening that the
Pap test is effective in reducing mortality from this disease.
 Natural Experiments
• When experimental studies are not possible in
humans, Natural circumstances that “mimic”
an experiment are identified.

• Example: Group of smokers and non-smokers

(naturally separated). Other population groups
involved include: migrants, religious or social
groups etc.

• John Snow’s discovery that cholera is a water

borne disease was an outcome of a natural
experiment.
Before and after comparison studies with
control
Before After
 Before and after comparison studies
with control
• Experiment serve as its own control.

• Incidence of disease before and after introduction of

intervention is measured here.

• Standard for comparison: events which took place

prior to use of new treatment or intervention.

• Examples:
‒ Prevention of scurvy among sailors (1750).
‒ Studies on cholera by John Snow (1854).
‒ Prevention of polio by Salk and Sabin.
 Before and after comparison studies
without control
• In absence of control group, results of comparison may be
misleading.

• Alternative is to utilize a “Natural control group” i.e., the one

provided by nature or natural circumstances.

• Example:
‒ Effect of seat belt legislation in one district on RTA related
mortality, compared with the another district with no seat
belt legislation
Randomized Control Trials
Randomized Control Trial
• An epidemiologic experiment in which subjects in a population
are randomly allocated in to groups, usually called study and
control groups to receive or not to receive an experimental,
preventive or therapeutic procedure, maneuver or intervention.
(John M Last Dictionary of Epidemiology 2nd Edition)

• Primary Goal
• To test whether an intervention works by comparing it to a control condition
(usually either no intervention or an alternative intervention).

• Secondary Goals
• Identify factors that influence the effects of the intervention (i.e., moderators)
 Steps in conducting a RCT
1. Clinical research question 5. Intervention
2. Type of trial • Experimental
• Control (other, placebo, nothing)
3. Selecting trial population
• Inclusion/exclusion criteria 6. Follow up and measurement
• Sampling • Baseline (at entry)
• Sample size • At the end of follow-up
• Confounders
4. Assignment
• Stratification 7. Analysis of results
• Randomized vs. non-randomized • Comparability of groups
• Outcomes
• Confounders
Steps in conducting a RCT
1.Drawing up a protocol
2.Selecting reference and experimental populations.
3.Randomization
4.Manipulation or intervention
5.Follow – up
6.Assessment of outcome.
 TARGET POPULATION General outline of an intervention trial.

EXPERIMENTAL POPULATION

ELIGIBILITY CRITERIA UNSUITABLE SUBJECTS

INFORMED CONSENT

STUDY POPULATION
REFUSAL
RANDOM ALLOCATION

INTERVENTION CONTROL GROUP

GROUP

FOLLOW-UP

OUTCOME OUTCOME OUTCOME OUTCOME OUTCOME OUTCOME

KNOWN UNKNOWN KNOWN UNKNOWN KNOWN UNKNOWN
Step 1. Drawing up a protocol
• Should be strictly adhered to throughout the study.
• Preventing bias and to reduce the source of error

• The Protocol specifies*:

1. Aims and Objectives of the study
2. Questions to be answered
3. Selection criteria for Study and Control group
4. Sample size
5. Procedures for allocation of subjects into study & control groups
6. Treatment or intervention to be applied
7. Standardization of working procedures and schedules
8. Responsibility of parties involved in trial.

* up to the stage of evaluation of outcome of study

Step 2a. Selection of Reference Population
• Also known as Target Population.
• It may be as broad as mankind or limited
to specific groups.

• It is the population to which findings of the

trial, if found successful, are expected to be
applicable.
Target
• Thus, it may comprise of the population of
whole city, or a population of school
children, industrial workers, obstetric
population and so on according to the
nature of the study.
 Step 2b. Selection of Experimental Population
• Choose randomly from the reference
population.

• If study population differs from the

reference population, it may not be
possible to generalize the findings of
Randomization the study to reference population

• Once defined, its members are invited

Experimental
to participate. Cooperation should be
Population
assured to avoid losses to follow up
 Step 2c. Participants fulfill these criteria
• Give “informed consent.”
• Representative of the population.
• Qualified or eligible for the trial.

• E.g.
• For testing a new drug for the
treatment of anemia participants
should be anemic.

• In the test of a new vaccine

against whooping cough,
participants already immune to
the disease in question, are not
qualified
Step 2d. Inclusion and Exclusion Criteria
• Inclusion Criteria:
• To specify who will be eligible to
be included in the study, based
on demographic and clinical
characteristics.
• Exclusion Criteria:
• To define who will not be eligible
to be included in the study.
• More the exclusion criteria:
• More precise findings, and lesser
requirement of sample size.
• More difficult to find subjects and
generalizability will be restricted
Step 3. Randomization
 Step 3. Randomization
• It is the heart of a control trial. Allow the available participants in experiment
and control group

• It is different from Random sampling.

• Gives confidence of like being compared to like.

• Randomization aims at:
• Achieving Internal Validity.
• Eliminate bias (Selection bias).
• Allowing comparability.
• Equally distributing the “other” factors which are important but not recognized or cannot be
determined, equally between the two groups.
Methods for Randomization
• Simple Random Allocation
• Randomization in groups of Two
• Systematic Allocation
• Stratified Allocation
Simple Random Sampling
 Simple Random Allocation
• Most elementary form of randomization.
• It is usually carried out by
• Lottery Method
• A random-number table or
• A computerized random number
generator.

• Advantage: Easy.
• Allot participants into Group A or Group B by:
• Disadvantage: Possibility of an imbalance • 0-4: Group A and 5-9: Group B
between the two groups at any one point Or
specially if the sample size is small. • Even no.: Group A
• Odd no.: Group B
Systematic Allocation
Systematic Allocation
• This method also ensures equal number of participants in experimental and
control groups if even no. of participants take part in the study.

• Here, first patient is randomly allocated to a group, next patient goes to

alternate group automatically.

• Subsequent patients are allocated into groups alternatively.

• Being convenient and having statistical advantages makes this method desirable
to use whenever possible.

• This can also be used for allocating study participants to three, four or more
groups.
Stratified Randomization
Stratified Randomization
• It is often called Prognostic Stratification.
Stratified Randomization • It is done to ensure that the treatment and
control groups are balanced on important
prognostic factors that can influence the
study outcome (e.g., gender, ethnicity, age,
socioeconomic status).
• Investigator decides which strata are important
and how many stratification variables can be
considered given the proposed sample size.
• A separate simple or blocked randomization
schedule is developed for each stratum.
• Large trials often use randomly permuted
blocks within stratification groups.
Step 4. Manipulation or Intervention
• Deliberate application or withdrawal or reduction of the
suspected causal factor as laid down in the protocol.

• Independent variable (e.g., drug, vaccine, a new procedure)

is created by intervention whose impact is measured in the
final outcome.

• This constitutes the dependent variable (e.g., incidence of

disease, survival time, recovery period).
Step 5. Follow-up
• Examination of experimental & control group subjects,
• At defined interval of time
• In a standard manner
• With equal intensity
• Under same given circumstances
• In same time frame till final assessment of outcome.

• Duration of trial is based on expectation that a significant difference (e.g.

mortality) will be demonstrable at a given point in time after the start of
trial ( i.e. the ‘natural history of disease’). Thus, it may be short or long,
depending upon study undertaken.
Follow-up
• Few events may also be associated with the trial:
• Adverse event (AE) : Undesirable health occurrence that occurs during the trial and that
may or may not have a causal relationship to the treatment.
• Serious adverse event (SAE): Something life-threatening, requiring or prolonging
hospitalization and/or creating significant disability. E.g., A suicide attempt - SAE in a
study of any treatment.

• The SAE needs to be reported regardless of whether it bears any relationship to

the treatment or the problem being studied

• Depending on the severity and frequency of adverse events, investigators and

data safety monitors may have to decide to terminate the trial prematurely.
Loss to follow-up
• Also known as Attrition. • During the trial subjects may deviate from the
protocol for various reasons including:
• Some losses to follow-up are
inevitable due to factors, such as: • Developing side effects.
• Forgetting to take their medication.
• Death • Simply withdrawing their consent after
• Migration randomization.
• Loss of interest • Avail alternative treatment on their own
initiative.
• If these losses (i.e. attrition) are • Contraindication of a therapy because of non
substantial than it may be difficult to compliance for a prolonged period.
generalize the result for reference
• So, every effort should be made to minimize the
population losses to follow up and increase the
compliance of the study participants.
Step 6. Assessment
• The final step is assessment in terms of:
• Positive results i.e., benefits of experimental measures such as
reduced incidence or severity of disease, cost to health service etc.
• Negative results i.e., severity and frequency of side-effects and
complications, if any, including death.

• Incidence of results compared rigorously and differences are tested for

significance.
• Data analysis may be done sequentially or at the end of trial (more
useful).
• Errors of assessment of outcomes due to human elements may give rise
to Bias
Analysis
• If all participants enter and leave the trial at the same time, the
risk can be calculated
• For example, if the follow-up period is uniformly three years, the
three-year risk can be computed for each study group. The two
study groups can be compared by calculating risk ratios and
risk differences as measures of relative and absolute effect,
respectively.
Outcome Exposure
Intervention Control Risk in intervention group (p1) = a/(a+c)
Risk in control group (p0) = b/(b+d)
Yes a b Risk ratio = p1/p0
No c d Risk differencea = p0 - p1
Example
• In the Physicians’ Health Study described in Examples 7.6 and 7.8, 22 071
US male physicians aged 40 to 84 years were randomized in 1982 to receive
one of four treatments:
1. Aspirin plus beta-carotene placebo
2. Beta-carotene plus aspirin placebo
3. Both active agents
4. Both placebos.
• The randomized aspirin complement of the trial was terminated early, in
1988, by the external data-monitoring board because it became apparent
that there was a 44% reduction (P < 0.001) in the risk of a first myocardial
infarction in those taking aspirin. The randomized beta-carotene component
continued uninterrupted until its scheduled termination in 31 December 1995.
• A total of 11 036 physicians received beta-carotene and 11 035 received
beta-carotene placebo and fewer than 1% were lost to follow-up. One of the
main aims of this component of the study was to assess whether beta-
carotene reduces the incidence of lung cancer
Results
• The results from this trial were consistent with the null
hypothesis of no treatment-associated difference in the risk of
lung cancer, that is, a risk ratio equal to 1 (or a risk difference
equal to zero).
Example (Person time)
• The objective of the ATBC trial was to assess whether daily
supplementation with alpha-tocopherol, betacarotene or both
would reduce the incidence of lung cancer and other cancers.

• A total of 29 133 male smokers aged 50 to 69 years from south-

western Finland were recruited between 1985 and 1988.
Follow-up continued for 5–8 years (median = 6.1), until death or
30 April 1993, when the trial ended (Alpha-Tocopherol, Beta
Carotene Cancer Prevention Study Group, 1994).
The results by type of treatment received
are shown in Figure
Result
• In example , the duration of follow-up varied from subject to
subject. Thus it is more appropriate to calculate person-time at
risk and rates as the measure of occurrence of disease.

• The results of this trial did not support the study hypothesis that
beta-carotene reduces the incidence of lung cancer.

• In fact, they provide evidence that administration of

betacarotene may increase the risk of lung cancer
Prevented fraction
• Another important measure in intervention trials. It measures
the proportion of cases of disease that were prevented by the
intervention under study among those who received it.

• Prevented fraction (%) = 100 ✕ [rate (or risk) difference / rate

(or risk) in the unexposed]

• If the aim of the trial is to assess the value of a vaccine, this

measure is called vaccine efficacy.
Example (vaccine efficacy)
• If the risk of developing a particular disease among those who
were vaccinated was 40 per 100 000 and 70 per 100 000 among
those not vaccinated, the vaccine efficacy would be

• Vaccine efficacy (%) = 100 ✕ [(70 per 100 000 – 40 per 100
000)/70 per 100 000]
• = 43%

• Thus, 43% of cases could have been prevented among the

unvaccinated if they had been vaccinated.
Vaccine Efficacy = 1-- 0.049=95.1%
Survival Analysis
• If we are particularly interested in the distribution of time until
occurrence of the event of interest (e.g., time from treatment to
death or time from treatment to recurrence), as is the case in
many clinical trials, the most appropriate approach is survival
analysis
Bias
• Any systematic error in design, conduct or analysis of the study that results in a
mistaken estimate of an exposure’s effect on risk of disease.*

• In experimental study this may arise from three sources:

• Subject variation: Bias on part of participant, who may feel better if they knew
they are receiving a new form of treatment.
• Observer Bias: Increased positive or negative findings in interview or physical
examination if observer knows beforehand the particular procedure.
• Investigator Bias: Bias in evaluation. Investigator may subconsciously give a
favorable report.

• Randomization can’t guard against these bias, nor the size of sample.
• In order to reduce these problems, a technique known as “Blinding” is adopted.
Blinding
• Also known as “masking”.

• It prevents patients, investigators, even statisticians involved in the study

from knowing the treatment a subject is receiving. Following types:
• Single blinding: Participant does not know about the group he belongs
to.
• Double blinding : Here, neither the participant nor the investigator knows
about the group allocation and intervention done.
• Triple blinding: participant, investigator and the person analyzing the
data are all blind.

• Triple blinding is considered best, while double blinding is used most.

Study designs of controlled trials
Some study designs of controlled trials are:

1. Concurrent parallel study designs

2. Cross-over type of study designs
3. Factorial designs
Concurrent parallel group design
• Also c/d as Parallel RCT.

• Comparisons are made between two randomly assigned study & control
groups.

• Participants remain in the study group or control group for the whole
duration of investigation.

• It is the commonest study design and most easy for analysis

Concurrent parallel group design
 Cross over design
• Each participant serve as his own control

• Initially study starts as before with random assignments of participants to study and control group

• Study group receives the treatment under consideration while the control group receives some
alternate form of active treatment or placebo. Both groups are observed over time.

• Then, pts. In each group are taken off their treatment. This is done to allow for elimination of
medications from the body and for the possibility of any “carry over” effects. This is wash out
phase.

• After this, two groups are switched i.e., those receiving treatment under study are changed to the
control group and vice versa.
Cross over design
Factorial Design
• More efficient when there is interest in studying more than one intervention at a time.
• Given the cost and feasibility issues in designing clinical trials, 2 or more hypothesis could
be tested simultaneously in a factorial design

• Types:
• Two-by-two factorial design: subjects are first randomized to treatments A or B to address
one hypothesis and then within each group there is further randomization to treatments X or
Y to evaluate second hypothesis.
• Two-by-two-by-two factorial design: here each subgroup is further randomized into two
more groups P or Q .
• Example: Physicians Health Study (By Hennekens CH et al)
Factorial Design
Thanks