NUT1002

Sampling &
Sources of bias

Zoe Davidson
 The Research Process

Step 1: Design
the research
question

Step 2: Design
Step 5: Report the research
findings protocol
Study Design
Study Sample

Step 4: Analyse
Data Step 3: Collect
Data
Interpret findings

Portney and Watkins (2009) Foundations of Clinical Research, p24

Objectives

 To explain observation bias and the methods for

protecting against this form of bias.

 To define sampling bias and distinguish between

conscious and unconscious bias.

 To describe probability and nonprobability sampling

techniques and explain how these influence
generalisability of findings

3
Bias

 A distortion of the results of the study in such a way that

is systematically different from the population of interest

– Systematic bias

– Random error

4
Bias

Sampling Bias
 Bias that is introduced due to how the participants are
selected to be in the study

Observation Bias
 Bias that is introduced due to the way information about
study variables is collected

5
Sampling Bias: Who will be studied?

 Goal is to be able to make generalisations beyond

individuals studied

 Populations
– the larger group to which the research results are generalised

 Samples
– a subgroup of the population that will participate in the study

6
Sampling Bias: Who will be studied?

 A good sample reflects the relevant characteristics and

variations of the population in same proportion as they
exist in the population

 Bias/error can be introduced due to the way a population

is sampled

7
Sampling bias

 Sampling bias occurs when individuals selected for a

sample over represent or under represent certain
population attributes that are related to the characteristic
being studied.

 Eg. study of car ownership in central London is not

representative of the UK

8
Sampling bias

Conscious Unconscious

9
Selection bias

 Bias introduced by initial differences between study

groups when these difference are not random

 Eg. Allocating subjects to receive intervention / placebo

based on certain characteristics. Subjects are not
randomly allocated.

10
Observation Bias

 Misclassification bias
– Subjects are misclassified as a case or control
– Cohort and case-control studies

 Interviewing bias
– Individual collecting data elicits, records or interprets
information differentially from study participants based
on their allocation (intervention/placebo group or
case/control)

11
Observation Bias

 Recall bias
– Subjects who have experienced the disorder/condition
may recall their exposure history differently from those
who are not affected

 Reporting bias
– When a case or person receiving the intervention
emphasizes the importance of an aspect he/she
believes to be important.

12
Reducing bias

Sampling Bias
 Bias can be minimised by sampling procedures

Observation Bias
 Blinding
 Corroboration of multiple information sources
 Use standard measures or standardised questionnaires

13
Sampling
Consider this PICO

In undergraduate students at Monash completing their first

degree, what is the effect of a structured high intensity
exercise program (5 x 1 hour session/week) for the duration
of semester compared to standard unstructured activity (30
mins/day) on their weighted average mark for semester 1?

15
Steps in choosing a sample

1. Identify the target population and the accessible

population
– inclusion and exclusion criteria

2. Select the study sample

– Sampling techniques

16
 1. Target population

2. Accessible population

3. Sample
S
Defining the target population

 Inclusion criteria
– The primary traits of the target and accessible population

 Exclusion criteria
– Indicates factors that would preclude someone from being a
subject.

18
In undergraduate students at Monash completing their first
degree, what is the effect of a structured high intensity
exercise program (5 x 1 hour session/week) for the duration
of semester compared to standard unstructured activity (30
mins/day) on their weighted average mark for semester 1?

19
Defining the target population

 Inclusion  Exclusion

20
Choosing the sample

 Next… sampling techniques

 From accessible population to study sample

21
Sampling techniques

 Probability samples
– Created through a process of random selection
– Every unit in the population has an equal chance or
probability of being selected.
– The probability of being selected is known
– Sampling error* is random

*Sampling error = variation between the samples

characteristics and the population characteristic

22
Sampling techniques

 Non probability samples

– From non random methods
– The probability of selection is unknown

23
Sampling techniques

Probability sampling Non probability sampling

 Random selection  Non random selection
 Equal chance of selection  Chance of selection is
 Chance of selection is unknown
known  Sample unlikely to be
 Sample is considered representative of
representative population

24
Sampling techniques

Probability sampling Non probability sampling

Examples Examples
 Simple random sampling  Convenience sampling
 Systematic sampling  Quota sampling
 Stratified random  Purposive sampling
sampling  Snowball sampling
 Cluster sampling

25
Probability Sampling

 Simple random sampling

– Most basic method of probability sampling
– No one member has any more of a chance of being
selected
– Drawn from the accessible population
 Listing of persons such as membership directories, enrolled
students
 List of institutions such as accredited hospitals

eg. Name out of the hat, random number lists (excel),

table of random numbers

26
Probability Sampling

 Systematic sampling
– Ordered list
– Determine the sampling interval
 Divide the total number in the accessible population by the
number to be selected
 Eg. Wish to select 10 out of 100 participants = 10/100 = 10
– Select every 10th person
– Starting point is determined at random

27
Probability Sampling

 Stratified random sampling

– Divide the sample on relevant characteristics
– Conduct simple random of systematic sampling in the
strata

– Proportional stratified sampling

 Proportions in the sample reflect the proportions in the target
population

28
 Accessible population
N = 1000 Monash Clayton

Med, Nursing Engineering Arts Law

Health Sciences N = 300 N = 200 N = 200
N = 300

Random Selection Random Selection Random Selection Random Selection

Med, Nursing Engineering Arts Law

Health Sciences N = 30 N = 20 N = 20
N = 30

Study Sample
n = 100
Probability Sampling

 Cluster sampling
– Multistage
– Successive random sampling of a series of units in the population
– When difficult to obtain a complete listing of the population

30
Randomly select 5
faculties

In each faculty, randomly

select 5 courses

In each course, randomly

select one lecture
10 regions in VIC

10 GP clinics

10 patients
Non Probability Sampling

 Convenience sampling
– Subjects are chosen on the basis of availability
– Consecutive sampling
 Recruiting all patients who meet the inclusion and exclusion
criteria as they become available
– Sample achieved through volunteers
 Advertising, polls in magazines

33
Non Probability Sampling

 Quota sampling
– Similar structure to proportional stratified sample
– Instead of random sampling, recruit until you have
achieved the desired proportions

34
 Accessible population
N = 1000 Monash Clayton

Med, Nursing Engineering Arts Law

Health Sciences N = 300 N = 200 N = 200
N = 300

Med, Nursing Engineering Arts Law

Health Sciences N = 30 N = 20 N = 20
N = 30

Study Sample
n = 100
Non Probability Sampling

 Purposive sampling
– Researcher handpicks subjects on basis of specific
criteria
– Similar to convenience however researcher will
consider more than just availability

36
Non Probability Sampling

 Snowball sampling
– Sensitive topics, rare trait, personal networks and
social relationships
– Several stages
– Few subjects identified and participate
 Asked to identify others who have requisite characteristics
– Chain referral or snowballing continued until adequate
sample is obtained

37
Sample size?

 How many people do you need?

– Important consideration
– Directly relates to the statistical power
– Power: ability to find a statistical difference when they
exist
– Generally
 Small sample = less power
 Large sample = more power

38
Let’s Practise!

39
Question: In employees of Bank Richalot who work in Costalot
Towers 1 & 2 in Melbourne CBD, what is the effect of a lunch time
exercise program on productivity between 2-5pm?

Study design: One armed intervention (pre post measures)

Sample size required: 500 employees (3000 employees in total)

Inclusion criteria:
 Full-time employees
 Able to attend the lunch time exercise program at least 3 d/wk

Exclusion criteria
 Injuries that prevent participation in the exercise program
Activity:
 Random or non-random sampling?

 Describe this sampling strategy in the context of the

question. Plan the steps to conduct this type of strategy.

 What are the pros and cons of the sampling strategy?

 Is this sampling technique appropriate for the research

question? Why/why not? Would you suggest another type
of sampling technique?

 Demonstrate your sampling strategy (another group will

act as your participants) and present your answers to the
class
Objectives

 To define sampling bias and distinguish between

conscious and unconscious bias.

 To explain observation bias and the methods for

protecting against this form of bias.

 To describe probability and nonprobability sampling

techniques and explain how these influence
generalisability of findings

42