Module 4

Research design
 Research design: meaning
• Research design- the specification of methods and procedures for
acquiring the information needed. It is the overall operational framework
of the project that stipulates what information is to be collected from
which sources by what procedures.
• Three basic tenets has to be ensured:
• Convert research questions and the stated assumptions/ hypothesis into
operational variables that can be measured
• Specify the process that would be followed to complete the above task, as
efficiently and economically as possible
• Specify the “control mechanisms” that would be followed to ensure that the
effect of other variables that could impact the outcome of the study has been
controlled
 Frameworks

Exploratory
Research
Design Single
Research Cross-
design sectional
Conclusive Descriptive Multiple
Research Longitudinal
Design
Causal
 Exploratory research design
• Flexible approach- mostly qualitative
investigation Secondary
• Sample size is not strictly representative
resource analysis
• Unstructured interviews Comprehensive

Tools
• Carried out at the start of a less case method
explored topic with the purpose of
identifying possible relations/ assess the Expert opinion
potential of a research study survey
Focus group
discussions
Two-tiered Research design
• By conducting an exploratory study a researcher can achieve the following:
 Sometimes the purpose of an exploratory study is to check whether a
research topic has enough potential for a more in-depth structured
research study.
 Develop a comprehensive and focused research question
 Whether a conclusive research study is required at all or decisions can
be taken based on the exploratory research alone
 Sharpen the hypotheses and objectives of the study to be undertaken
 Improve the methodology and framework best suited to achieve the
formulated objectives
 Descriptive research design
• Structured and formal in nature
• Intended objectives of a descriptive research design might be:
• Provide detailed sketch or profile of the population being studied
Eg: If a Company wants to launch a new feature in a phone, they would like to first survey
the market and generate a research report on who are their potential customers- their
demographic profile, preferences, type of work their customers are engaged in, etc.
• Introduce a temporal component to the design
Eg: If the Company is interested to see how the preferences of their customers change with
age, they might be interested to capture information of brand loyal customers and what
they expect from the brand.
• To explore the simultaneous occurrence of certain phenomena and variables.
Eg: If the same Company would like to know the relationship between work from home
lifestyle and feature preferences among their customers/ the relationship between pay cuts
and investment behavior among their customers.
 Cross-sectional studies
• Carried out at a single moment in time and thus applicable for a specific period only.
• The sample is under consideration and investigation only for the time coordinate of the
study.
• Cross-sectional studies lack information on causality.
 Multiple cross-sectional studies

• Two or more samples of respondents collected at the same time or

different time points.
 Longitudinal studies

• Longitudinal studies are temporal in nature and follows the same

respondents over a period of time.
• Involves selection of a representative panel
• Repeated measurement of the panel of selected individuals over fixed time
intervals
• It is critical to take into account attrition, as the number of panel members
should be the same over time. Hence, if members drop-out due to some
reason it is essential to replace them thereby maintaining the structure of
the panel.
• These studies are also called time-series design/ panel design.
 Longitudinal studies (contd.)
True panel Omnibus panel
Uses same section of respondents Uses different groups/ replaces most of the past
respondents
Committed sample group It is not always easy to get a committed group of sample
Profile information is a one-time task, thus most of the Mortality and attrition are inevitable. The replaced new
survey time can be dedicated in asking useful research recruits might be vastly different from the old ones.
questions

 Challenge: Highly structured surveys might be responsible for consistent and structured behavior
different from reality.
 To deal with this research agencies take up some precautionary steps:
 Ensure people act normally and not display exaggerated/ artificial behavior
 While replacing members care is taken to ensure new recruits have similar characteristics with
older recruits
 After a certain period of time, the panel members are changed so that new perspectives can be
obtained.
 Necessary conditions for making
causal inferences
• Concomitant variation: The extent to which a cause X and effect Y occur/ vary
together. For one to be a cause of another the two variables need to be
strongly associated. However, strong association does not imply causal
relationship as other external factors might be at play too.

• Time order of occurrence of variables: The cause X must occur prior to or

simultaneously with effect Y.

• Absence of other possible causal factors: Ensure that all other external factors
that may influence the relationship between X and Y are controlled through
experimental designs.
 Concepts used in experiments
• Independent variables: Also known as explanatory variables. Researchers
often use these variables in their models to assess their association/ effect
on the outcome/ effect variable.

• Test units: Test units are those entities on which the treatment is applied.
Often these are the sampled individuals, organizations, geographic areas. It
is the unit of statistical analyses.

• Dependent variables: Also known as the outcome variable/ effect variable.

It is expressed as a function of all possible independent variables.
 Concepts used in experiments
(contd.)
• Experiment: In case of social science, business research statistical
manipulations and analyses carried out using available data is referred to
as an experiment.

• Extraneous variables: These are variable besides the mentioned

independent variables that may affect the outcome/ dependent variable.
Variables
Moderating Variables (MV)
• Variable that affects the direction and/or strength of the relation
between an independent or predictor variable and a dependent or
criterion variable

Drug / Corona Vaccine Recovery Rate

(IV) (DV)

Age

15
 Intervening variables (IVV)
• Is a temporal occurrence which follows the IV and precedes DV
• It is a conceptual mechanism through which the IV and MV might
affect the DV.
• This is a variable that probably does influence the relationship
between the independent and dependent variables, but it is one
that we do not control or manipulate.
Increase in
Job
Satisfaction
A flexi-time Work Schedule Individual’s Productivity
(IV) (DV)

16
Control Variable
• A control variable is a variable that affects the dependent variable
• When we "control a variable" we wish to balance its effect across
subjects and groups so that we can ignore it and just study the
relationship between the independent and the dependent variables.
• You control for a variable by holding it constant

Type of City
Advertising Media Sales
(IV) (DV)

17
 Goals of conducting an experiment
• A Researcher has two goals while conducting an experiment:
• To draw valid conclusions about the effect of independent variables on the
dependent variable.  internal validity
• To make generalizations about the results to a wider population.  external
validity
Validity in experimentation
examines whether the
observed effects on
Internal the dependent
variables is caused by
the independent
variables only
Validity examines whether
the results of an
External experiment can be
generalized beyond
the experimental
situations or not
Factors affecting internal validity

History Maturation Testing Instrumentation

Statistical regression Selection bias Test unit mortality

Factors affecting external validity

Environment Population

Time Mode of treatment

 Methods to control extraneous
variables
Randomization Matching Use of experimental designs
• Random assignment of test units and • Matching using background • Systematic method of eliminating effects
treatments to experimental groups. characteristics, demographic, socio- of extraneous variables.
• Requirement: large sample economic profile.
• Benefit: No selection bias • Requirement: Similar group composition
• Benefit: Control of behavourial patterns/
confounding variables like age, incomes,
work experience.

Statistical control
• Statistical methods like Analysis of
Covariance (ANCOVA) might be used to
deal with confounding effects and make
causal inferences.
 Classification of experimental
designs
Experimental
designs

Pre-experimental Quasi experimental True experimental Statistical

One-shot case Pre-test post-test Completely

Time series randomized
study control groups

One-group pre- Multiple time Post-test only control Randomized

test Post-test series groups blocks

Static group Solomon four group Latin Square

Factorial
Definition of symbols:
• 0t = Observation in time ‘t’ of experimental group
• X = Treatment
• R = random assignment of test units or groups to separate
treatments.
• 0c = Control group or CG
Pre-experimental design
• One-shot case study: Only one time observation is made on the respondents after the treatment
has been administered. This is not randomized. Eg: The sales manager might have selected the
group of sales persons he/she wanted to get the training and then evaluated their performance
post training.
• One-group pre-test Post-test : Observation on the test units are made twice, once before the
treatment and once after. This is usually not randomized.
• Static group: Observations are taken in the experimental and the control group. Experimental
group is the group where treatment (X) is given and control group is a group where treatment is
not given. The design appears as follows:
• Group 1 (experimental group)- X O1
• Group 2 (control group)- O2
 Quasi experimental
• Researchers can control when measurements are taken and on whom they are taken.
However, the lack complete control in the sense that they cannot control the
scheduling of the test and also lacks the ability to randomize test units’ exposure to
treatment.
• Time series: O1 O2 O3 O4 X O5 O6 O7 O8
Series of observations are made on a group of test unit without randomization. The timing of
treatment and the test units who are getting exposed to the treatment is not within the researcher’s
control. Due to multiple observations maturation, testing effect, instrumentation and statistical
regression can be ruled out. Randomizing this can deal with selection bias and strong incentives can
also reduce attrition.
• Multiple time series: Same as time series but multiple time series has a control group where no
treatment is given.
Group 1 (experimental group)- O1 O2 O3 O4 X O5 O6 O7 O8
Group 2 (control group)- O’1 O’2 O’3 O’4 O’5 O’6 O’7 O’8
True experimental
• Researchers are able to randomly assign test units and treatment to an experimental group.
• Pre-test post-test control groups: Also called before-after with control group.
Experimental group: R O1 X O2
Control group: R O3 O4
Let, A=O2- O1  treatment+ extraneous variables (interactive testing effect)
and B= O4 - O3  Extraneous variables
Therefore, A-B=(O2- O1 )-(O4 - O3 )  treatment effect
True experimental (Contd.)

• Post-test only control groups: Also called after-only with one control group
Experimental group: R X O1
Control group: R O2
Let, A=O2- O1
 treatment+ extraneous variables (interactive testing effect)-extraneous variables treatment effect
• Such experiments are largely used in research where the goal might be to evaluate the
effectiveness of a programme, say online education. Researchers could randomly choose to
educate 10 children of the same class through online mode and 10 through traditional mode and
evaluate their performance post treatment. Here, they don’t need to worry about the effect of
instrumentation or interactive test effects.
 True experimental (Contd.)
• Solomon four group: Also called four group six study design. Guarantees
maximum internal validity and useful for cause-effect relationship in studies
where survival is crucial.
Experimental group 1: R O1 X O2
Control group 1: R O3 O4
Experimental group 2: R X O5 No pre-test measurement
Control group 2: R O6
 Solomon 4 group- mathematics
Here, Experimental group 1:
A= O2 - O1= treatment effect+ extraneous effect without interactive
testing effect + interactive testing effect
= T+I+E
Control group 1:
B= O4 – O3= Extraneous effects= E
X=A-B= T+I (treatment effects along with interaction effects)
Experiment Group 1 and 2:
C=O2 – O5 = (treatment effect+ interaction effects+ extraneous effect) –
(treatment effect+ extraneous effect) = interaction effects
= (T+I+E)-(T+E)= I (Only interaction effects)
D= X-C= (T+I)-I= T (Pure treatment effects)
Statistical designs
• Advantages of a statistical design:
• The effect of more than one level of independent variable on the dependent
variable can be manipulated.
• The effect of multiple independent variables can be examined.
• The effects of specific extraneous variables can be controlled.
Statistical Design: CRD (contd.)
• Completely randomized design (CRD):
• Used when effect of one independent variable on dependent variable is being examined.
• Independent variable should have categories- each category is a treatment.
• Assumption: No difference in test units  no extraneous variables can influence the
outcomes
• One-way ANOVA (Analysis of Variance)
• Example: effects of three categories of price on sales

Price
Low
Sales
Medium
High
Statistical Design: RBD (contd.)
• Randomized block designs (RBD):
• Main limitation of CRD was that all
Store: Medium
extraneous variables were assumed
to be constant over treatment Price:
• Low
groups, which may not be true. • Medium
• In RBD it is possible to separate out Store: Small
• High
Store: Large
the influence of one extraneous Price: Price:
variable on the dependent variable. • Low • Low
• Eg. All stores taken as test units • Medium • Medium
• High • High
may not be of the same size, and
might be small, medium or large.
This blocking will be done prior to
the experimentation. Now, we
analyze the treatment effect Sales
(categories of price) on the
outcome (sales) by checking their
effect in homogenous blocks (store
sizes).
Statistical Design: LSD(contd.)

• Latin Square Design:

• LSD is applied when the researcher wants to separate out the
influence of two extraneous variables.
• Necessary condition: No. of categories of the two extraneous
variables=No. of categories of treatment variable.
• Eg: Suppose along with store size we are also interested to explore
type of packaging as an extraneous variable.
 Statistical design: LSD (contd.)

Store Size Packaging

I II III

1. Small Price 1 Price 2 Price 3

2. Medium Price 2 Price 3 Price 1

3. Large Price 3 Price 1 Price 2

• Treatment should be assigned in such a manner that one treatment occurs once and only one in each row and
column
 Statistical Design: FD (contd.)

• Factorial Design:
• Measure the effect of two or more independent variables at various levels.
• Interaction is suppose to take place when simultaneous effect of two variables is
different from a simple addition of their individual effects (say an individual might
love mangoes and also love ice-creams but might not like mango flavoured ice-
cream!)
 Statistical Design: FD (contd.)
A two-factor design looks like:
Price Store
Small (B1) Big (B2) Low price in small
store
1. Low (A1) A1 X B1 A1 X B2
3X2= 6 cells
2. Medium (A2) A2 X B1 A2 X B2
3. High (A3) A3 X B1 A3 X B3 High price in big
store

• Respondent receive a specified treatment combination.

• Advantages:
• Possible to measure main effects and interaction effects of two or more independent variables
• Saves time and effort as all observations are employed to study the effects of each factor
• Has wider acceptance as the design acknowledges the different combinations of factors
• Challenge: when the number of categories and factors increase
No-Sampling Error
 This is an error occurring for reasons other than sampling.
 Example – If the questionnaire is poorly designed with many ambiguities, the
response will be poor. Non-sampling error consists of non-response errors and
response errors.
Non-response Error
 This happens when chosen respondents do not respond
 This may be due to
 Non availability of respondent
 Refusal to answer
 This will cause the resulting sample to be of different size compared to original
sample.
Response Error
 This occurs due to any of the eventualities
A. Mis-recording
B. Inaccurate answers
C. Wrong analysis errors made by researcher include
i. Surrogate information
ii. Measurement
iii. Population Definition
iv. Sampling Frame
v. Data Analysis Error