Subject Name: Research Methodology

Subject code: MGT507

Credit hours: 4

Research Design

Copyright © 2016-2018 Lincoln University College 1

• Research Design:
• Experimental:

• Experimental research is when a researcher is able to manipulate the predictor

variable and subjects to identify a cause-and-effect relationship. This typically
requires the research to be conducted in a lab, with one group being placed in
an experimental group, or the ones being manipulated, while the other is placed
in a placebo group, or inert condition or non-manipulated group.

Copyright © 2016-2018 Lincoln University College 2

• Experimental Research is often used where:
• There is time priority in a causal relationship (cause precedes effect)
• There is consistency in a causal relationship (a cause will always lead to the same effect)
• The magnitude of the correlation is great.

• Aims of Experimental Research

• Experiments are conducted to be able to predict phenomenon. Typically, an
experiment is constructed to be able to explain some kind of causation.
Experimental research is important to society - it helps us to improve our
everyday lives.

Copyright © 2016-2018 Lincoln University College 3

• Typical Designs and Features in Experimental Design

• Pre-test-Post-test Design
• Check whether the groups are different before the manipulation starts and the effect of the
manipulation. Pre-tests sometimes influence the effect.
• 
• Control Group
• Control groups are designed to measure research bias and measurement effects, such as the
Hawthorne Effect or the Placebo Effect. A control group is a group not receiving the same
manipulation as the experimental group. Experiments frequently have 2 conditions, but rarely
more than 3 conditions at the same time.

Copyright © 2016-2018 Lincoln University College 4

• Randomized Controlled Trials
• Randomized Sampling, comparison between an Experimental Group and a
Control Group and strict control/randomization of all other variables
• 
• Solomon Four-Group Design
• With two control groups and two experimental groups. Half the groups have a
pre-test and half do not have a pre-test. This to test both the effect itself and
the effect of the pre-test.

Copyright © 2016-2018 Lincoln University College 5

• Between Subjects Design
• Grouping Participants to Different Conditions
• 
• Within Subject Design
• Participants Take Part in the Different Conditions - See also: Repeated Measures
Design

• Counterbalanced Measures Design

• Testing the effect of the order of treatments when no control group is
available/ethical

Copyright © 2016-2018 Lincoln University College 6

• Matched Subjects Design
• Matching Participants to Create Similar Experimental- and Control-Groups

• Double-Blind Experiment
• Neither the researcher, nor the participants, know which the control group is. The results can
be affected if the researcher or participants know this.

• Bayesian Probability
• Using Bayesian probability to "interact" with participants is a more "advanced" experimental
design. It can be used for settings where there are many variables which are hard to isolate.
The researcher starts with a set of initial beliefs, and tries to adjust them to how participants
have responded
Copyright © 2016-2018 Lincoln University College 7
• Pilot Study
• It may be wise to first conduct a pilot-study or two before you do the real experiment. This
ensures that the experiment measures what it should, and that everything is set up right.

• Minor errors, which could potentially destroy the experiment, are often found during this
process. With a pilot study, you can get information about errors and problems, and improve
the design, before putting a lot of effort into the real experiment.

• If the experiments involve humans, a common strategy is to first have a pilot study with
someone involved in the research, but not too closely, and then arrange a pilot with a person
who resembles the subject(s). Those two different pilots are likely to give the researcher good
information about any problems in the experiment.

Copyright © 2016-2018 Lincoln University College 8

• Conducting the Experiment
• An experiment is typically carried out by manipulating a variable, called the
independent variable, affecting the experimental group. The effect that the
researcher is interested in, the dependent variable(s), is measured.

• Identifying and controlling non-experimental factors which the researcher does

not want to influence the effects, is crucial to drawing a valid conclusion. This is
often done by controlling variables, if possible, or randomizing variables to
minimize effects that can be traced back to third variables. Researchers only
want to measure the effect of the independent variable(s) when conducting an
experiment, allowing them to conclude that this was the reason for the effect.

Copyright © 2016-2018 Lincoln University College 9

• Analysis and Conclusions
• In quantitative research, the amount of data measured can be enormous. Data
not prepared to be analysed is called "raw data". The raw data is often
summarized as something called "output data", which typically consists of one
line per subject (or item). A cell of the output data is, for example, an average of
an effect in many trials for a subject. The output data is used for statistical
analysis, e.g. significance tests, to see if there really an effect is.

• The aim of an analysis is to draw a conclusion, together with other observations.

The researcher might generalize the results to a wider phenomenon, if there is
no indication of confounding variables "polluting" the results.

Copyright © 2016-2018 Lincoln University College 10

• Non-experimental

• Nonexperimental research is research that lacks the manipulation of an

independent variable, random assignment of participants to conditions or
orders of conditions, or both.

Copyright © 2016-2018 Lincoln University College 11

• Non-experimental research is the label given to a study when a researcher
cannot control, manipulate or alter the predictor variable or subjects, but
instead, relies on interpretation, observation or interactions to come to a
conclusion. Typically, this means the non-experimental researcher must rely on
correlations, surveys or case studies, and cannot demonstrate a true cause-and-
effect relationship. Non-experimental research tends to have a high level of
external validity, meaning it can be generalized to a larger population.

Copyright © 2016-2018 Lincoln University College 12

• There appears to be only disadvantages to non-experimental research. It cannot find cause-
and effect relationships, cannot manipulate predictor variables and the methods of study are
often correlation or case studies. There are clear cut disadvantages to non-experimental
designs. However, non-experimental research does have at least some advantages over
experimental design. A non-experimental study picks up the slack from an experimental
design. As discussed earlier, to study the effects of gender, you have to be able to manipulate
a person's gender. Other examples of non-experimental research include predictor variables
like:
• Age
• Ethnicity
• Prison sentences
• Bereavement
• Current opinions

Copyright © 2016-2018 Lincoln University College 13

• There are many ways in which preferring non-experimental research can be the
case.
• The research question or hypothesis can be about a single variable rather than a statistical
relationship between two variables (e.g., How accurate are people’s first impressions?).
• The research question can be about a non-causal statistical relationship between variables
(e.g., Is there a correlation between verbal intelligence and mathematical intelligence?).
• The research question can be about a causal relationship, but the independent variable
cannot be manipulated, or participants cannot be randomly assigned to conditions or orders
of conditions (e.g., Does damage to a person’s hippocampus impair the formation of long-
term memory traces?).
• The research question can be broad and exploratory, or it can be about what it is like to have
a particular experience (e.g., what is it like to be a working mother diagnosed with
depression?).

Copyright © 2016-2018 Lincoln University College 14

• When to Use Nonexperimental Research

• Experimental research is appropriate when the researcher has a specific research

question or hypothesis about a causal relationship between two variables—and it is
possible, feasible, and ethical to manipulate the independent variable and randomly
assign participants to conditions or to orders of conditions. It stands to reason, therefore,
that nonexperimental research is appropriate—even necessary—when these conditions
are not met. There are many ways in which preferring nonexperimental research can be
the case.

• The research question or hypothesis can be about a single variable rather than a statistical
relationship between two variables (e.g., How accurate are people’s first impressions?).

Copyright © 2016-2018 Lincoln University College 15

• The research question can be about a non-causal statistical relationship
between variables (e.g., Is there a correlation between verbal intelligence and
mathematical intelligence?).

• The research question can be about a causal relationship, but the independent
variable cannot be manipulated, or participants cannot be randomly assigned to
conditions or orders of conditions (e.g., Does damage to a person’s
hippocampus impair the formation of long-term memory traces?).

• The research question can be broad and exploratory, or it can be about what it is
like to have a particular experience (e.g., What is it like to be a working mother
diagnosed with depression?).
Copyright © 2016-2018 Lincoln University College 16
• Types of Nonexperimental Research

• Nonexperimental research falls into three broad categories: single-variable

research, correlational and quasi-experimental research, and qualitative
research.

• Research can be nonexperimental because it focuses on a single variable rather

than a statistical relationship between two variables. Although there is no
widely shared term for this kind of research, we will call it single-variable
research.

Copyright © 2016-2018 Lincoln University College 17

• Research can also be nonexperimental because it focuses on a statistical
relationship between two variables but does not include the manipulation of an
independent variable, random assignment of participants to conditions or
orders of conditions, or both. This kind of research takes two basic forms:
correlational research and quasi-experimental research. In correlational
research, the researcher measures the two variables of interest with little or no
attempt to control extraneous variables and then assesses the relationship
between them.

Copyright © 2016-2018 Lincoln University College 18

• In quasi-experimental research, the researcher manipulates an independent
variable but does not randomly assign participants to conditions or orders of
conditions. For example, a researcher might start an antibullying program (a
kind of treatment) at one school and compare the incidence of bullying at that
school with the incidence at a similar school that has no antibullying program.

• The final way in which research can be nonexperimental is that it can be

qualitative. The types of research we have discussed so far are all quantitative,
referring to the fact that the data consist of numbers that are analyzed using
statistical techniques. In qualitative research, the data are usually nonnumerical
and are analyzed using nonstatistical techniques.

Copyright © 2016-2018 Lincoln University College 19

• Research Field:

• Research field or research area is the area of focus on which a researcher is

devoting his time, knowledge and other resources to prove or validate a
statement, i.e. the research topic. This field of research can be mathematics or
philosophy or biology or chemistry. It can be anything depending upon the skill
and knowledge of the researcher.

Copyright © 2016-2018 Lincoln University College 20

• How to choose a research field/area:
• Read scientific literature:
• Published scientific literature gives a good idea about the research field and the
big unanswered questions that are left to be studied. Scientific reviews on the
topic are often useful to understand the big discoveries in the field and the
anticipated future studies that will provide more information. It is also
important to know if the research area has sufficient unanswered questions that
will be interesting to funding agencies. Try to understand if the field has long-
term potential. Scientists often work on certain research areas for decades and
so thinking ahead about hypothetical questions and probable answers is one key
to success.

Copyright © 2016-2018 Lincoln University College 21

• Attend conferences/seminars:
• Attending both large and small meetings help us connect with our peers and
have insightful discussions. Meetings also have poster sessions on various topics
that may be useful to learn about the different research areas out there. Such
meetings are also a good place to learn about technical details or new
experimental strategies, which are often important when forging into a different
field.

Copyright © 2016-2018 Lincoln University College 22

• Brainstorm ideas with peers:
• When looking for a research laboratory it is important to find something that
interests you. Working on an interesting question will help you go the extra mile
and aid in making significant discoveries. Talk to your peers about their
experiences and the pros/cons in their research field. Peers can also help review
research grants and their experience and perspectives may provide useful
feedback.

Copyright © 2016-2018 Lincoln University College 23

• Define focused questions in the research area:
• Research areas can be very broad. It is easy to digress into multiple directions
without focus. Before diving into the research, decide on a few hypotheses and
preliminary experiments. Having more than one hypothesis will be important in
case the primary hypothesis does not hold. Once experiments work and the
project progresses, remember to stay focused. As part of your scientific growth,
learn to think of tangential experiments that may be useful projects for other
members of the laboratory.

Copyright © 2016-2018 Lincoln University College 24

• Ensure the research is fundable:
• To continue doing research it is essential to have funding. Before delving into the
research define the significance of the proposed research. It is always useful if
discoveries can, in the future, cure or treat diseases. Significance also helps us
explain our research to non-scientists and family, so that they can relate to the
research and understand what we study. Public outreach will help get more
funding and aid in conducting more research.

Copyright © 2016-2018 Lincoln University College 25

• Survey:
• If you've ever been sitting at a train station, a particular lecturer's classroom, or in a public
area and a person with a stack of papers in his hands comes up to you out of the blue and
asks if you have a few minutes to talk, then you have likely been asked to take part in a
survey.
• There are a lot of ways to conduct research and collect information, but one way that makes
it really easy is by doing a survey. A survey is defined as a brief interview or discussion with
individuals about a specific topic. The term survey is, unfortunately, a little vague, so we need
to define it better. The term survey is often used to mean 'collect information.' For instance,
you may imagine a researcher or a television scientist saying, 'We need to do a survey!' (I
know, riveting television).
• So, besides our definition above, survey also means to collect information. We have our first
definition of a brief interview, and we have a second definition of collecting data. There is a
third definition for survey. This third definition of survey is a specific type of survey research
Copyright © 2016-2018 Lincoln University College 26
• Here are the three specific techniques of survey research:
 
• Questionnaires - a series of written questions a participant answers. This method gathers
responses to questions that are essay or agree/neutral/disagree style.

• Interviews - questions posed to an individual to obtain information about him or her. This
type of survey is like a job interview, with one person asking another a load of questions.

• Surveys - brief interviews and discussions with individuals about a specific topic. Yes,
survey is also a specific type of survey, to make things even more confusing. A survey is a
quick interview, with the surveyor asking only a few questions.

Copyright © 2016-2018 Lincoln University College 27

• Questionnaire Design
• The two most common types of survey questions are closed-ended questions
and open-ended questions.

• Open-Ended Questions
• Survey respondents are asked to answer each question in their own words.
• Responses are usually categorized into a smaller list of responses that can be
counted by the study team for statistical analysis.

Copyright © 2016-2018 Lincoln University College 28

• Closed-Ended Questions
• The respondents are given a list of predetermined responses from which to choose their
answer.
• The list of responses should include every possible response and the meaning of the
responses should not overlap.
• An example of a close-ended survey question would be, "Please rate how strongly you agree
or disagree with the following statement: 'I feel good about my work on the job.' Do you
strongly agree, somewhat agree, neither agree nor disagree, somewhat disagree, or strongly
disagree?"
• A Likert scale, which is used in the example above, is a commonly used set of responses for
closed-ended questions.
• Closed-ended questions are usually preferred in survey research because of the ease of
counting the frequency of each response.

Copyright © 2016-2018 Lincoln University College 29

• Considerations for Designing a Questionnaire
• It is important to consider the order in which questions are presented. Sensitive questions,
such as questions about income, drug use, or sexual activity, should be put at the end of the
survey. This allows the researcher to establish trust before asking questions that might
embarrass respondents. Researchers also recommend putting routine questions, such as age,
gender, and marital status, at the end of the questionnaire.
• Double-barrelled questions, which ask two questions in one, should never be used in a
survey. An example of a double-barrelled question is, "Please rate how strongly you agree or
disagree with the following statement: 'I feel good about my work on the job, and I get along
well with others at work.'" This question is problematic because survey respondents are
asked to give one response for two questions.
• Researchers should avoid using emotionally loaded or biased words and phrases.

Copyright © 2016-2018 Lincoln University College 30

• Survey Administration
• Surveys can be administered in three ways:
• Through the mail
• Advantage: Low cost
• Disadvantage: Low response rate
• By telephone
• Advantages: Higher response rates; responses can be gathered more quickly
• Disadvantage: More expensive than mail surveys
• Face-to-face
• Advantages: Highest response rates; better suited to collecting complex information
• Disadvantage: Very expensive

Copyright © 2016-2018 Lincoln University College 31

• Sampling Procedures
• One of the primary strengths of sampling is that accurate estimates of a
population's characteristics can be obtained by surveying a small proportion of
the population. Four sampling techniques are described here:
• Simple Random Sampling
• Simple random sampling is the most basic form of sampling.
• Every member of the population has an equal chance of being selected.
• This sampling process is similar to a lottery: the entire population of interest could be
selected for the survey, but only a few are chosen at random.
• Researchers often use random-digit dialing to perform simple random sampling. In this
procedure, telephone numbers are generated by a computer at random and called to
identify individuals to participate in the survey.

Copyright © 2016-2018 Lincoln University College 32

• Cluster Sampling

• Cluster sampling is generally used when it is geographically impossible to

undertake a simple random sample.
• Cluster sampling requires that adjustments be made in statistical analyses.
• For example, in a face-to-face interview, it is difficult and expensive to survey
households across the nation. Instead, researchers will randomly select
geographic areas (for example, counties), then randomly select households
within these areas. This creates a cluster sample, in which respondents are
clustered together geographically.

Copyright © 2016-2018 Lincoln University College 33

• Stratified Sampling

• Stratified samples are used when a researcher wants to ensure that there are
enough respondents with certain characteristics in the sample.
• The researcher first identifies the people in the population who have the desired
characteristics, then randomly selects a sample of them.
• Stratified sampling requires that adjustments be made in statistical analyses.
• For example, a researcher may want to compare survey responses of African-
Americans and Caucasians. To ensure that there are enough African-Americans
in the survey, the researcher will first identify the African-Americans in the
population and then randomly select a sample of African-Americans.

Copyright © 2016-2018 Lincoln University College 34

• Non-random Sampling

• Common non-random sampling techniques include convenience sampling and

snowball sampling.
• Non-random samples cannot be generalized to the population of interest.
Consequently, it is problematic to make inferences about the population.
• In survey research, random, cluster, or stratified samples are preferable.

Copyright © 2016-2018 Lincoln University College 35

• Measurement Error

• Measurement error is the difference between the target population's

characteristics and the measurement of these characteristics in a survey. There
are two types of measurement error: systematic error and random error.

Copyright © 2016-2018 Lincoln University College 36

• Systematic Error

• Systematic error is more serious than random error.

• Occurs when the survey responses are systematically different from the target
population responses.

• For example, if a researcher only surveyed individual who answered their phone
between 9 and 5, Monday through Friday, the survey results would be biased
toward individuals who are unemployed.

Copyright © 2016-2018 Lincoln University College 37

• Sources of bias include

• Non-observational error -- Individuals in the target population are systematically excluded

from the sample, such as in the example above.

• Observational error -- When respondents systematically answer surveys question

incorrectly. For example, surveys that ask respondents how much they weigh will
probably underestimate the population's weight because respondents are likely to
underreport their weight.

Copyright © 2016-2018 Lincoln University College 38

• Random Error

• Random error is an expected part of survey research, and statistical techniques

are designed to account for this sort of measurement error.

• Occurs because of natural and uncontrollable variations in the survey process,

i.e., the mood of the respondent.

Copyright © 2016-2018 Lincoln University College 39

• Ethics of Survey Research

• Informed Consent
• Respondents should give informed consent before participating in a survey. In
order for respondents to give informed consent,
• The researcher must inform the respondents of the study's purpose, content, duration,
and potential risks and benefits.
• The researcher must inform the respondents that they do not have to answer all the
survey questions.
• The researcher must inform the respondents that they can stop participating in the study
at any point.

Copyright © 2016-2018 Lincoln University College 40

• Confidentiality and Anonymity
• It is absolutely imperative that researchers keep respondents' identities
confidential. To ensure confidentiality, researchers should not link respondents'
identifiers to their survey responses when using data. Common identifiers
include names, social security numbers, addresses, and telephone numbers.
• 
• Anonymity
• Anonymity is an even stronger safeguard of respondent privacy. If a researcher
assures anonymity, it means that the researcher is unable to link respondents'
names to their surveys.

Copyright © 2016-2018 Lincoln University College 41

• Advantages and Disadvantages of Survey Research

• Advantages
• Sample surveys are a cost-effective and efficient means of gathering information
about a population.
• Survey sampling makes it possible to accurately estimate the characteristics of a
target population without interviewing all members of the population.
• Survey sampling is particularly useful when the population of interest is very
large or dispersed across a large geographic area.

Copyright © 2016-2018 Lincoln University College 42

• Disadvantages

• Surveys do not allow researchers to develop an intimate understanding of

individual circumstances or the local culture that may be the root cause of
respondent behaviour.

• Respondents often will not share sensitive information in the survey format.

• A growing problem in survey research is the widespread decline in response

rates.

Copyright © 2016-2018 Lincoln University College 43