Answer 5 – Methods of data collection (Survey Research)

Research is defined as a systematic gathering of data and information and its analysis for
advancement of knowledge in any subject. Whereas Data collection is a systematic process of
gathering observations or measurements. While techniques and goals may vary per field, the
general data collection methods used in the process are essentially the same. In other words,
there are specific standards that need to be strictly followed and implemented to make sure that
data is collected accurately. If the appropriate procedures are not given importance, a variety of
problems might arise and impact the study or research being conducted. The most common risk
is the inability to identify answers and draw correct conclusions for the study, as well as failure
to validate if the results are correct. Whether you are performing research for business,
governmental or academic purposes, data collection allows you to gain first-hand knowledge and
original insights into your research problem. They are primarily scientific, planned, and value-
neutral in nature. A few examples are theoretical methods, observations, experimental
investigations, numerical techniques, and statistical methodologies. Methods of research assist in
collecting samples and data and in determining a solution to problems.

Three things to consider before collecting data:

1. The aim of the research
2. The type of data that you will collect
3. The methods and procedures you will use to collect, store, and process the data

Methods for collecting data –

a. Observation method - Scientific observation is made under precisely defined conditions,
in a systematic and objective manner, and with careful record keeping. The primary goal
of observational methods is to describe behavior. Researchers / Scientists strive to
describe behavior fully and as accurately as possible.
b. Interview method - An interview is either structured or unstructured, depending upon
whether a formal questionnaire has been formulated and the questions asked are in a
prearranged order or not. There are different types of interviews are used in the research
data collection.
 c. Survey research - Survey research is designed to deal more directly with the nature of
people’s thoughts, opinions, and feelings. Surveys also are used to meet the more
pragmatic needs of the media, political candidates, public health officials, professional
organizations, and advertising and marketing directors. The scope and purpose of surveys
can be limited and specific, or they can be more global.
d. Questionnaire - Most survey research relies on the use of questionnaires to measure
variables. Demographic variables describe the characteristics of people who are surveyed.
The accuracy and precision of questionnaires requires expertise and care in their
construction. Self-report scales are used to assess people’s preferences or attitudes.
Here, we will focus on Survey Research.
Survey Research
Surveys typically are conducted with samples of people. The primary instrument of survey
research is the questionnaire.
Uses –
a. Survey research is used to assess people’s thoughts, opinions, and feelings.
b. Surveys can be specific and limited in scope or more global in their goals.
c. The best way to determine whether results of a survey are biased is to examine the survey
procedures and analyses.
An important characteristic of survey research is it involves selecting a sample (or samples) and
using a predetermined set of questions.
The sampling in survey research should be in a way where there is careful selection of a survey
sample which allows researchers to generalize findings from the sample to the population. The
sample is chosen from the sampling frame, or list of all members of the population of interest.
Researchers seek to describe the larger population from which the sample was drawn. The ability
to generalize from a sample to the population depends critically on the representativeness of the
sample. Two approaches to selecting a survey sample are nonprobability sampling and
probability sampling. Nonprobability sampling (such as convenience sampling) does not
guarantee that every element in the population has an equal chance of being included in the
sample. Probability sampling is the method of choice for obtaining a representative sample. In
simple random sampling, each element of the population has an equal chance of being included
in the sample; in stratified random sampling, the population is divided into subpopulations
(strata), and random samples are drawn from the strata.

Survey Methods –
Selecting the sample is only one of several important decisions to make when doing survey
research. You also need to decide how you will obtain information from the respondents. There
are four general methods: mail surveys, personal interviews, telephone interviews, and Internet
surveys.

Mail surveys –
Mail surveys are used to distribute self-administered questionnaires that respondents fill out on
their own.

Advantages, they usually can be completed relatively quickly. Because they are self-
administered, they also avoid the problems due to interviewer bias. Mail surveys are the best
for dealing with highly personal or embarrassing topics, especially when anonymity of
respondents is preserved.

Disadvantages, the questionnaire used in the survey must be completely self-explanatory. The
researcher has little control over the order in which the respondent answers the questions. A low
response rate that can result in response rate bias (Response rate refers to the percentage of
people who complete the survey.).

The typical return rate for mail surveys is only around 30%. Return rates generally will be higher
when
—the questionnaire has a “personal touch” (e.g., respondents are addressed by name and not
simply “resident” or “student”);
—responding requires minimal effort from the respondent;
—the topic of the survey is of intrinsic interest to the respondent;
—the respondent identifies in some way with the organization or researcher sponsoring the
survey.
Personal Interviews –
When personal interviews are used to collect survey data, respondents are usually contacted in
their homes or in a shopping mall, and trained interviewers administer the questionnaire. During
an interview the respondent can obtain clarification when questions are unclear, and the trained
interviewer can follow up incomplete or ambiguous answers to open ended questions. The
interviewer controls the order of questions and can ensure that all respondents complete the
questions in the same order. The response rate to personal interviews has been higher than that
for mail surveys.

Advantages, the personal interview allows greater flexibility in asking questions than does the
mail survey. It also allows the respondent to obtain clarification when questions are unclear as it
is face to face conversation.

Disadvantages, increasing fear of urban crime and an increasing number of households with no
one home during the day have reduced the attractiveness of using personal interviews in the
home. The cost is also a significant disadvantage. The potential for interviewer bias is another
critical disadvantage. The best protection against interviewer bias is to employ highly motivated,
well-paid interviewers who are trained to follow question wording exactly, to record responses
accurately, and to use follow-up questions judiciously. Interviewers should be closely supervised
by the director of the survey project.

Computer technology makes it possible to use a hybrid of a self-administered survey and a

personal interview. A person can listen to computer-recorded questions read by an interviewer
and then respond to the questions on the computer. With this technology each respondent
literally hears the questions read by the same interviewer in the same way, thereby reducing the
risk of interviewer bias. This technology also allows respondents to answer very personal
questions in relative privacy (Rasinski, Willis, Baldwin, Yeh, & Lee, 1999).
Telephone Interviews –
The prohibitive cost of personal interviews and difficulties supervising interviewers have led
survey researchers to turn to telephone or Internet surveys. telephone interviews are frequently
used for brief surveys.

Advantages, telephone interviewing provides better access to dangerous neighborhoods, locked

buildings, and respondents available only during evening hours. Interviews can be completed
more quickly when contacts are made by phone, and interviewers can be better supervised when
all interviews are conducted from one location.

Disadvantages, A possible selection bias exists when respondents are limited to those who have
telephones and the problem of interviewer bias remains. There is a limit to how long respondents
are willing to stay on the phone, and individuals may respond differently when talking to a
“faceless voice” than they would to a personal interviewer. The proliferation of cell phones also
adds an unknown effect.

Hippler and Schwarz (1987) suggest that people take less time to form judgments during phone
interviews and may have difficulty remembering the response options offered by the interviewer.

Internet Surveys –
Surveys were among the earliest Internet-based behavioral studies. Participants complete a
questionnaire online and click on a “submit” button to have their responses recorded. Programs
also exist to permit manipulation of variables and the random assignment of participants to
experimental conditions. (See, for example, Fraley, 2004, for a “ beginner’s guide” to HTML-
based psychological research on the Internet, and Kraut et al., 2004, for useful Internet
resources.)

Advantages, At the top of the list are efficiency and cost (e.g., see Buchanan, 2000; Skitka &
Sargis, 2005). Thousands, if not millions, of participants who vary in age, ethnicity, and even
nationality can be contacted through a few keystrokes on a computer. Time and labor are
dramatically reduced. Online questionnaires are paperless, thus saving natural resources and
copying costs. Participants may respond when it is convenient and do so without leaving the
comfort of their home, office, dorm room, or other Internet site. The Internet is truly a worldwide
source of participants, it also opens up new possibilities for cross-cultural research (e.g., Gosling
et al., 2004).

Disadvantages, At the top of this list is the potential for sample biases (Birnbaum, 2000; Kraut et
al., 2004; Schmidt, 1997). Both response rate bias and selection bias are likely to be present.
Problems with low response rates can occur due to nonresponding of the participant. Lack of
control over the research environment. Respondents may participate alone or in groups, under
distracting conditions, without the knowledge of the researcher (Skitka & Sargis, 2005).

Survey-Research Design
A survey-research design is the overall plan or structure used to conduct the entire study. There
are three general types of survey-research designs: the cross-sectional design, the successive
independent samples design, and the longitudinal design. There is no all-purpose survey-
research design. Researchers choose a design based on the goals of the study.

Cross-Sectional Design –
In a cross-sectional design, one or more samples are drawn from the population at one time. The
focus in a cross-sectional design is description—describing the characteristics of a population or
the differences among two or more populations at a particular time and correlational findings
from cross-sectional designs allow researchers to make predictions. Cross-sectional designs are
ideally suited for the descriptive and predictive goals of survey research.

Successive Independent Samples Design –

In the successive independent samples design, a series of cross-sectional surveys are conducted
over time (successively). The samples are independent because a different sample of respondents
completes the survey at each point in time.

There are two key ingredients:

(1) The same set of questions should be asked of
each sample of respondents, and
(2) the different samples should be drawn from the same population.
If these two conditions are met, researchers can legitimately compare survey responses over
time.

This design is most appropriate when the major goal of the study is to describe changes in the
attitudes or behaviors within a population over time. The purpose of the successive independent
samples design is to describe changes over time in the distribution of population characteristics,
not to describe changes in individual respondents. But we cannot determine who has changed
their views or by how much. It also creates problems when the successive samples are not
representative of the same population.

Longitudinal study –
The distinguishing characteristic of the longitudinal design is that the same sample of
respondents is surveyed more than once. The investigator can determine the direction and extent
of change for individual respondents. The longitudinal design is the best survey design when a
researcher wishes to assess the effect of some naturally occurring event. But it is difficult to
pinpoint the exact causes for individuals’ changes over time. It can be difficult to obtain a sample
of respondents who will agree to participate over time in a longitudinal study. There is a possible
problem due to attrition. As samples
decrease over time, they are less likely to represent the original population from which the
sample was drawn. It is usually possible, however, to determine whether the final sample is
comparable to the original sample in a longitudinal design.

Thinking Critically About Survey Research –

Regardless of how carefully survey data are collected and analyzed, the value of these data
depends on the truthfulness of respondents’ answers to the survey questions. By its very nature,
survey research involves reactive measurement. Social desirability refers to pressure that
respondents sometimes feel to respond as they “should” believe rather than how they actually
believe. For example, if respondents are asked whether they favor giving help to the needy, they
may say “yes” because they believe this is the most socially acceptable attitude to have. In
survey research, as was true with observational research, the best protection against reactive
measurement is to be aware of its existence. Another way researchers can assess the accuracy of
verbal reports is directly or by observing respondents’ behavior. The potential discrepancy
between observed behavior and verbal reports illustrates again the wisdom of a multimethod
approach in helping us identify and address potential problems in understanding behavior and
mental processes.

Correlation and Causality –

Surveys are often used in correlational research, and correlational research is
an excellent method for meeting the scientific goals of description and prediction. Correlational
evidence allows researchers to make predictions for the correlated
variables. However, the familiar maxim, “Correlation does not imply
causation,” reminds us that our ability to make causal inferences based solely
on a correlation between two variables is very limited. It is impossible to determine
the correct causal direction simply by knowing the correlation between the two
variables. When two variables are related (correlated), we can make predictions for
the variables; however, we cannot, simply knowing a correlation, determine
the cause of the relationship. A correlation that can be explained by a third variable is called
a spurious relationship.
It is extremely important to understand why it is not possible to make a causal inference based
only on a correlation between two variables. It is equally important to recognize that
correlational evidence can be very useful in identifying potential causes of behavior.
Sophisticated statistical techniques can be used to help with causal interpretations of
correlational studies. Path analysis is one sophisticated statistical technique that can be used with
correlational data. Path analysis involves the identification of mediator variables and moderator
variables. A mediator variable is a variable that is used to explain the correlation between two
variables. A moderator variable is a variable that affects the direction or strength of the
correlation between two variables. Although correlational research is not an absolutely fi rm
basis for making causal inferences, patterns of correlations observed in path analysis provide
important
clues for identifying causal relationships among variables. The next step for researchers who
wish to make causal inferences is to conduct experiments.