7 Tips for

Writing Surveys
Survey design and flow give power to your research.
But great questions are the foundation for great research.

Scott M. Smith, Ph.D.

 qualtrics.com

1. Keep It Simple
Do you remember taking the SAT or ACT? It’s a long and boring process.

Your average survey respondent can start to feel that way about 15 minutes into a
survey. Fifteen minutes is a good upper-limit for most surveys.

WHEN A SURVEY IS TOO LONG, THREE BAD THINGS CAN HAPPEN

Respondents Drop Out: They simply quit taking the survey. It costs money to find
respondents, and a high drop-out rate can not only cost a lot, but can influence the
quality of your results. Having a reward for completion can reduce drop-outs, but you
can’t stop it completely.

People Stop Paying Attention: Remember your elementary-school classmate who

just filled in random bubbles during a test? He grew up. If it takes too long to take your
survey, he might do it again. We actually see this a lot, and encourage researchers to
use attention filters to prevent this problem.

Clients Get Angry: The irony of upsetting customers with an overly long satisfaction
survey is not lost on your respondents.

The best way to collect quality data is to keep your surveys short, simple,
and well organized.

2. Use Scales Whenever Possible

Rather than asking respondents a basic yes or no question, use scales that measure
both the direction and the intensity of opinions. This is critical for research. Someone
who “Strongly Supports” a decision is very different from someone who “Slightly
Supports” it.

Scales extend the power of analysis from basic percentages to high level analyses
based on means and variance estimates (like t-test, ANOVA, regression, and the like).

2 7 Tips for Writing Surveys

 qualtrics.com

EXAMPLE: MEAN: Often referred to as an average, it is the sum of all the values divided
by the number of values.
Assuming we have data
points 1, 2, and 6. MEDIAN: The middle point in a data set. To determine the median, lay out a
distribution from lowest to highest and select the middle value.
MEAN: 3 = (9/3)
MEDIAN: 2 RANGE: The highest and lowest data points in a distribution from the range.
RANGE: 1-6
VARIANCE: 7 VARIANCE: A dispersion measure of how far a set of numbers is spread out.

Use scales whenever you can. You will get more information from each question.

3. Keep Coded Values Consistent

Every survey response, option, question, or answer is coded as a numeric value that is
reported as a percent of responses or as a mean, median, range, etc. These values are
the basis for analysis.

Values must be coded consistently. Generally, we assign the highest value to the best
outcome (i.e. “Strongly Agree” that customer service is responsive) and then move
down from there.

For simplicity, keep your scale direction consistent throughout your survey. This makes
it easier for respondents to answer and for you as a researcher to conduct your analysis.
If scales have the same scale of points, you can quickly compare responses to different
questions. For example, if a survey asks respondents to rate a series of statements from
Strongly Disagree to Strongly Agree, the responses are given these values:

Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree

3 7 Tips for Writing Surveys

 qualtrics.com

Standard scaling helps managers to quickly understand customer service ratings by

simply looking at averages. For example, once managers understand that a 5-point
agreement scale is being used, they could be given the mean results for the following
customer evaluations (agreement) statements:

• I am completely satisfied with the customer service – 3.15

• The customer service is prompt – 4.12
• Customer service representatives are polite – 4.67
• Customer service representatives are knowledge – 2.08

Since all the statements are positive and the values are scaled consistently, a higher
mean reflects better results in that area. A manager can look at these means and
quickly identify the 2.08.

We see that customer service representatives are prompt and polite, but they don’t
seem to know what they’re talking about. As a result, overall satisfaction with customer
service is perhaps much lower than it could be.

You can reverse scales (or word questions negatively) to encourage respondents to read
more carefully. However, if you use reversed scales or negative wording for some items,
be sure to recode the scales so that all scales point in the same direction. This will allow
you to quickly compare multiple areas of customer service. (You can do recodes easily
in Qualtrics.)

The simplest solution is just to keep all scales consistent throughout every survey.

4. Explain Why
Respondents are more likely to help you if they see something of positive value for
them. Value offerings can range from a very general altruistic appeal for their help to
a very specific offer of an economic incentive. For instance, with a customer feedback
survey, you can explain that feedback will help improve customer service.

MAKE IT SPECIFIC TO THEM

With employee evaluations, you can explain that feedback will be used to determine
awards, promotions, and pay raises and will help management make organizational
decisions that will affect them.

4 7 Tips for Writing Surveys

 qualtrics.com

EXPLAIN UNEXPECTED QUESTIONS

Explain unexpected questions. For instance, if it’s important for you to ask toy store
customers their preferred color of jeans, you might want to explain why that’s relevant.

JUSTIFY REQUESTS FOR SENSITIVE INFORMATION

This is particularly true with any information that may be deemed sensitive. You can
often diffuse respondents’ concerns about sharing information by telling them how it
is going to be used. For instance, you can explain that purchasing habits will only be
analyzed in aggregate for benchmarking purposes or that results will not be shared
outside your organization.

5. Speak Your Respondents’ Language

Asking about caloric content, bits, bytes, and other industry specific jargon and
acronyms is confusing. Make sure your audience understands your language level and
terminology and above all, that they understand what you are asking.

The best move is to write to your least-informed respondent. If a respondent won’t

understand an acronym, either define it, or don’t put it in.

6. Follow a Logical Order

Make your survey easier for respondents by keeping questions in their logical order.
Avoid changing topics unnecessarily.

USE THE FUNNEL APPROACH

The funnel approach makes the respondent’s job easier.

5 7 Tips for Writing Surveys

 qualtrics.com

• Start with broad and general questions that qualify the respondent
and introduce the topic
• Move into more specific questions
• Finish with general, easy-to-answer questions (like demographics)

This approach allows respondents to warm up with broad and general questions, work
into more specific and in-depth questions, and cool down at the end.

This turns the survey into a smooth road for respondents, which decreases drop-out
rates and may even increase the quality of answers you receive.

7. Take Your Survey for a Test Drive

Even the best researchers have the occasional typo, misdirected question, or unfamiliar
buzz word in their surveys. Finding these last little issues is a difficult process.
Fortunately, there is an easy solution:

ASK 5 PEOPLE FROM YOUR TARGET DEMOGRAPHIC TO TAKE YOUR SURVEY

Then ask them:

• How long did it take? (be sure to keep it short.)

• Which questions were confusing?
• Were there any other problems while taking the survey?

This allows you to quickly correct lingering problems before distribution.

Summary
Survey building is as much an art as it is a science. It involves attention to detail in
the design and flow of your survey. Keep it simple, keep your scales consistent, and
communicate well. Review your question flow and then pretest. These simple tips will
go a long way toward building your “perfect” survey.

6 7 Tips for Writing Surveys

 qualtrics.com

Qualtrics is the world’s leading

enterprise survey technology provider.
That’s the classy way of saying we
help you get the data and insight you
need to make better decisions.

To create a free account, visit www.qualtrics.com

Customer Sat & NPS Tracking Market Research Voice of the Customer Academic Research Employee Feedback