Questionnaire Design - Considerations  

 Most problems with questionnaire analysis can be traced back to the design phase of the project. Well-defined goals are the
best way to assure a good questionnaire design. When the goals of a study can be expressed in a few clear and concise
sentences, the design of the questionnaire becomes considerably easier. The questionnaire is developed to directly address
the goals of the study.

One of the best ways to clarify your study goals is to decide how you intend to use the information. Do this before you begin
designing the study. This sounds obvious, but many researchers neglect this task. Why do research if the results will not be
used?

Be sure to commit the study goals to writing. Whenever you are unsure of a question, refer to the study goals and a solution
will become clear. Ask only questions that directly address the study goals. Avoid the temptation to ask questions because it
would be "interesting to know".

As a general rule, with only a few exceptions, long questionnaires get less response than short questionnaires. Keep your
questionnaire short. In fact, the shorter the better. Response rate is the single most important indicator of how much
confidence you can place in the results. A low response rate can be devastating to a study. Therefore, you must do everything
possible to maximize the response rate. One of the most effective methods of maximizing response is to shorten the
questionnaire.

If your survey is over a few pages, try to eliminate questions. Many people have difficulty knowing which questions could be
eliminated. For the elimination round, read each question and ask, "How am I going to use this information?" If the information
will be used in a decision-making process, then keep the question... it's important. If not, throw it out.

One important way to assure a successful survey is to include other experts and relevant decision-makers in the questionnaire
design process. Their suggestions will improve the questionnaire and they will subsequently have more confidence in the
results.

Formulate a plan for doing the statistical analysis during the design stage of the project. Know how every question will be
analyzed and be prepared to handle missing data. If you cannot specify how you intend to analyze a question or use the
information, do not use it in the survey.

Make the envelope unique. We all know how important first impressions are. The same holds true for questionnaires. The
respondent's first impression of the study usually comes from the envelope containing the survey. The best envelopes (i.e.,
the ones that make you want to see what's inside) are colored, hand-addressed and use a commemorative postage stamp.
Envelopes with bulk mail permits or gummed labels are perceived as unimportant. This will generally be reflected in a lower
response rate.

Provide a well-written cover letter. The respondent's next impression comes from the cover letter. The importance of the cover
letter should not be underestimated. It provides your best chance to persuade the respondent to complete the survey.

Give your questionnaire a title that is short and meaningful to the respondent. A questionnaire with a title is generally
perceived to be more credible than one without.

Include clear and concise instructions on how to complete the questionnaire. These must be very easy to understand, so use
short sentences and basic vocabulary. Be sure to print the return address on the questionnaire itself (since questionnaires
often get separated from the reply envelopes).

Begin with a few non-threatening and interesting items. If the first items are too threatening or "boring", there is little chance
that the person will complete the questionnaire. People generally look at the first few questions before deciding whether or not
to complete the questionnaire. Make them want to continue by putting interesting questions first.

Use simple and direct language. The questions must be clearly understood by the respondent. The wording of a question
should be simple and to the point. Do not use uncommon words or long sentences. Make items as brief as possible. This will
reduce misunderstandings and make the questionnaire appear easier to complete. One way to eliminate misunderstandings is
to emphasize crucial words in each item by using bold, italics or underlining.

Leave adequate space for respondents to make comments. One criticism of questionnaires is their inability to retain the
"flavor" of a response. Leaving space for comments will provide valuable information not captured by the response categories.
Leaving white space also makes the questionnaire look easier and this increases response.

Place the most important items in the first half of the questionnaire. Respondents often send back partially completed
questionnaires. By putting the most important items near the beginning, the partially completed questionnaires will still contain
important information.

Hold the respondent's interest. We want the respondent to complete our questionnaire. One way to keep a questionnaire
interesting is to provide variety in the type of items used. Varying the questioning format will also prevent respondents from
falling into "response sets". At the same time, it is important to group items into coherent categories. All items should flow
smoothly from one to the next.

If a questionnaire is more than a few pages and is held together by a staple, include some identifying data on each page (such
as a respondent ID number). Pages often accidentally separate.
Provide incentives as a motivation for a properly completed questionnaire. What does the respondent get for completing your
questionnaire? Altruism is rarely an effective motivator. Attaching a dollar bill to the questionnaire works well. If the information
you are collecting is of interest to the respondent, offering a free summary report is also an excellent motivator. Whatever you
choose, it must make the respondent want to complete the questionnaire.

Use professional production methods for the questionnaire--either desktop publishing or typesetting and keylining. Be creative.
Try different colored inks and paper. The object is to make your questionnaire stand out from all the others the respondent
receives.

Make it convenient. The easier it is for the respondent to complete the questionnaire the better. Always include a self-
addressed postage-paid envelope. Envelopes with postage stamps get better response than business reply envelopes
(although they are more expensive since you also pay for the non-respondents).

The final test of a questionnaire is to try it on representatives of the target audience. If there are problems with the
questionnaire, they almost always show up here. If possible, be present while a respondent is completing the questionnaire
and tell her that it is okay to ask you for clarification of any item. The questions she asks are indicative of problems in the
questionnaire (i.e., the questions on the questionnaire must be without any ambiguity because there will be no chance to
clarify a question when the survey is mailed).