Questionnaire Design

At-a-glance I. II. III. A SURVEY IS ONLY AS GOOD AS THE QUESTIONS IT ASKS QUESTIONNAIRE DESIGN: AN OVERVIEW OF THE MAJOR DECISIONS WHAT SHOULD BE ASKED? A. Questionnaire relevancy B. Questionnaire accuracy PHRASING QUESTIONS A. Open-ended response versus fixed-alternative questions 1. Single-dichotomy or dichotomous-alternative 2. Multi-choice alternatives B. Phrasing questions for mail, telephone, and personal interview surveys THE ART OF ASKING QUESTIONS A. Avoid complexity: use simple, conversational language B. Avoid leading and loaded questions C. Avoid ambiguity - be as specific as possible D. Avoid double-barreled items E. Avoid making assumptions WHAT IS THE BEST QUESTION SEQUENCE? A. Order Bias WHAT IS THE BEST LAYOUT LAYOUT OF INTERNET QUESTIONNAIRES? HOW MUCH PRETESTING AND REVISING IS NECESSARY? DESIGNING QUESTIONNAIRES FOR GLOBAL RESEARCH

IV.

V.

VI. VII. VIII.

X.

Lecture Outline I. A SURVEY IS ONLY AS GOOD AS THE QUESTION IT ASKS The importance of question wording is easily overlooked, but questionnaire design is one of the most critical stages in the survey research process. Unfortunately, newcomers, who naively believe that common sense and good grammar are all one needs to construct a questionnaire, generally learn that their hasty efforts are inadequate. II. QUESTIONNAIRE DESIGN: AN OVERVIEW OF THE MAJOR DECISIONS Relevance and accuracy are the two most basic criteria to be met if the questionnaire is to achieve the researchers purpose. In order to achieve this, several decisions must be made; this chapter deals with these decisions.

III.WHAT SHOULD BE ASKED? The specific questions to be asked will be a function of the previous decisions. The later stages of the research process will have an important impact on the questionnaire wording. When designing the questionnaire, the researcher must also be thinking about the types of statistical analysis that will be conducted. A. Questionnaire relevancy: A questionnaire is relevant if no unnecessary information is collected and if only the information needed to solve the marketing problem is obtained. To ensure information relevance, the researcher must be specific about data needs; there should be a rationale for each item, and all possible omissions should be considered. B. Questionnaire accuracy. Accuracy means that the information is reliable and valid. While it is generally believed that one should use simple, understandable, unbiased, unambiguous, and nonirritating words, no step-by-step procedure can be generalized. Respondents tend to be most cooperative when the subject of the research is interesting - if questions are lengthy, difficult to answer, or ego threatening, there is a high probability of biased answers. Question wording and sequence substantially influence accuracy. IV.PHRASING QUESTIONS There are many ways to phrase questions and many standard question formats have been developed in previous research. A. Open-ended response versus fixed-alternative questions: We may categorize two basic types of questions asked on the amount of freedom respondents are given in answering. Open-ended response questions pose some problem or topic and ask the respondents to answer in their own words. Open-ended response questions are free response questions. In contrast, fixed-alternative questions, or closed questions, give the respondents specific, limited, alternative responses and ask the respondent to choose the response closest to his or her viewpoint. Open-ended response questions are most beneficial when the researcher is conducting exploratory research. By gaining free and uninhibited responses, the researcher may find some unanticipated reaction toward the project. They may also be useful at the beginning of an interview as they allow the respondent to warm up to the questioning process. The cost of open-ended response questions is substantially higher than that of fixedalternative questions, since the job of coding, editing, and analyzing the data is quite extensive. Also, open-ended response questions allow potential interviewer bias to influence the answer - even the best interviewer can take shortcuts in recording answers. B. Fixed-alternative questions require less interviewer skill, take less time, and are easier for the respondent to answer. There are various types of fixed-alternative questions. 1. Single-dichotomy or dichotomous-alternative questions: require that the respondents choose one of two alternatives. The answer can be a simple yes or no or a choice between this and that. Multi-choice alternatives: there are several types. a) Determinant choice questions ask a respondent to choose one and only one - response from among several possible alternatives.

2.

Chapter 15 Questionnaire Design

139

b) The frequency determination question is a determinant choice question that asks for an answer about the general frequency of occurrence. c) Attitude rating scales, such as the Likert scale, are discussed in chapter 13. d) The check list question allows respondents to provide multiple answers to a single question. In many cases, the choices are adjectives that describe a particular object. There should be no overlap among categories in the check list - each alternative should be mutually exclusive, that is, only one dimension of an issue should be related to that alternative. The researcher should strive to ensure that there are sufficient response choices to include almost all possible answers. However, including a category lower than the answers you expect often helps to negate the potential bias caused by respondents avoiding an extreme category. Respondents, rather than stating why they chose a given product, may select an alternative among those presented. Or, as a matter of convenience, they may select a given alternative rather than thinking of the most correct alternative. Most questionnaires include a mixture of open-ended and closed questions. Each form has unique benefits; in addition, respondent boredom and fatigue are eliminated with a change of pace offered by a mixture of question types. C. Phrasing questions for mail, telephone, and personal interview surveys: In general, mail and telephone questions must be less complex than those utilized in personal interviews. Questionnaires for telephone and personal interviews should be written in a conversational manner. Exhibit 15.1 illustrates how a question may be revised for a different medium. V. THE ART OF ASKING QUESTIONS In developing a questionnaire, there are no hard and fast rules. Some guidelines have been developed to avoid the most common mistakes. A. Avoid complexity: use simple, conversational language: Words used in a questionnaire should be readily understandable to all respondents. The technical jargon of top executives should be avoided. B. Avoid leading and loaded questions: Asking leading and loaded questions is a major source of bias in question wording. Leading questions suggest or imply certain answers. Such questions may result in a bandwagon effect which threatens the studys validity. Partial mention of alternatives is a variation of this phenomena. Loaded questions suggest social desirability or are emotionally charged. Some questions invite only positive answers. In other cases, respondents are able to interpret which answers are most socially acceptable, and the resulting answers may not portray the respondents true feelings. Invoking the status quo is a form of loading that results in bias because the majority of people tend to be resistant to change. Asking respondents how often leads them to generalize about their behavior and one is more likely to portray ones ideal behavior rather than ones average behavior. An introductory counterbiasing statement to a question, that reassures respondents that their embarrassing behavior is not abnormal, may help yield truthful responses. Also, an assurance of anonymity may help elicit honest responses to embarrassing questions.

A question statement may be leading because it is phrased to reflect either the negative or positive aspects of an issue. To control for this bias, split-ballot technique, which reverses the wording of attitudinal questions for 50 percent of the sample, can be used. C. Avoid ambiguity - be as specific as possible: Items on questionnaires are often ambiguous because they are too general. Indefinite words such as frequently, often, ready, etc., have many different meanings. Use of such words should be avoided - the questions should be as specific as possible. D. Avoid double-barreled items: A question covering several items at once is referred to as a double-barreled question and should always be avoided. There is no need for the confusion that results in a double-barreled question. E.Avoid making assumptions: The researcher should not place the respondent in a bind by including an implicit assumption in the question. Another mistake that question writers sometimes make is assuming that the respondent has previously thought about an issue -research that induces people to express attitudes on subjects that they do not ordinarily think about is meaningless. VI.WHAT IS THE BEST QUESTION SEQUENCE? The order of questions may serve several functions for the researcher. For example, if the respondents curiosity is not aroused at the outset, they can become disinterested and terminate the interview. A. Order bias: Order bias results from an alternative answers position in a set of answers or from the sequencing of questions. Order bias tends to distort survey results. Specific questions tend to influence more general ones. Thus, it is advisable to ask the general questions before the specific questions to obtain the freest of open-ended responses. This technique is known as the funnel technique, and it allows researchers to understand the respondents frame of reference before asking more specific questions about the respondents particular level of information and intensity of opinion. When using attitude scales, there also may be an anchoring effect. That is, the first concept measured tends to become a comparison point from which subsequent evaluations are made. Randomization of these items on a questionnaire helps to minimize this order bias. Rarely do marketing researchers print alternative question forms to eliminate problems arising from order bias. A more common practice is to pencil Xs or check marks on the printed questionnaires to indicate that the interviewer should start a series of repetitive questions at a certain point. Filter questions minimize the asking of questions that may be inapplicable, and pivot questions may be used to obtain information that the respondent may be reluctant to provide. For example, a respondent is asked Is your family income over $30,000? If under, ask Is it over or under $10,000? If over, ask Is it over or under $50,000? Exhibit 15.2 gives an example of a flow chart plan for a questionnaire. Notice the logical order of questions which can help ensure the respondents cooperation, and can help eliminate any confusion or indecision. VII. WHAT IS THE BEST LAYOUT? The layout and attractiveness of the questionnaire are of crucial importance. In mail questionnaires, often

Chapter 15 Questionnaire Design

141

the rate of return can be improved by adding the money, that might have been used as an incentive, to improve the attractiveness and quality of the questionnaire. Questionnaires should be designed to appear as short as possible and experienced researchers have found that it pays to carefully phrase the title to be printed on the questionnaire. The researcher can design the questionnaire to make the interviewees job of following interconnected questions much easier by utilizing several forms, special instructions, and other tricks of the trade. VIII. LAYOUT OF INTERNET QUESTIONNAIRES A web questionnaire site should be easy to use, have a logical flow, and have a graphic look-and-feel that creates an overall experience that motivates the respondent to cooperate from start to finish. Many of the guidelines for layout of paper questionnaires apply to Internet questionnaires. There are, however, many important differences. Graphical User Interface (GUI) refers to software that provide an attractive and easy to use interface between a computer user and an application. With respect to Internet surveys, it refers to the researchers control over the background, colors, fonts, and other visual features displayed on the computer screen. It allows the researcher to design questionnaires that allow respondents to click on what they want rather than having to type answers or codes. Web publishing software, such as FrontPage or Netscape Composer, is often used to format the questionnaire so the researcher will know how it should appear online. However, several features of a respondents computer may influence the appearance of an Internet questionnaire. For example, a respondents computer settings for screen configuration (e.g. 640 x 480 pixels versus 800 x 600 pixels) may result in questions that are not fully visible on the respondents screen, misaligned text, or other visual appearance problems. The fact that the questionnaire image the researcher/designer constructs on his or her computer may be different from the questionnaire image on the respondents computer should always be considered when designing Internet surveys. One sophisticated remedy is to use the first few questions on an Internet survey to ask about operating system, browser software, and other computer configuration issues so that questionnaire delivered is as compatible as possible. Another, simpler solution, is to limit the horizontal width of the questions to 70 characters or less to decrease the likelihood of wrap-around text. Even if the questionnaire designers computer and the respondents computer are compatible, there are several other layout issues. The first layout decision is whether the questionnaire will appear page-bypage, showing individual questions on single screens (Web pages) or on a scrolling basis with the entire questionnaire appearing on a one screen (Web page). The paging layout going from screen to screen greatly facilitates skip patterns. Based on respondents answers to filter questions, the computer automatically inserts relevant questions on subsequent pages. If the entire questionnaire appears on the screen, the display should advance smoothly, as if it were a piece of paper being moved up or down. The scrolling layout gives the respondent the ability to scroll down to read any portion of the questionnaire at any time and because there are no page boundaries problems can arise. For example, avoiding the problems associated with splitting questions and response categories may be difficult. On paper Likert scales are often shown in a multiple-grid format. Suppose a Likert scale consists of 15 statements and a grid-format layout places the response categories strongly agree, agree, disagree, and strongly disagree at the beginning of the question. Scrolling down beyond the first few statements may not allow the respondent to simultaneously see both the statements at the end of the list and the response categories at the top of the grid. When a scrolling questionnaire is long, using multiple category or section headings is helpful to respondents. Provide appropriate links to allow users to go to the top and bottom parts of each section, enabling the respondent to navigate through the questionnaire more easily than having to scroll through the entire document.

Whether a page-by-page or scrolling format is utilized push buttons with labels should clearly describe the actions to be taken. For example, if the respondent needs to go to the next page a large arrow labeled NEXT might appear in color at the bottom of a screen. Layout decisions must also be made about the use of color, graphics, animation, sound, and other special features that the Internet makes possible. One thing to remember is that for people with very powerful computers sophisticated graphics are not a problem. However, many respondents computers are not powerful enough to have complex features operationally delivered at a satisfactory speed, if at all. For example, using a textured background, colored headings, and small graphics can make a questionnaire become more interesting and appealing, but they may present problems for respondents with older computers and/or low bandwidth Internet connections. With a paper questionnaire, the respondent knows how many questions he or she must answer. Because many Internet surveys offer no visual clues about the number of questions to be asked, it is important to provide a status bar or another visual indicator of questionnaire length. For example using a partially filled rectangular box as a visual symbol and a statement such as: The status bar at top right indicates approximately what portion of the survey you have completed. increases the likelihood that the respondent will finish the entire sequence of questions. An Internet questionnaire uses special windows known as dialog boxes to display questions and record answers. The Exhibit entitled Alternative Ways of Displaying Internet Questions in the textbook portrays four common ways of displaying questions on a computer screen. Many responses to Internet questions require the respondent to activate his or her answer by clicking a radio button on a response. Like push buttons on automobile radios, clicking on an alternative response deactivates the first choice and replaces it with the new response. A drop-down box, such as the one shown in the Exhibit, is a space saving device that allows the researcher to provide a list of responses that are hidden from view. Initially a general statement, perhaps please select or click here is shown. Clicking on a down-facing arrow makes the full range of choices appear. If the first choice in a list, such as strongly agree, is shown with other responses hidden, the researcher is increasing his or her chances that response bias will occur. Drop-down boxes may present a problem for individuals with minimal computer skills who may not know how to reveal hidden responses behind a drop-down menu or how to move from one option to another in a moving-bar menu. Checklist questions may be placed in a check box where several, none, or all responses may be checked. Open-ended boxes require respondents to use their keyboards to enter text for open-ended questions. Open-ended boxes may be designed as One line Text Boxes or Scrolling Text Boxes depending on the breadth of the expected answer. Of course, open-ended response questions require that respondents have both the skill and the willingness to keyboard lengthy answers into the computer. Some open-ended boxes are designed so numbers can be entered for frequency response, ranking, or rating questions. Pop-up boxes are message boxes that can be used to provide highlighted information. For example, popup boxes may be use to provide a privacy statement. Internet software allows questioning to branch off into two or more different lines depending on each respondents answer to skip or filter questions. Questionnaire software with Boolean skip and branching logic is readily available. Most of these programs have hidden skip logic so respondents never see any evidence of skips. It is best if the questions the respondent sees always flow in numerical sequence. However, some programs number all potential questions in a numerical order and the respondent sees only the number on the questions he or she answers. Thus, a respondent may answer questions 1 thru 11 and then next see a question numbered 15 because of the skip logic. Software can systematically or randomly manipulate the question a respondent sees. Variable piping software allows variables, such as answers from previous questions, to be inserted in to unfolding

Chapter 15 Questionnaire Design

143

questions. Other software can randomly rotate questions, blocks of questions, and response alternative order from respondent to respondent. The researchers can use software to control of the questionnaire flow. Respondents can be blocked from backing up, or can be allowed to stop mid-questionnaire and come back later to finish. If the respondent fails to answer a question, or answers it with an incorrect type of response, an immediate error message appears. This is called error trapping. With forced answering software, respondents cannot skip over questions as they do in mail surveys. The program will not let them continue if they fail to answer a question. The software may insert a bold-faced error message on the question screen or insert a pop-up box instructing the respondent how to continue. Some research suppliers offer interactive help desks to solve problems that might be encountered in completing a questionnaire. IX. HOW MUCH PRETESTING AND REVISING IS NECESSARY Usually, the questionnaire is tried out on a group selected on a convenience basis and similar in makeup to the one that ultimately will be sampled; it is not necessary to get a statistical sample for pretesting. Pretesting allows the researcher to determine if the respondents have any difficulty understanding the questionnaire or if there are any ambiguous or biased questions. This process can save the potential disaster of administering an invalid questionnaire to several hundred individuals. A preliminary tabulation of the pretest results often illustrates that while a question is easily comprehended and answered by the respondent, it is inappropriate because it is does not solve the marketing problem. Pretests are typically conducted to answer questions about the questionnaire such as: . Can the questionnaire format be followed by the interviewer? . Does the questionnaire flow naturally and conversationally? . Are the questions clear and easy to understand? . Can respondents answer the questions easily? . Which alternative forms of questions work best? Pretests provide the means for testing the sampling procedure and also provide estimates for the response rates for the mail surveys and completion rates for telephone surveys. X. DESIGNING QUESTIONNAIRES FOR GLOBAL MARKETS International business researchers must take cultural factors into account when designing questionnaires. The most widespread problem involves translation into another language. International questionnaires are often back translated. Back Translation is the process of translating the questionnaire from one language to another and then having it translated back again by an second, independent translator. The back translator is often a person whose native tongue is the language that will be used on the questionnaire. Questions/Answers

1. Evaluate and comment on the following questions from several different questionnaires: See the textbook for the actual set of questions. A) The fixed alternative responses to this question about SPSS are somewhat ambiguous. Infrequently, occasionally, frequently, and all the time may mean different things to different people. The inclusion of the time period in parentheses helps; however, the answers here are not exhaustive and mutually exclusive. Where does one who uses the computer center twice a semester record his or her response? B) This question about the Federal Income Tax Returns is complex because there are really two issues. First, whether the individual understands the policy and, second, whether or not the individual likes the policy. It would be best to ask two questions. C) The first question asked the respondents to imagine that they do 90 percent of their driving in town. If this is not the case, the answer has to be pure fantasy. The second question asks the respondent to compare the electric car with the Big Three. Some of us may interpret this to be Ford, GM, and Chrysler. Others may interpret this to be Honda, GM and Ford. The term Big Three should be defined. Questions that ask the percentage of usage, time, and the like are fine if every respondent makes sure that his or her answer amounts to 100 percent. Unfortunately, many times respondents will only partially answer the question or incorrectly account for 100 percent. They may indicate walk 60 percent of the time without filling in any of the other responses. Or they may indicate walk 60 percent of the time, bicycle 30 percent of the time, and public transportation 30 percent of the time. The answer exceeds 100 percent. This requires editing on the part of the researcher. If possible, questions like this should be avoided.

D)

E) In this question it may be that not all respondents will believe that the medical insurance program is generous. The answers are limited to yes and no. By responding yes, one assumes that everything is just fine as it is. If a negative answer is given, it implies that the policy should be discontinued. A solution might be to ask degrees of satisfaction with the program such as very satisfied, somewhat satisfied, etc. F) Perhaps the use of more simple, conversational language would be appropriate. Respondents may be affected by the desire to make a socially desirable response. G) The questions begins with to make up for past discrimination. The statement is very general and It assumes that the respondent will agree with this statement. H) This survey question, perhaps needless to say, is extremely complex. You can give another example. For example, researchers should avoid making statements such as, The biota exhibited a 100 percent mortality response when all they need to say is, All the fish died. I) J) This question seems to be somewhat loaded and emotionally charged. This question is from a personal interview. Although the question asks for one of two alternatives, up or down, if the respondents answers stays the same or dont know, these responses will be recorded. To some extent up and down are forced choice alternatives which will reduce the number of stay the same answers because it is not part of the options listed in the question.

K) This is a long and complex question. By the time the question is asked it is likely that the

Chapter 15 Questionnaire Design

145

respondent will have forgotten what was read to him or her. L) This question starts with an assumption: Since agriculture is vital to our states economy it assumes that the respondent will agree with this statement. 2. The following question was asked: We are going to ask you to classify the type of fan you consider yourself to be for different sports and sports programs. Diehard Fan - Watch games, follow-up on scores and sports news multiple times a day Avid Fan - Watch games, follow-up on scores and sports news once a day Casual Fan - Watch games, follow-up on scores and sports news occasionally Championship Fan - Watch games, follow-up on scores and sports news only during championships/playoffs Non-Fan - Never watch games or follow-up on scores Anti-Fan - Dislike, oppose, or object to a certain sport Does this question do a good job of avoiding ambiguity? This is an excellent way of defining terms that the respondent may not formally use. For example, Diehard Fan could mean a lifelong Chicago Cubs fan. However, the definition here makes it clear what this term means in the context of this study. 3. How might the wording of a question asking about income influence a respondents answer? This question allows the instructor to discuss the fact that wording of questions on sensitive issues, such as income, may provide the respondent with a tendency to answer more heavily in a particular category. 4. Design an open-ended question(s) to measure reactions to a Xerox magazine ad. An aided recall question sequence might go somewhat like this after having identified a reader of a particular magazine: a. Do you remember seeing a commercial for Xerox? b. (IF YES) Would you please tell me everything you remember about the Xerox commercial? What else did the commercial show? What else did the commercial say? What do you think was the main point they were trying to get across? (ASK IF AD WAS NOTICED OR READ) c. Now think back to when you first went through the magazine. advertisement? (ASK IF AD WAS NOT SEEN) d. People often dont notice every advertisement. Could you give me any ideas as to why you did not notice this advertisement? Why did you notice this

(ASK EVERYONE) e. What comes to mind when you look at this advertisement? Tell me anything that occurs to you. (PROBE) f. Which part or parts of the advertisement, that is, headline, illustration, or copy, particularly caught your attention? Why? g. Taking the advertisement as a whole, what do you think the advertiser was trying to convey? h. From some advertisements you learn things that you were not aware of previously. From other ads you dont learn anything new. What do you think about this advertisement? Did you learn anything that you were not aware of previously or had perhaps forgotten? YES... (IF YES) What did you learn? (PROBE) i. Now think about this for a moment. If you were presently considering the purchase of this kind of product, would this advertisement help you to make the decision to purchase the product? YES... NO... NO...

i1. (IF YES) How would it help you decide to purchase it? i2. (IF NO) Why wouldnt it help you to decide to purchase? j. What additional information do you feel you need? k. What action did you take after reading this advertisement--or what action do you plan to take? Were interested in any action at all taken as a result of this ad. 5. What is the difference between a leading and a loaded question? It is difficult to draw a fine line between the two. However, the wording of a leading question influences the answers by providing a hint about the purpose of the question or a clue that hints at something such that the respondent thinks a particular answer is an acceptable answer. A loaded question is emotionally charged. 6. Design a complete questionnaire to evaluate job satisfaction. This is an individual student project. The Job Description Index in the text is one alternative. The Survey Research Center at the University of Michigan publishes a Quality of Employment survey every five years. This is another alternative. Many other alternatives may be found in John Robinson Measures of Social Psychological Attitudes (Ann Arbor: Survey Research Center, University of Michigan). 7. Design a complete (but short) questionnaire to measure student evaluations of a college course. This is an individual student project. The questionnaire utilized at the instructors school may be an interesting standard of comparison. See Exhibit 1 and Exhibit 2.

Chapter 15 Questionnaire Design

147

Exhibit 1 STUDENT EVALUATION OF FACULTY Compared to other courses and instructors at this university, rate the following aspects of this course: Below Average 1. 2. 3. 4. 5. 6. 7. Open communication is encouraged in the classroom. The professor is enthusiastic about the subject being taught. Ideas are supported with examples, comparisons and facts. Students are encouraged to think for themselves. Concepts presented in class are explained clearly. Useful feedback on progress in the course is provided. The professor evidences a broad, accurate, up-to-date knowledge of the subject. The professor motivates me to do my best work. Exhibit 2 STUDENT SURVEY OF INSTRUCTION All questions below are to be answered by blackening with #2 pencil the appropriate answer space at the bottom of the page. Please make broad pencil marks that completely fill the area indicated. Do not mark beyond the edges of the circles, and erase any pencil marks you wish to delete. FOR items 1 through 5 mark your responses in answer spaces 1 through 5 below. 1. My college is: (A) Agriculture (B) Arts & Sciences (C) Business (D) Education (E) Engineering (F) Graduate (G) Home Economics (H) School of Technology (I) Veterinary Medicine (J) None of These 2. Classification: (A) Freshman (B) Sophomore (C) Junior (D) Senior (E) Graduate or Special 3. Purpose for taking course: (A) Major (B) Related to Major (C) General Studies (D) Elective 4. Course was required: (A) Yes (B) No 5. Type of course: (A) Lecture (B) Lab (C) IPI (D) Short Course (E) Other 1 1 1 1 1 1 2 2 2 2 2 2 Average 3 3 3 3 3 3 4 4 4 4 4 4 Above Average 5 5 5 5 5 5

1 1

2 2

3 3

4 4

5 5

8.

FOR items 6 through 12, RANK THE INSTRUCTOR using this scale: (A) Very High (B) High (C) Average (D) Low (E) Very Low 6. Preparation and organization 7. Effort devoted to teaching 8. Presentation of material 9. Knowledge of subject 10. Ability to explain subject matter 11. Positive attitude toward students 12. Overall INSTRUCTOR appraisal FOR items 13 through 19 give your views of THE COURSE using this scale: (A) Definitely yes (B) Yes (C) Not applicable (D) No (E) Definitely no 13. 14. 15. 16. 17. 18. 19. I learned a lot in this course. The workload was appropriate for the hours of credit. Assignments were relevant and useful. Testing and evaluation procedures were good. Students were adequately involved. This course was worthwhile to me. Overall, this was a good COURSE.

After marking your remarks on this side of the form, please add any additional comments on the reverse side. 8. Develop a check list of things to do in questionnaire construction. Hundreds of questions might be listed. Here are some questions that might be included: Are simple words used? Is the language vague? Are the questions short? Are the questions easy to understand? Are the questions specific or general? Are the questions leading? Are the questions loaded? Is any objectional language utilized? Have any assumptions been made in the questions? Are any hypothetical questions used? Did you talk down to the respondent? Are any of the questions double-barreled? Are you asking two questions or one? Are any questions too sensitive? Is the question necessary? Is the questionnaire layout appealing? Is the questionnaire layout easy to follow? Is there enough room to answer the questions? Are the instructions easy to understand? 9. The Apple Assistance Center is a hotline to solve problems for users of Macintosh computers and other Apple products. Design a short postcard-sized consumer satisfaction/service quality questionnaire for the Apple Assistance Center.

Chapter 15 Questionnaire Design

149

Apple Computer The postcard cover letter and questionnaire that was sent to a buyer of a Macintosh computer who called and asked the Apple Assistance Center about installing software appear below. Dear Apple Customer: Recently you contacted the Apple Assistance Center giving us the opportunity to respond to your request for assistance. The Center was established so that we could better serve our customers. To help us determine how well we responded to your call, please take a few minutes to complete the questionnaire on the other side. Apple Computer has paid the return postage. Thank you for taking the time to let us know how were doing. We value your opinion as much as your business. Sincerely,

Director-Apple Assistance Center Apple Assistance Center Customer Satisfaction Survey 1. Which ONE of the statements below BEST describes your feelings about the action taken by Apple Computer to respond to your request for assistance? 1. I was completely satisfied. 2. I was not completely satisfied, but the action taken was acceptable. 3. I was not completely satisfied, but some action was taken. 4. I was not at all satisfied with the action taken. 5. I was not at all satisfied; no action was taken. 2. On a scale of 1 to 5, how satisfied were you with the Apple Assistance Center in EACH of the following areas? PERSONAL: Very Very Dissatisfied ---------------------------------------Satisfied a. Responsive 1 2 2 1 2 3 3 3 4 4 4 5 5 5

b. Professionalism 1 c. Knowledge RESPONSE:

a. Timeliness

b. Clarity 1 2 3 4 5 c. Helpfulness 1 2 3 4 5 3. Would you recommend Apple computer equipment to a friend or business associate? Might or Definitely 1 Might Not 3 Probably Not 4 Definitely Not 5

Probably 2

10. A client tells a researcher that she wants a questionnaire that evaluates the importance of 30 product characteristics and to determine how her firms brand rates her brand and 10 competing brands rate on these characteristics. The researcher believes that this questionnaire will induce respondent fatigue because it is far too long. Should the researcher do exactly what the client says or risk losing the business by suggesting a different approach? This is an eternal question. Does consumer oriented mean doing exactly what the client requests or telling the client there is a better or more appropriate way. In many situations, clients will dogmatically say this is exactly what I want do it or I will get someone else to do it for me. One interpretation of the marketing concept of consumer orientation is that the executive (researcher) thinks about the clients long term best interest. If the researcher believes the research would not accomplish its task, he or she should tell this to the client and then determine what to do. 11. A lobbying organization designs a short questionnaire about its political position. It also includes a membership solicitation with the questionnaire. Is this approach ethical? This is wrong. The purpose of research is research not promotion or membership solicitation. The researcher has an obligation to the respondent to keep everything above board because the respondent donates his or her time. 12. A public figure who supports cost cutting in government asks the following question in a survey: Do you support a Presidential line item veto to eliminate waste in government? Is this ethical? This is a question from an independent candidates effort to learn the publics opinion. Many in the media criticized the effort because the question is emotionally loaded. Everyone is against waste in government. Thus the question does not adequately address this issue about line item vetoes. Intentionally writing loaded questions is not the purpose of marketing research.