Chapter 9 Designing questionnaires

● How will the responses be analysed?

– Computer
– Manually.
● How will we ask the questions?

Overview of the process

The aim of the questionnaire design process is to convert the research objectives into mean-
ingful questions and assemble the questions into an effective and workable questionnaire.
There are several stages:
● clarifying what it is exactly that you need the questions to measure;
● wording the questions;
● deciding on the types of question and the response format;
● putting the questions into an effective and logical order;
● designing the layout;
● testing out a draft version;
● revising the draft and agreeing a final version.

Question content

The purpose of a questionnaire is to collect data – valid and reliable data that can be used
to address the research problem. The first task in designing a questionnaire (or a discussion
guide) therefore is to clarify the research objectives – the information requirements – and
agree what exactly it is that the questions need to measure.
If the research objectives are not clear it is important to spend time clarifying them. You
cannot design an effective questionnaire without being crystal clear about exactly what infor-
mation it has to deliver. Some exploratory research may be needed to understand the subject
area from the point of view of the target population (often different from how the researcher
or the client might see it) and to uncover the language used to talk about the issues. This
exploratory work might involve a review of secondary data sources (previous research on
the topic, for example) and/or formal or informal qualitative research. The nature of the
Copyright © 2013. Pearson Education, Limited. All rights reserved.

exploratory phase, and the extent of it, will depend on the topic and your familiarity with it
and the time and resources available.

Standard questions
As well as questions that relate directly to the research objectives, you will almost certainly need
questions to determine eligibility to take part in the survey and the characteristics or circum-
stances of those who do. In a consumer or social survey these classification questions might
include questions on age, marital status, working status, social class, total household income,
housing tenure and so on. In a business-to-business survey they might include questions on
type of organisation, job title, number of employees and so on. In addition, in consumer sur-
veys in particular, you might also have questions on awareness (of products, services, brands,
advertising), buying behaviour, usage and satisfaction, for example. For these commonly asked
277

McGivern, Yvonne. The Practice of Market Research EBook : An Introduction, Pearson Education, Limited, 2013. ProQuest Ebook Central,
M09_MCGI3115_01_SE_C09.indd 277 4/22/13 12:55 PM
http://ebookcentral.proquest.com/lib/immgsm/detail.action?docID=5173519.
Created from immgsm on 2021-04-26 09:52:16.
 Part 2 Getting started

questions there is often a standard format and so no need to design them anew each time. Using
standard or consistent questions not only makes questionnaire preparation easier (and, since
these questions are tried and tested, more effective) but it is essential to use a standard format
should you wish to compare responses to these questions across surveys conducted in different
time periods, or even on different topics. It is also essential should you wish to combine or fuse
data from different surveys. Research and client organisations may have their own ‘standard’
versions – check before designing your own. Some examples are given in Box 9.4. Standard ver-
sions of a range of questions (and concepts) used in government surveys have been developed
by experts at the UK’s Office of National Statistics (ONS) to provide a standard way of gathering
information about a particular topic. They refer to them as ‘harmonised concepts and ques-
tions’. You can download examples from the ONS website: http://www.ons.gov.uk/ons/guide-
method/harmonisation/primary-set-of-harmonised-concepts-and-questions/index.html.

BOX 9.4 

Example: standard questions

Marital status Source: ARK. Northern Ireland Life and Times Survey 2010,
www.ark.ac.uk/nilt. Used with permission.
Q. 2 Are you: CODE FIRST TO APPLY
Single (never married) 1 Awareness, purchase and usage
Married and living with husband/wife 2
A civil partner in a legally registered civil 3
Q. 2 Which of these brands of X [SHOW CARD], if
partnership any, have you ever heard of?
Married and separated from husband/wife 4 Q. 3a Thinking about the last time you bought
Divorced 5 [product], which brand did you buy?
Widowed 6 Q. 3b Which brand or brands do you buy most
Spontaneous only – In a legally recognised Civil 7 often?
Partnership and separated from his/her civil
Q. 4 Next time you need [product] how likely or
partner
Spontaneous only – Formerly a civil partner, the 8 unlikely are you to buy brand X?
Civil Partnership now legally dissolved
Spontaneous only – A surviving civil partner: his/ 9
her partner having since died Definitely will 1
Probably will 2
Change in household income Might or might not 3
Probably will not 4
Copyright © 2013. Pearson Education, Limited. All rights reserved.

Q. 6 Looking back over the last year or so, would you

say that your household income has . . . READ OUT . . .  Definitely will not 5
Don’t know 8
. . . fallen behind prices 1
kept up with prices 2
Or, gone up by more than prices? 3
Don’t know 8

Screening and eligibility questions

It may be that, for reasons of client confidentiality or because you believe that certain
groups of people may not be typical or representative of the target group, you want to
exclude those involved in a similar or related area to that being studied. For example,
in an advertising pre-test for a hair shampoo, you may want to exclude those who work

278

McGivern, Yvonne. The Practice of Market Research EBook : An Introduction, Pearson Education, Limited, 2013. ProQuest Ebook Central,
M09_MCGI3115_01_SE_C09.indd 278 4/22/13 12:55 PM
http://ebookcentral.proquest.com/lib/immgsm/detail.action?docID=5173519.
Created from immgsm on 2021-04-26 09:52:16.
 Chapter 9 Designing questionnaires

in the hair care or beauty products industry as well as those who work in marketing,
advertising, public relations or journalism. You may also need a series of questions to
determine if the person contacted is eligible to take part in the research. For example, if
you need to interview representatives of organisations whose customers are primarily
the general public (the consumer market) rather than other businesses (the business-to-
business market), you will need to include a question to establish this. Some examples
are given in Box 9.5.

BOX 9.5 

Example: screening and eligibility questions

Q. A. Do you or anyone in your household work in Q. 1 I am working on a survey among motorists. Do
any of these occupations? SHOW CARD you have a current driving licence for a car?

Market research 1 Yes 1 CONTINUE

Advertising 2 CLOSE No 2 CLOSE
Journalism 3
Public relations 4
Q. 2 Is there a car in your household that is
Marketing 5
­available for you to drive?
Petrol or oil company 6
Motorists’ shop or garage 7 Yes 1 CONTINUE
None of these 8 CONTINUE No 2 CLOSE

Designing questions for some topics may seem to be, or may even be, fairly straight-
forward. The topic might be familiar, or you might be using standard or tried and tested
questions from previous studies. There are, however, some things that are more difficult
to measure, and many things that are more difficult than they at first appear. In such cases
much work is needed to clarify the meaning and define clearly what is to be measured so
that there is no ambiguity about what the question you design is measuring (and how the
response to it is interpreted).
Copyright © 2013. Pearson Education, Limited. All rights reserved.

Clarifying the meaning: concepts, definitions and indicators

Being clear about what is being measured (the concept or the variable) means agreeing a
definition of the concept or variable. This should happen before the questionnaire design
process begins (it is a good idea to do it at the problem definition and research design
stage) so that it is clear what the question needs to measure. For example, think about
something simple such as the age of the respondent – are you measuring the respondent’s
age at the time of the interview or at last birthday? Take another example, the housing
status of the respondent. Do you want to know the respondent’s housing type, that is
whether the type of dwelling he or she lives in is a detached house or an apartment, for
example; or do you want to know about housing tenure, that is whether the house is
owned (on a mortgage or outright) or rented (from a local authority, a housing associa-
tion or a private landlord)?

279

McGivern, Yvonne. The Practice of Market Research EBook : An Introduction, Pearson Education, Limited, 2013. ProQuest Ebook Central,
M09_MCGI3115_01_SE_C09.indd 279 4/22/13 12:55 PM
http://ebookcentral.proquest.com/lib/immgsm/detail.action?docID=5173519.
Created from immgsm on 2021-04-26 09:52:16.
 Part 2 Getting started

Case Study 9.1

What do you mean, anti-social behaviour?

The client wanted to know about anti-social behaviour threats and throwing missiles. It stated that defining
on buses but how do you define the ‘intangible concept’ anti-social behaviour in this way was not wholly ideal as
of anti-social behaviour for use in a questionnaire? Here certain definitions lacked clarity in a practical context
the researchers explain how they did it. as they were not necessarily independent of the context
in which they occurred (e.g. a certain level of noise may
Why this case study is worth reading be acceptable in one area but not in another).
This case study is worth reading for several reasons: it
shows why having a working definition of the ‘thing’ the Using concrete examples
client wants to find out about is important in the ques- While we accepted that this definition was not beyond
tionnaire design and data collection process; it shows criticism, we felt that the overall approach of prompt-
what was involved in the process of arriving at a defini- ing respondents with concrete examples of anti-social
tion; and it shows the link between the concept – the behaviour was the correct one for this study. In particu-
‘thing’ the client wanted to ‘measure’ – and concrete lar, there was the concern that a subjective and open
examples or indicators of it. approach may end in results that reflected popular or
The key words are: definition, alternative defini- media hype about anti-social behaviour rather than
tion, concrete examples, working definition, final actual experience of specific behaviour.
definition, approved, questionnaire, clarity, con-
text, subjective, tested, ‘other’ option. Testing a working definition
A working definition of anti-social behaviour listing cer-
Introduction tain individual types of relevant behaviour was drawn up.
Drawing up a comprehensive definition of anti-social It was tested in a rigorous manner. First, bus drivers and
behaviour would serve three purposes: bus company management staff were asked to discuss the
types of behaviour they considered anti-social on buses
● Explain what anti-social behaviour refers to when
or at bus stops. Secondly, care was taken in the pilot
used in any questionnaire.
interviews to ensure that those answering the question-
● Form the basis of a pre-coded list, for when respond- naire were able to outline any ‘other’ options that they
ents are asked to state spontaneously which types of had not had the chance to mention, with the possibility
activities on buses or at bus stops they perceive as that frequently mentioned answers could be added to the
being anti-social. prompted list. As a result it was decided to include an
● Form the basis of a prompted list, for when respond- ‘other’ option as part of the questionnaire not only when
ents are asked which different types of anti-social asking respondents what types of activity they perceived
behaviour they have actually experienced. as anti-social but also when asking them to detail the
types of anti-social behaviour they experienced. In this
Copyright © 2013. Pearson Education, Limited. All rights reserved.

Defining anti-social behaviour way respondents were given room to include any types
What anti-social behaviour is is a matter of debate of behaviour they considered anti-social and did not fit in
among experts. The Crime and Disorder Act 1998 the existing definition. Thus while a subjective approach
defined anti-social conduct (which includes speech) as (i.e. not prompting respondents at all with elements of
those actions or behaviours that take place ‘in a manner anti-social behaviour) had originally been rejected, a
that caused or was likely to cause harassment, alarm or small element of this approach was incorporated within
distress to one or more persons not of the same house- the question wording.
hold [as the defendant]’. An alternative definition was
drawn up based on information in the UK Government Agreeing a final definition for the quantitative
Home Office Review of Anti-Social Behaviour Orders research
(London School of Economics and Political Science, At the end of this process the following final definition
2003). This review noted 17 different types of behav- of anti-social behaviour relating to buses and bus stops
iour including harassment, noise, drunk and disorderly, was approved:
➨
280

McGivern, Yvonne. The Practice of Market Research EBook : An Introduction, Pearson Education, Limited, 2013. ProQuest Ebook Central,
M09_MCGI3115_01_SE_C09.indd 280 4/22/13 12:55 PM
http://ebookcentral.proquest.com/lib/immgsm/detail.action?docID=5173519.
Created from immgsm on 2021-04-26 09:52:16.
 Chapter 9 Designing questionnaires

‘Anti-social behaviour is defined as behaviour that ● engaging in threatening behaviour in large groups at
threatens the physical or mental health, safety or secu- bus stops or on buses
rity of individuals or causes offence or annoyance to ● smoking of cigarettes or illegal drug taking on buses or
individuals. For the purposes of this particular study at bus stops.’
this definition includes:
This definition was included in all questionnaires (after
● harassment and intimidating behaviour that creates any spontaneous questions concerning the definition of
alarm or fear, towards bus drivers and/or other pas-
anti-social behaviour) to ensure that each respondent
sengers, including verbal or physical abuse
understood what types of behaviour were covered when
● drunken and abusive behaviour towards bus drivers the questionnaire mentioned anti-social behaviour. A
and/or other passengers shortened version was also used as the pre-coded list
● assault of bus drivers and/or other passengers for the question asking respondents to define anti-
● vehicle crime such as vandalism, graffiti, throwing mis- social behaviour and as the prompted options when
siles or other deliberate damage to buses or bus company asking for experience of anti-social behaviour.
property Source: Adapted from Granville, S., Campbell-Jack, D. and Lam-
● dumping litter or rubbish on buses plugh, T. (2005) ‘Perception, prevention, policing and the chal-
lenges of researching anti-social behaviour’, MRS Conference,
● conflicts or racist abuse/incidents www.mrs.org.uk.

Concepts and conceptualisation

In some cases it is relatively easy to decide what is to be measured and relatively easy to reach an
agreed definition; in other cases it is not so easy. Think, for example, how you might meaning-
fully measure the incidence of sexism. Before deciding how, you need to define what you mean
by the term ‘sexism’. You need a nominal or working definition of the fairly abstract concept of
sexism and you need to specify a set of more concrete ‘indicators’ of it. This process of moving
from the abstract to the concrete is known as conceptual ordering or conceptualisation.

Case Study 9.2

Cross-cultural research: defining indicators

Graham Mytton from BBC World Service describes will be an important factor in making comparisons
some of the issues involved in defining key indicators with household data from elsewhere where households
for cross-cultural use. may be defined differently. Different interpretations of
some words used in questionnaires can be problematic.
Why this case study is worth reading
Copyright © 2013. Pearson Education, Limited. All rights reserved.

Precise terms may not be understood in the same way.

This case study is worth reading for two reasons: it illus- Even periods of time such as a week, a month or a year
trates one of the barriers to be overcome in doing cross- may have different lengths. The day may begin not at
cultural research. midnight, but at dawn or dusk. Collecting data on age
The key words are: indicators, compared across can be a major problem. The practice of knowing one’s
cultures, household, interpretations, precise terms. birthday, or even birth year, is certainly not universal.
One way is to try to work out the person’s age by finding
Indicators out events which happened at key times in the respond-
Terms like ‘household’, ‘occupation’ and the demo- ent’s life. Can he or she remember independence or the
graphic indicators of education, social class and start of the civil war or certain floods, volcanic eruptions
income level are a problem to define in a way which or other events? Were these before or after puberty?
can be compared across cultures. In Northern Nigeria,
for example, where people can live in large extended Source: Adapted from Mytton, G. (1996) ‘Research in new fields’,
International Journal of Market Research, 38, 1, pp. 19–31,
family compounds, what does ‘household’ mean? If such www.ijmr.com.
a compound or gida is to be defined as a household it

281

McGivern, Yvonne. The Practice of Market Research EBook : An Introduction, Pearson Education, Limited, 2013. ProQuest Ebook Central,
M09_MCGI3115_01_SE_C09.indd 281 4/22/13 12:55 PM
http://ebookcentral.proquest.com/lib/immgsm/detail.action?docID=5173519.
Created from immgsm on 2021-04-26 09:52:16.
 Part 2 Getting started

Definitions
So how do you arrive at a working definition? You could, for example, using formal or
informal qualitative research, ask members of the target group what sexism means to them;
you could check what definitions others have used (via a search of secondary sources).
Whichever method you use, the outcome should be a clear specification of exactly what it
is you are going to measure with the question or set of questions you construct and exactly
what you mean when you use a particular word or phrase to describe that concept (or vari-
able). The nominal definition of sexism might be something like ‘the view that one sex is
inherently superior to the other and/or that particular roles or tasks are suited to one sex
or the other’.

Indicators
Once you have a clear and agreed definition of the concept the next step is to develop
a set of concrete ‘indicators’ of it. These indicators will be used in designing the ques-
tion or set of questions to measure the concept. To get from the abstract concept to
the concrete indicators of it you may need to think about the ‘dimensions’ or aspects of
the concept. You might decide that really you are interested in the gender stereotyping
dimension of sexism (the view that particular roles or tasks are suited to one sex or the
other). You might go further and specify that you are interested in gender stereotyping in
relation to home or family duties or in relation to work and job roles, or both. In making
these sorts of decisions you would refer back to the research objectives and the question
of why you are interested in measuring the incidence of sexism in the first place. You
might be interested in measuring the incidence of gender stereotyping in relation to work
in order to design equality awareness courses for employees, for example. So how do
you develop indicators of gender stereotyping? Again a review of the relevant literature
and/or exploratory qualitative research can be useful. The indicators in relation to work
roles might include a view that men are more suited to jobs with a physical aspect, or
less suited to jobs involving children. A question from the Life and Times Survey 2000
(reproduced in Box 9.6) shows the sort of question that you might design based on your
indicators.
The task does not end with the design of the question. The next step is to think about how
to interpret the responses to the question. What pattern of response would indicate or could
be interpreted to mean that the respondent tends to gender stereotype? You might first of
Copyright © 2013. Pearson Education, Limited. All rights reserved.

all make explicit which roles you regard as traditionally male and traditionally female. For
example: firefighter, soldier in ‘front line’ action and priest or minister – male; and child-
minder, midwife, staying at home to look after the children and secretary – female; and pri-
mary school teacher – both. You might then devise a scoring system or scale so that a higher
score indicates a stronger tendency to gender stereotype (assigning traditionally male jobs
as appropriate to men only and traditionally female jobs to women only) and a lower score
indicates a weaker tendency. In reporting on the incidence of gender stereotyping you should
make it clear to the audience or reader not only how you defined the concept and how you
measured it, but also how you analysed and interpreted the data. This is important as you
could almost certainly come up with a different set of findings about gender stereotyping if
you used a different definition, a different set of indicators and a different way of analysing
and interpreting the data.

282

McGivern, Yvonne. The Practice of Market Research EBook : An Introduction, Pearson Education, Limited, 2013. ProQuest Ebook Central,
M09_MCGI3115_01_SE_C09.indd 282 4/22/13 12:55 PM
http://ebookcentral.proquest.com/lib/immgsm/detail.action?docID=5173519.
Created from immgsm on 2021-04-26 09:52:16.