User experience

Questionnaire
What we will cover today
• more on questionnaires
• asking the right questions
 Questionnaires
(can replace questionnaire with survey, interview)
Questionnaires
• questionnaire is a method to collect information by
asking a standard set of questions
• results from questionnaires can be analysed to
provide insight into customers' needs, preferences
and opinions
• results from analysis allow us to make decisions

• questionnaires typically comprise:

• closed questions
• open questions
• short-form questions
• long-form questions
Types of data

quantitative qualitative
things that can be measured insights into thoughts, feelings, opinions
structured questionnaires with and motivation
predefined response options open-ended questions and free-text
responses
Advantages of questionnaires
• large scale data collection • timely
• diverse participants • standardised responses
• widespread locations • consistent format
• broad representation • structured framework
• show trends and patterns • multi-choice questions, Likert
• anonymity scales, rating scales, semantic
• more honest answers differential scales
• more accurate information • simplification of analysis
• statistical tools (SPSS, Excel)
• cost-effective

adapted from: https://pointerpro.com/blog/questionnaire-pros-and-cons

Disadvantages of questionnaires
• potential for limited detail in responses
• low response rates
• survey fatigue
• lack of personal relevance
• try and make it enjoyable (gamification?)
• offer incentives
• does not provide non-verbal information
• tone of voice
• facial expression
• body language
• difficult to follow up responses to get further detail

adapted from: https://pointerpro.com/blog/questionnaire-pros-and-cons

Tips for writing good questionnaires
• clearly outline the purpose of the survey
• explain importance / relevance of questionnaire, and context of questions
• provide clear and concise instructions
• state how long the questionnaire will take to complete
• ensure logical flow to questions
• include "branching" that redirects participants to questions based on previous answers
• always allow user to skip
• tell users that questions are optional (and anonymous)
• allow options such as:
• not sure
• not applicable
• prefer not to say

• split questions into clearly defined and labelled sections

• pilot test with a small representative group
adapted from: https://www.formpl.us/blog/good-bad-survey
Length of questionnaires
• keep it short
• let user know it is short
• shorter = increased completion rate
• but you still need to get all the required information
• don't ask irrelevant questions!
• age?
• gender??
• ethnicity???
Questions
Open and closed questions
• commonly used in conjunction:
closed question followed up by open question for more detail
• closed: • open
• restricted responses • details of experience, opinions,
• yes / no, please select understanding, wishes
• Likert scale • time-consuming to answer
• agree or disagree with a statement • difficult to analyse
• rate our site
• odd number of responses
• easy to complete
• easy to analyse
taken from: https://andyeklund.com/different-types-of-questions/
Likert scale
• statement and several points on a scale, with extremes at either end
• frequently an odd number to allow "neutral" response
• get opinions and attitudes

adapted from: https://www.formpl.us/blog/point-likert-scale

Dichotomous questions
• question with two possible options:
• true : false
• yes : no
• agree : disagree
Multiple-choice questions
• 2 forms:
• choose one option
• choose multiple options
Tips for writing good questions
• make questions easy to understand
• avoid making use of vague and ambiguous words
• avoid technical jargon
• avoid too many options in multiple-choice and ranking questions
• avoid "double-barrelled" questions
• include neutral answer options
• avoid leading and loaded questions
• consider including duplicate questions with slightly different wording
• carefully consider relevance of each question to ensure that only essential
information is collected

adapted from: https://www.formpl.us/blog/good-bad-survey

Choose the correct words
• make it clear:
• avoid technical jargon (while you are not the user, THE USER IS NOT YOU)
• remain neutral and objective:
• avoid leading questions
• provide a middle choice:
• don't know, not applicable, no feeling...
• avoid absolutes:
• never? always?
Avoid double-barrelled questions
• do not ask 2 things at the same time!
• double-barrelled questions
• do you like tea and coffee (yes or no)?
• how do you answer?
• split questions even if it increases length
Avoid bias
• selection bias:
• results are skewed due to only selecting a subset of users
• or because you did not get a response from each subset of users
• response bias:
• survey encourages a certain type of answer
• social desirability bias
• respondents may give answers that are socially acceptable
• "yes" to everything out of politeness
• also "primacy effect" - select first choice (normally when reading)
• and "recency effect" - select last choice (normally when speaking)
Avoid leading questions
• which of the following is best?
• "I saw you were having difficulty with navigation. What
happened?"
• "What did you find difficult with the navigation?"
• "Was it easy or hard to access the content you wanted?"

• how would you improve the following?

• "how healthy are you?"
• "to what extent do you agree with each of the following?"
• "we encourage employees to enhance their skills. Would
you be interested in receiving training?"
Avoid negative questions
• requires negative answer for a positive response...
• ...and a positive answer for a negative response
• complex and potentially confusing
Order questions correctly
• start with basic demographics
• but make them relevant...
• put higher priority questions towards start
• people get bored and may not complete survey
• and order answers
• where should strongly agree appear?
• primary vs recency
Avoid repetition
• don't confuse, annoy, frustrate users
• if questions look similar
• highlight differences
• randomise the order

• don't confuse, annoy, frustrate users

• if questions look similar
• highlight differences
• randomise the order
Avoid vagueness
Provide the correct options
Test it out

• is it logical?
• is it correct?
• is it complete?
TO DO: it's a wrap
• a new company called "It's a wrap" want a website that allows:
• customers to order burritos etc. on a mobile device
• "wrappers" to make wraps based on orders shown on a tablet

• write 10 questions - 5 for customers and 5 for "wrappers" to extract

user requirements
• find out more about each type of user, their activities and
environment
• incorporate questions from the lecture material as appropriate
• the rest of the group will be everyone else's users (ie. a group activity)
Any questions?