Why Do Research?

Surveys, focus groups and other fun ways to hear

from your constituents or customers

Evans/McDonough Company, Inc.

Ruth Bernstein, Principal
436 14th Street, Suite 820 | Oakland, CA 94621
510-844-0680, x312 | [email protected]
• Reasons for doing opinion
research
• Research methods
• Steps to starting
• Discussion: Suggestions on
your specific situation
Research is a tool used to develop,
enhance, assess, and refine plans for:
– Customer Service
– Communications Program
– Community Relations
– Customer/Community Needs Assessment
– Product Development/Marketing
– Initiative Development
Other Effects of Research

• Fodder for communications

• Demonstration of concern
• Raises expectations
• As a continuing process, shows
progress over time
• Good research saves money
• Good research offers surprises
• Good research saves time
• All research is a snapshot
• Your research firm is a consultant

"It ain't so much the things we don't know that get us in

trouble. It's the things we know that ain't so.“
Artemus Ward
Quantitative Research
• Statistically reliable information that can be
generalized to a larger population.
• Provides specific numeric measures such as:
current performance ratings; effectiveness of
various messages; awareness or interest in
issues or products.
• Can measure performance over time or set
specific quantifiable goals.
• Collection methods can include telephone,
mail/written, web, intercept or IVR
Qualitative Research
• Allows you to see and hear participant opinions
expressed in-depth and in their own words.
• Participant-driven and often leads to unexpected
insights.
• Can help to show why participants think, feel
and react the way they do.
• Allows you to see first hand reactions to
products, materials, messages and ideas.
• Methods include: focus groups or one-on-one
interviews and sometimes web, mail or intercept
studies
Telephone Survey
Advantages:
– Can be generalized to a larger population
– Typically high response rates
– Short turnaround time
– Interviewing can be monitored for quality control,
professional interviewers can probe for incomplete or
invalid responses.
Disadvantages:
– Respondent attitudes and opinions are “artificially”
translated into numerical measures
– Difficult to “get behind” respondent attitudes and
opinions when you haven’t witnessed them.
– Higher cost per complete than other quantitative
research methodologies, such as mail surveys and web
surveys.
Telephone Survey
Uses include:
– Social marketing research
– Measuring effectiveness of information
campaigns
– Measuring community awareness
– Measuring customer satisfaction
– Determining voter support
Telephone Survey
Examples:
Santa Clara Valley Watershed Watch –
Measuring success of information effort and
willingness to participate in pollution
prevention activities
City of Alameda – Community attitudes
regarding housing for preparation of housing
element
City of Richmond - Assessment of voter
support for Utility User Tax increase and
sales tax increase
Mail/Written Survey
Advantages:
– Low cost per completed interview
– Best used when:
• target population is very well defined and easy to
mail/distribute
• population is highly motivated to respond
• providing opportunity to participate more important than
need for quantitative data
Disadvantages:
– Low response rates
– Loss of quality control. Respondent can skip or respond
out of desired order
– Distorted results. Those with the greatest interest more
likely to respond
– Long time from project start to finish
– Not necessarily representative
Mail/Written Survey
Uses include:
– Measuring customer satisfaction
– Public relations
– Customer feedback
Examples:
San Jose Redevelopment Agency –
relocated renter satisfaction
Santa Clara Valley Water District –
ongoing customer feedback cards
Web Survey
Advantages:
– Low cost per completed interview
– Control over respondent experience
– Ability to use multi-media
– Best used when:
• target population is internet savvy
• Email list is available
• population is highly motivated to respond
Disadvantages:
– Low response rates
– Those with the greatest interest more likely to respond
– Less frequent internet users underrepresented (such as
seniors and minorities)
– Dependent on user’s ability to navigate survey
Web Survey
Uses include:
– Opinion tracking
– Public relations/customer feedback
– Feedback on web sites/online services
Examples:
Bay Area Council - Business Confidence
Survey, quarterly tracking of opinion leader
opinions regarding the economy
Santa Clara Valley Water District – ongoing
customer feedback from written cards and
web site link
Intercept Survey
Advantages:
– Direct control over respondent experience
– Ability to show materials or products
– Best used when:
• target population frequents certain geographies
• listed telephone sample is unavailable
• Population is unlikely or unable to respond to a
telephone or mail survey
Disadvantages:
– High cost per completed interview
– Trust in confidentiality and honesty may be
compromised with face-to-face interviews
– Time-consuming
Intercept Survey
Uses include:
– Customer satisfaction
– Customer feedback
Examples:
Mineta San Jose International Airport –
annual customer satisfaction survey
Bay Area Rapid Transit (BART) -
Customer feedback regarding
proposed fare machine change
IVR (Interactive Voice
Response) Survey
Advantages:
– Cost effective in large quantities
– Best used when:
• target population can be reached easily, either by
intercept, mail, or internet
• population is highly motivated to respond
• survey instrument is very short and simple
Disadvantages:
– Low response rates
– Distribution of call-in number
– Those with the greatest interest are more
likely to respond
IVR (Interactive Voice
Response) Survey
Uses include:
– Customer feedback
– Customer satisfaction

Example:
Pierce Transit – customer feedback
survey with bus riders given cards
asking them to call in to participate.
Focus Group
Advantages:
– Permits observation of participants' thinking process,
language, reactions and the effect of various themes,
messages, and strategies.
– Interactions among the participants stimulates thinking.
– Loose structure of a focus group allows the
conversation to move in unexpected directions.
– Ability to observe the effects of groupthink and
effectiveness of individuals who appear knowledgeable
on a subject.
Disadvantages:
– Not statistically significant and, therefore, cannot be
generalized to larger population.
– Results are less clear-cut than in a quantitative study.
Focus Group
Uses include:
– Public information message and materials
development
– Survey language development

Examples:
Alameda County Waste Management –
Development of strategy to encourage
composting
Transportation Agency of Monterey County –
Development of survey language and
information regarding possible transportation
tax measure
Steps to Starting
• A good research firm will ask
– What do you want to know?
– Why do you need that information?
– What will you do with the data?
• The appropriate methodology, sample design,
and instrument can be derived from these
questions.
• It is not necessary to know what kind of
research you want to do, just want you want to
do with the research.
• Often your available budget will determine the
scope of the research you conduct, but it should
not change your question.
Discussion

What do you want to know?