UNIT 1: WRITING REPORT

What will be covered in this unit?

1. An introduction to report writing

2. Differences between reports and essays
3. Structures of reports
4. Academic writing style in reports
5. Use of graphical data
6. Analysis of the findings
7. Checklist for effective reports

1. An introduction to report writing

The report form is used mainly in science, social science and business-related subjects,
and in each discipline the form will vary. It is a very flexible form of writing and can be
adapted to any investigative context - which is why it is one of the most used forms of
writing in the working world, and why students are asked to write them.

Purpose

An academic report can document a trial, experiment, investigation or intervention. Some

examples include:

 Reporting on a marketing survey

 A laboratory report (in psychology, medical or physical sciences)
 Exploring the traffic patterns through a building and making recommendations
 Exploring the efficacy of a medical treatment

Sources

Academic reports draw from both:

 research and readings and

 experiences and learning activities such as:
 Reviewing and appraising group work or work experience,
 Documenting and assessing a clinical, professional or pedagogical activity
 Mapping and optimizing traffic patterns through a building
 Documenting and analyzing a particular business activity as a case study
2. Differences between reports and essays

A report is a piece of informative writing that describes a set of actions and analyses any
results in response to a specific brief. A quick definition might be: “This is what I did,
and this is what it means.”

You may also have assignments which are not called reports, but which are still pieces of
informative writing; for instance, some dissertations and project write-ups fall into this
category.

A good way to understand the key features of reports is to see how they differ from
essays as a type of academic writing.

Reports Essays

 Formally structured  Semi-structured

 Informative and fact-based  Argumentative and idea-based
 Written with a specific purpose and  Not written with a specific reader in
reader in mind mind
 Written in a style appropriate to each  Written in single narrative style
section throughout
 Always include section headings  Usually do not include sub-headings
 Sometimes use bullet points  Usually do not include bullet points
 Often include tables or graphs  Rarely include tables or graphs
 Offer recommendations for action  Offer conclusions about a question

Practice 1: Decide whether the following topics are more likely to be written as
reports or essays.

Topic Report Essay

1 The development of trade unions in South Africa
2 Two alternative plans for improving the sports centre
3 A study you conducted to compare male and female attitudes
to eating
4 An overview of recent research on the human genome
5 The arguments for and against capital punishment
3. Structure of a report

A key feature of reports is that they are formally structured in sections. The use of
sections makes it easy for the reader to jump straight to the information they need. Unlike
an essay which is written in a single narrative style from start to finish, each section of a
report has its own purpose and will need to be written in an appropriate style to suit – for
example, the methods and results sections are mainly descriptive, whereas the discussion
section needs to be analytical. Understanding the function of each section will help you to
structure your information and use the correct writing style. Reports for different briefs
require different sections, so always check carefully any instructions you've been given.

Title

The title needs to concisely state the topic of the report. It needs to be informative and
descriptive so that someone just reading the title will understand the main issue of your
report. You don’t need to include excessive detail in your title but avoid being vague and
too general.

Abstract

(Also called the Summary or Executive Summary) This is the ‘shop window’ for your
report. It is the first (and sometimes the only) section to be read and should be the last to
be written. It should enable the reader to make an informed decision about whether they
want to read the whole report. The length will depend on the extent of the work reported
but it is usually a paragraph or two and always less than a page. A good way to write an
abstract is to think of it as a series of brief answers to questions. These would probably
include:

 What is the purpose of the work?

 What methods did you use for your research?
 What were the main findings and conclusions reached as a result of your
research?
 Did your work lead you to make any recommendations for future actions?

Introduction

(Also called Background or Context) In this section you explain the rationale for
undertaking the work reported on, including what you have been asked (or chosen) to do,
the reasons for doing it and the background to the study. It should be written in an
explanatory style. State what the report is about - what is the question you are trying to
answer? If it is a brief for a specific reader (e.g. a feasibility report on a construction
project for a client), say who they are. Describe your starting point and the background to
the subject, for instance: what research has already been done (if you have been asked to
include a Literature Review later in the report, you only need a brief outline of previous
research in the Introduction); what are the relevant themes and issues; why are you being
asked to investigate it now? Explain how you are going to go about responding to the
brief. If you are going to test a hypothesis in your research, include this at the end of your
introduction. Include a brief outline of your method of enquiry. State the limits of your
research and reasons for them, for example; “Research will focus on native English
speakers only, as a proper consideration of the issues arising from speaking English as a
second language is beyond the scope of this project”.

Literature Review

This is a survey of publications (books, journals, authoritative websites, sometimes

conference papers) reporting work that has already been done on the topic of your report.
It should only include studies that have direct relevance to your research. A literature
survey should be written like an essay in a discursive style, with an introduction, main
discussion grouped in themes and a conclusion.

 Introduce your review by explaining how you went about finding your materials,
and any clear trends in research that have emerged.
 Group your texts in themes.
 Write about each theme as a separate section, giving a critical summary of each
piece of work and showing its relevance to your research.
 Conclude with how the review has informed your research (things you’ll be
building on, gaps you’ll be filling, etc.).

Methods

(Also called Methodology) You need to write your Methods section in such a way that a
reader could replicate the research you have done. There should be no ambiguity here, so
you need to write in a very factual informative style. You need to state clearly how you
carried out your investigation. Explain why you chose this particular method
(questionnaires, focus group, experimental procedure, etc.), include techniques and any
equipment you used. If there were participants in your research, who were they? How
many? How were they selected? Write this section concisely but thoroughly – go through
what you did step by step, including everything that is relevant. You know what you did,
but could a reader follow your description?

Results

(Also called Data or Findings) This section has only one job which is to present the
findings of your research as simply and clearly as possible. Use the format that will
achieve this most effectively e.g. text, graphs, tables or diagrams. When deciding on a
graphical format to use, think about how the data will look to the reader. Choose just one
format - don’t repeat the same information in, for instance, a graph and a table. Label
your graphs and tables clearly. Give each figure a title and describe in words what the
figure demonstrates. Writing in this section should be clear, factual and informative. Save
your interpretation of the results for the Discussion section.

Discussion

This is probably the longest section and worth spending time on. It brings everything
together, showing how your findings respond to the brief you explained in your
introduction and the previous research you surveyed in your literature survey. It should
be written in a discursive style, meaning you need to discuss not only what your findings
show but why they show this, using evidence from previous research to back up your
explanations. This is also the place to mention if there were any problems (for instance, if
your results were different from expectations, you couldn’t find important data, or you
had to change your method or participants) and how they were or could have been solved.

Conclusion

Your conclusions should be a short section with no new arguments or evidence. Sum up
the main points of your research - how do they answer the original brief for the work
reported on? This section may also include:

Recommendations for action

Suggestions for further research

References

(Also called Reference List or Bibliography) List here full details for any works you have
referred to in the report, including books, journals, websites and other materials. You
may also need to list works you have used in preparing your report but have not explicitly
referred to - check your instructions for this and for the correct style of referencing to use.

Appendices

The appendices hold any additional information that may help the reader but is not
essential to the report’s main findings: anything that 'adds value'. That might include (for
instance) interview questions, raw data or a glossary of terms used. Label all appendices
and refer to them where appropriate in the main text (e.g. ‘See Appendix A for an
example questionnaire’).
Practice 2: For this exercise, read the following short extracts from reports found in
various journals. Decide which of the above sections they are from, and why they
are appropriate for this section.

Example 1

There were several barriers identified by the pharmacists to Informed Shared Decision
Making (ISDM). One major barrier was perceived lack of collaboration between
pharmacists and physicians. Some of the pharmacists said that they were often reluctant
to intervene because physicians were not receptive to their interventions. The current
literature agrees with this point of view, suggesting that most pharmacist-physician
relationships in the community setting are not at a stage to allow seamless
interdisciplinary collaboration. However, it was surprising to find such a large number of
pharmacists from the hospital category also express lack of teamwork as a barrier.

Example 2

In the fatal accident case outlined, health and safety legislation had been at best
misunderstood and at worse ignored. Consideration of fundamental aspects of health and
safety planning under the Construction Design and Management Regulations (1994) was
clearly lacking, with deficiencies apparent at both pre-contract and site work stages.
Moreover, there was disregard to fundamental health, welfare and safety provision under
current legislation.

Example 3

During the first 33 weeks, the mean weight decreased among those who completed the
course by 5.0 kg (6.5%) in women (p<0.001) and 11.1 kg (8.25%) in men (p<0.001). The
initial weight loss was maintained during the first year, but during the second year some
of the body weight was regained (1.2 kg among women and 6.5 kg among men).
However, at 2 years the mean weight was still significantly decreased by almost 4% in
both sexes.

Example 4

Tourism-led development is clearly an emerging theme in South African Local Economic

Development (LED) (Rogerson, 1997, 2001), with tourism promotion geared towards
community development being perceived as a viable growth option (Goudie et al., 1999;
Kirsten and Rogerson, 2002; Mahoney and van Zyl, 2002). However, despite the
prominence accorded to tourism in South Africa's broad development vision and in many
local level strategies, as Rogerson notes, in studies of LED '...tourism-led LED is
markedly under-represented and little discussed' (Rogerson, 2002: 1).

Example 5

Taking a memory test not only assesses what one knows, but also enhances later
retention, a phenomenon known as the testing effect. We studied this effect with
emotionally relevant materials and investigated whether testing facilitates learning only
because tests offer an opportunity to restudy material. In two experiments, students
studied prose passages and took one to three immediate free-recall tests, without
feedback, or restudied the material the same number of times as the students who
received tests. Students then took a final retention test 5 min, 2 days, or 1 week later.
When the final test was given after 5 min, repeated studying improved recall relative to
repeated testing. However, on the delayed tests, prior testing produced substantially
greater retention that studying, even though repeated studying increased students'
confidence in their ability to remember the material. Testing is a powerful means of
improving learning, not just assessing it.

4. Academic writing style in reports

A scientific paper is a formal text written in an impersonal, objective, neutral and

professional way. The writing is concise with specific wordings and formulations. The
writing should be clear as well and lead to a deeper understanding of the subject.

 Avoid colloquial or informal language. Eliminate all 'filler' and needless words.
Biased language is not acceptable in academic writing.
 Avoid personal or familiar language. Do not directly address the reader and do
not ask rhetorical questions. Remember that the use of personal pronouns does
not fit into an objective, scientific paper.
 Avoid ambiguous, imprecise or vague words such as ‘various', 'some',
'particular', 'numerous'. Try to avoid impersonal expressions. Be clear,
concrete, specific, precise and direct. If possible, choose specific wordings
which will lead to more concise writing.
 Do not use 'wordy' expressions either. To illustrate:
o 'Since' or 'because' are easier to read than 'for the reason that' or 'owing
to the fact that'.
o 'Although' is easier to read than 'despite the fact that' or 'regardless of
the fact that'.
 o 'If' is easier to read than 'in the event that'.

Practice 3

Read the following extracts from reports.

In each case identify the main problem with the writing style, then rewrite the
problematic sentences so they are more suitable for an academic report.

Example 1
After adding the solution, the mixture in the test tube went a bright scarlet red, which we
did not expect, as this was not the same as the washed out pink color it was supposed to
go according to the book. We shook the test tube up and left it for a while in the test tube
stand. When we came back, the mixture had settled to the bottom and dried out, which it
was not supposed to have happened; this was a bit of a problem.

Example 2
The results of the aforementioned experiment were collated in a systemized manner with
all due care and attention. Once the results had been collected and processed with the
appropriate process, it was necessary to perform the analysis upon them which was
conducted using the computer statistical package previously mentioned in the
methodology.

Example 3
The questionnaire will be carried out in the pubic space next to the town hall, were local
residents can be found and stopped. If we can't find enough local people to answer our
survey, we'll do down the street and see if we can find some more in the shopping centre
as there tends to be local youths hanging around their who we could ask.

Example 4
This is a feasibility study of the proposed extension to the property "22 Trumpington
Terrace" commissioned by the residents Mr. and Mrs. Phelps. The house is a semi-
detached, two-bedroom property with glazed windows and a red door. It is situated on a
quiet, winding street with nearby houses of a similar design and the residents have started
a local Neighborhood Watch group. The Phelps inherited the house from Mr. Phelps'
uncle and have been in residence there since 31st August 1998.

5. Using graphical data

Sometimes words are not the most effective way to communicate. Using graphs, tables
and charts can help your reader to get a clearer picture of your research findings and how
they compare with other data. Using drawings, diagrams and photographs can clearly
present information that would be difficult to explain in words. The key is to consider
what is needed to do the job you need doing. It's especially important with drawings,
diagrams and photographs to always ask yourself whether they are necessary - what
would happen if they weren't there?

Tables are useful when you need to

present a quantity of numerical data in an
accessible format and you need to show
exact numbers.

Line graphs are especially effective at

showing trends (how data changes over
time) and relationships (how two
variables interact).

Bar charts/graphs are good when you

want to compare discrete items. The bars
can be vertical or horizontal. Making
them different colors can help the reader
to differentiate each result.

Pie charts show the proportion of the

whole that is taken by various parts.
 Drawings and diagrams can be used to
reinforce or supplement textual
information, or where something is more
clearly shown in diagrammatic form.

Photographs can be useful as illustrations

that help to explain what is being
discussed in the text.

6. Analyzing the findings

Analyzing critically means looking at your findings and asking yourself, 'what do I think
about this?', then taking it one step further and asking 'what is making me think that?'
Here's an example of how this process might work.

Finding 95% of the students you surveyed have problems managing

their time at university.

What do you think I expected it to be less than that.

about this? 

What makes you Research I read for my literature survey was putting the
think that? figure at 60-70%.
What conclusions There must be reasons why the figures are so different. The
can you draw sample I surveyed included a large number of mature
from this? students, unlike the samples in the previous research. That
was because the brief was to look at time management in a
particular department which had a high intake of post-
experience students.
Final paragraph The percentage of students surveyed who experienced
for Discussion problems with time management was much higher at 95%
 section  than the 60% reported in Jones (2006: 33) or the 70%
reported in Smith (2007: 17). This may be due to the large
number of mature students recruited to this post-experience
course. Taylor (2004: 16-21) has described the additional
time commitments reported by students with young families
and the impact these may have on effective management of
study time. The department clearly recognizes this already
(as shown by the flexibility of seminar times described
earlier). However, it may be that students would benefit
from more advice in this area. 

7. Checklist for effective reports

Before you submit your work, check that it includes these features of good reports:

1. Does it answer the purpose stated (or implied) in the brief?

2. Does it answer the needs of the projected reader?
3. Has the material been placed in the appropriate sections?
4. Has all the material been checked for accuracy?
5. Are graphs and tables carefully labelled?
6. Is data in graphs or tables also explained in words and analyzed?
7. Does the discussion/conclusion show how the results relate to objectives set out in
the introduction?
8. Have you discussed how your results relate to existing research mentioned in your
literature survey?
9. Has all irrelevant material been removed?
10. Is it written throughout in appropriate style (i.e. no colloquialisms or contractions,
using an objective tone, specific rather than vague)?
11. Is it jargon-free and clearly written?
12. Has every idea taken from or inspired by someone else’s work been acknowledged
with a reference?
13. Have all illustrations and figures taken from someone else’s work been cited
correctly?
UNIT 5: QUESTIONNAIRE DESIGN

This unit is intended to help the students to:

· Understand the attributes of a well-designed questionnaire, and
· Adopt a framework for developing questionnaires.

I. USING SURVEYS IN SOCIAL SCIENCE RESEARCH

In social science research, a variety of methodological approaches exist for individuals
interested in conducting research. Selection of a research approach depends on a number of
factors, including the purpose of the research, the type of research questions to be answered, and
the availability of resources.
One of the most common approaches to the conduct of research is using surveys. Survey
research is defined as "the collection of information from a sample of individuals through their
responses to questions" This type of research allows for a variety of methods to recruit
participants, collect data, and utilize various methods of instrumentation. Survey research can
use quantitative research strategies (e.g., using questionnaires with numerically rated items),
qualitative research strategies (e.g., using open-ended questions), or both strategies (i.e., mixed
methods). As it is often used to describe and explore human behavior, surveys are therefore
frequently used in social and psychological research. Survey research may use a variety of data
collection methods with the most common being questionnaires and interviews.
Questionnaires may be self-administered or administered by a professional, may be
administered individually or in a group, and typically include a series of items reflecting the
research aims. Among the types of information that can be collected by means of a questionnaire
are facts, activities, level of knowledge, opinions, expectations and aspirations, membership of
various groups, and attitudes and perceptions.
Questionnaires may be in paper form and mailed to participants, delivered in an electronic
format via email or an Internet-based program such as Survey Monkey, or a combination of both,
giving the participant the option to choose which method is preferred. Using a combination of
methods of survey administration can help to ensure better sample coverage (i.e., all individuals
in the population having a chance of inclusion in the sample) therefore reducing coverage error.
For example, if a researcher were to only use an Internet-delivered questionnaire, individuals
without access to a computer would be excluded from participation. Self-administered mailed,
group, or Internet-based questionnaires are relatively low cost and practical for a large sample.
Conducting interviews is another approach to data collection used in survey research.
Interviews may be conducted by phone, computer, or in person and have the benefit of visually
identifying the nonverbal response(s) of the interviewee and subsequently being able to clarify
the intended question. An interviewer can use probing comments to obtain more information
about a question or topic and can request clarification of an unclear response Interviews can be
costly and time intensive, and therefore are relatively impractical for large samples.
Some authors advocate for using mixed methods for survey research when no one method is
adequate to address the planned research aims, to reduce the potential for measurement and non-
response error, and to better tailor the study methods to the intended sample. For example, a
mixed methods survey research approach may begin with distributing a questionnaire and
following up with telephone interviews to clarify unclear survey responses Mixed methods might
also be used when visual or auditory deficits preclude an individual from completing a
questionnaire or participating in an interview.
II. DESIGNING A QUESTIONNAIRE
A questionnaire is often the first tool that people consider when undertaking a research
project. However, designing a questionnaire is complex and time-consuming and the quality of
the data collected is determined by the quality of the questionnaire used. Therefore, one needs to
carefully consider the advantages and disadvantages of using questionnaire as suggested below:
Advantages Disadvantages
 The data gathered is standardised and  Responses may be inaccurate, especially through mis-
therefore, easy to analyse; interpretation of questions in self-completing questionnaires;
 Data can be gathered quickly from a  A reasonable sample size is needed before the responses can
large number of respondents; be used to represent the population as a whole;
 It is possible to compare your results  Response rates can be poor, people may lack the motivation
with similar surveys used in other to complete or return the questionnaire. Consequently, some
institutions; sort of incentive may have to be included, for example, a
 Respondents can answer anonymously prize draw;
which may produce more honest  The complexity of designing, producing, distributing, and
answers; analysing the questionnaires may make them expensive and
 Online surveys are relatively time consuming;
inexpensive;  Quantitative data (i.e, research that produces numbers and
 One person can administer the whole percentages etc.) may not be enough to answer the questions
process if they have the necessary you are seeking to answer in your research. More in-depth
 skills. interviews or focus groups may be needed.

2.1. Structures of questions

One important aspect of questionnaire design is the decision on structure of the questions as
this decides the types of data one gets for later analysis. Broadly speaking, survey questions can
be classified into three structures: closed, open-ended, and contingency questions.
2.1.1. Closed questions
Closed (or multiple choice) questions ask the respondent to choose, among a possible set of
answers, the response that most closely represents his/her viewpoint. The respondent is usually
asked to tick or circle the chosen answer. Questions of this kind may offer simple alternatives
such as ‘Yes’ or ‘No’. They may also require that the respondent chooses among several answer
categories, or that he/she uses a frequency scale, an importance scale, or an agreement scale.
How often do your parents ask you about your homework?
(Please, circle one answer only)
Never . . . . . . . . . . . . . . . . . . .1
1 or 2 times a week . . . . . . . . 2
3 or 4 times a week . . . . . . . . 3
Nearly every day . . . . . . . . . . 4

The main advantages of closed questions are:

• the respondent is restricted to a finite (and therefore more manageable) set of responses,
• they are easy and quick to answer,
• they have response categories that are easy to code, and
• they permit the inclusion of more variables in a research study because the format enable
the respondent to answer more questions in the same time required to answer fewer open- ended
questions.
The main disadvantages with closed questions are:
• they can introduce bias, either by forcing the respondent to choose between give
alternatives or by offering alternatives that otherwise would not have come to mind,
• they do not allow for creativity or for the respondent to develop ideas,
• they do not permit the respondent to qualify the chosen response or express a more complex
or subtle meaning,
• they can introduce bias, where there is a tendency for the respondent to tick systematically
either the first or last category, to select what may be considered as the most socially desirable
response alternative, or to answer all items in a list in the same way, and
• they require skill to write because response categories need to be appropriate, and mutually
exclusive.
The response format for closed questions can range from a simple yes/no response, to an
approve/disapprove alternative, to asking the respondent to choose one alternative from 3 or
more response options.
The possibility of format effects or response bias for this type of question can be reduced by
changing the sequence of response categories and values. For example, if responses to an item
range from 1 to 5, going from negative to positive, then a number of items in the questionnaire
can be designed to have 1 as the most positive alternative and 5 as the most negative. This is a
particularly important technique for the construction of attitude scales.
Some closed questions may have a dichotomous response format, which means only two
mutually exclusive responses are provided.
What is your sex? (Please tick one box only)
Male Female

For the above example a dichotomous response format is appropriate. However, this type of
format should not be overused in a survey because it elicits much less information than multiple
choice formats. For example, if seeking information on degree of interest in public affairs, the
question “Do you read a daily newspaper?” yields a yes/no response. This could be reworded to:
“How many times per week do you read a daily newspaper?”, to which multiple choice
responses could be:
1. Seven times a week
2. Five to six times a week
3. Three to four times a week
4. One to two times per week
5. Less than once per week
6. Never
Such a multiple category response format would provide more specific and more useful
information than the dichotomous one.
2.1.2. Open-ended questions
Open-ended or free-response questions are not followed by any choices and the respondent
must answer by supplying a response, usually by entering a number, a word, or a short text.
Answers are recorded in full, either by the interviewer or, in the case of a self- administered
survey, the respondent records his or her own entire response.
What are your favourite TV programmes? (Please specify their
titles) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
........................
What do you like most about school?
...........................................................................
. . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The main advantages of open-ended questions are:
• they allow respondents to express their ideas spontaneously in their own language,
• they are less likely to suggest or guide the answer than closed questions because they are
free from the format effects associated with closed questions, and
• they can add new information when there is very little existing information available about
a topic.
The main disadvantages of open-ended questions are:
• they may be difficult to answer and even more difficult to analyze,
• they require effort and time on behalf of the respondent,
• they require the development of a system of coded categories with which to classify the
responses,
• they require the respondent to have some degree of writing ability, and respondent
handwriting can be illegible.
There is always the possibility with open-ended questions that responses may come in very
different forms, and these may lead to answers that cannot be systematically coded for analysis.
For example, if asked “When did you leave school?”, the respondent may answer in a variety of
ways: “Seven years ago”. “When I got my first job”. “When my brother started going to high
school”. “When my parents moved into this house”.
If the survey is administered by an interviewer, appropriate probing helps clarify such answers.
In the case of a self-administered survey, guidance by writing specific instructions on how to
answer the question can often minimize the number of responses that have very different
dimensions.
Care should be taken in writing open-ended questions so as to avoid formats that elicit a
dichotomous yes/no or agree/disagree response. In addition, the wording of questions should
seek to reduce the possibility of eliciting responses that are aligned along very different
dimensions and therefore cannot be systematically coded. For example, asking “What do you
think about your school?” can elicit responses such as ‘nothing’ or ‘school is useless’. However,
asking “What recommendations would you have for improving your school?” would be more
likely to elicit informative answers.
A good case for using open-ended questions is when the aim is to have the respondents reply
spontaneously, or when the investigator is pilot testing the first version of the questionnaire, or
when the investigator wants to collect evidence on the parameters of an issue with the aim of
later formulating a multiple choice or closed version of a question.
Generally, open-ended questions can produce useful information in an interviewer
administered survey, provided that the interviewers are alert and trained to probe ambiguous
responses. In self-administered surveys, it is useful to provide instructions on the format of the
response that is required so as to minimize opportunities for the respondents to answer the
question according to very different dimensions.
2.1.3. Contingency questions
A contingency question is a special case of a closed-ended question because it applies only to
a subgroup of respondents. The relevance of the question for a subgroup is determined by asking
a filter question. The filter question directs the subgroup to answer a relevant set of specialized
questions and instructs other respondents to skip to a later section of the questionnaire.
The advantage of contingency questions is that detailed data may be obtained from a specific
subgroup of the population. Some questions may apply only to females and not to males; others
may apply only to people in school, and not to those who are employed. At the base of good
contingency questions are clear and specific instructions to respondents.
The formats for filter and contingency questions can vary. One option is to write directions
next to the response category of the filter question.
Are you enrolled in secondary school?
1. Yes (answer the following question)
2. No (skip to question 5)

Alternatively, the contingency question can be placed at the end of the questionnaire set apart
from ordinary questions that are to be answered by everybody:

ANSWER THIS FINAL SET OF QUESTIONS ONLY IF YOU PLAN ON ENTERING

AN ADULT EDUCATION COURSE NEXT YEAR.
 OTHERWISE, YOU HAVE NOW COMPLETED THE QUESTIONNAIRE.

2.2. Guidelines for writing questions

There are no all-purpose rules that, if followed, will automatically result in a well-written
questionnaire. There are, however, some basic principles that, when violated, usually result in
respondent confusion, misunderstanding, lack of comprehension, or response bias.
2.2.1. Keep the vocabulary simple
A first rule concerns the vocabulary used in writing questions and answer categories. The rule
is ‘keep it as simple as possible’. This implies using simple words, avoiding acronyms,
abbreviations, jargon, technical terms, and abstract or general words.
If a rare or technical term has to be used, then its meaning should be explained. For example,
a question concerning the frequency with which teachers teach their students to understand
different styles of text should be accompanied by a definition of each kind of text.
 Narrative: texts that tell a story or give the order in which things happen.
 Expository: texts that provide a factual description of things or people or explain how
things work or why things happen.
 Documents: tables, charts, diagrams, lists, maps.
Acronyms and abbreviations should always be spelled out in the questionnaire. Do not
assume that respondents will or should know what an acronym represents.
When a general term is used, concrete examples should be given to clarify its meaning. For
example, a question on learning activities included in the International Educational Achievement
(IEA) Reading Literacy Teacher Questionnaire included the following items, for which the
respondent had to answer on a four-point frequency scale.
How often are your students typically involved in the following activities?
• silent reading in class
• learning new vocabulary systematically (for example, from lists)
• learning to use illustrations (for example, graphs, diagrams, tables)
The words ‘systematically’ and ‘illustrations’ were too general to be understood in the same
way by all respondents. Examples were therefore provided to clarify their intended meaning.
Finally, it is recommended to avoid words that may have an ambiguous meaning. In
education, the word ‘hour’ may have different meanings. For example, many education systems
refer to a lesson length or period as an hour even though the lesson is only forty-five minutes
long. In order to measure the yearly instructional time at a given educational level, it is therefore
necessary to know the length (in minutes) of an ‘hour’ of instruction, the number of minutes of
instruction per week, and the number of school weeks per year. If this information is known,
then calculations can be made later for instructional time per day, or week, or year.
2.2.2. Keep the question short
Closely related to keeping vocabulary simple is avoiding lengthy questions. Generally, it is
recommended to hold questions to 25 words or less. If a longer sentence is used then it should be
broken up so that there will be several shorter sentences.
2.2.3. Avoid double-barrelled questions
These are single questions that ask for two things and therefore require two answers. “Do you
have your own table or your own room to do your homework?” “Do you think it is good idea for
children to study geography and history in primary school?” In such instances, respondents do
not know what to do if they want to say ‘Yes’ to one part of the question but ‘No’ to the other.
2.2.4. Avoid hypothetical questions
Evidence has shown that hypothetical questions such as “Would you use this resource in your
class if it were available?” are not good for the prediction of behaviour. People are generally
poor predictors of their own behaviour because of changing circumstances and because so many
situational variables intervene. Investigators are able to collect more valid data if they question
respondents’ about their past behaviour and present circumstances, attitudes, and opinions.
2.2.5. Don’t overtax the respondent’s memory
It is risky to ask the respondent to recall past behaviour over a long retrospective period. This
is true especially when recurrent events or behaviours are concerned. No student, especially
young students, will be able to answer reliably a question such as “In the last month how many
hours of homework did you do on an average day?” because the time is just too long to
remember what happened in detail. If such a question must be asked, a one-week recall period
might be more appropriate for this type of event.
2.2.6. Avoid double negatives
Double negatives, either in the question or an answer category (or both), create difficulties for
the respondent. For example a statement such as ‘Student self-evaluation should not be allowed’
followed by agree/disagree is problematic to answer for respondents who are in favour of
students’ self-evaluation, that is those who do not agree that students’ self evaluation should not
be allowed. It is usually possible to solve problems of this kind by formulating the initial
statement in a positive way.

2.2.7. Avoid overlapping response categories

Answer categories should be mutually exclusive. It should not be possible to agree with or
choose more than one category – unless the instructions explicitly allow the respondent to check
more than one alternative. Examples of questions with overlapping categories are:
Do teachers generally receive their salaries: (Check one only)
usually on time ……………...1
sometimes a week late ………2
more than a week late ……….3

How old are you?

under 20……………1
20-30 ……………... 2
30-40 ……………... 3
40-50 ……………... 4
50-60 ……………... 5
60 or more …………6

The categories of the first question could be made mutually exclusive by removing the
qualifiers ‘usually’ and ‘sometimes’. In order to avoid overlap in the second question it should be
modified as follows.

How old are you?

under 20……………1
21-30 ……………... 2
31-40 ……………... 3
41-50 ……………... 4
51-60 ……………... 5
61 or more …………6

2.2.8. Beware of ‘leading’ questions

A leading question is a question phrased in such a way that it seems to the respondent that a
particular answer is expected. For example:
 “Do you favour or oppose school on Saturday morning?”
might read in a leading question as:
“You wouldn’t say that you were in favour of school on Saturday morning, would
you?”,
or in a more subtle form:
“Would you say that you are not in favour of school on Saturday morning?”

2.2.9. Examples of bad questions and ways to correct

Double Barreled question:
Please indicate how much you agree or disagree with each of the following statements about
the childcare program.
Incorrect:
I feel welcomed by staff and other youth at the center
Correction:
I feel welcomed by staff at the center I feel welcomed by other youth at the center
Biased/leading question
Incorrect:
Community organizing is hard. Do leadership trainings help you feel prepared for community
organizing?
o Much more prepared
o Somewhat more prepared
o Slightly more prepared
o Not more prepared
Correct:
The leadership trainings prepare me for community organizing.
Strongly agree Agree Disagree Strongly disagree
Double Negative
Incorrect
Does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews
never happened? (22%)
Very possible Possible Impossible Very impossible
Correct
Do you doubt that the Holocaust actually happened or not? (9%)
Very possible Possible Impossible Very impossible
2.3. A checklist for reviewing questionnaire items
The following list of questions provides a framework for reviewing each item that is to be
included in a questionnaire.
• Will the item provide data in the format required by the research questions or the hypotheses?
• Is the item unbiased?
• Will the item generate data at the level of measurement required for the analysis?
• Is there a strong likelihood that most respondents will answer the item truthfully?
• Do most respondents possess sufficient knowledge to answer the item?
• Will most respondents be willing to answer the item, or is it too threatening or too
sensitive?
• Does the item avoid ‘leading’ respondents to a specific answer?
• Is the language used in the questionnaire clear and simple – so that all respondents are able
to understand all of the questions?
Practice 1

1. Explain the uses of closed, open, and contingency questions.

2. Draft five closed and open questions related to some aspect of educational research.
3. Formulate a contingency question with accompanying instructions.
4. The following are additional examples of bad survey questions. As a group, apply our class
discussion, evaluating each question and its response choices to identify one error in each
question (ie. double barreled). Explain the problem that your group identifies in each question
and propose an alternative that solves the problem.

Question 1. More people have attended the movie, Gone with the Wind, than any other motion
picture produced this century. Have you seen this movie?
Yes No
Question 2. In your opinion, how would you rate the speed and accuracy of your work?
Excellent Good Fair Poor
Question 3. Did you first hear about the bombing:
______ from a friend or relative
______ from your spouse
______ at work
______ from a newspaper
______ from the television or radio or other electronic media
Question 4. How do you feel about the following statement? We should not reduce military
spending.
Strongly agree Agree Disagree Strongly disagree
Question 5. People grow up in all different types of families. What type of family did you grow
up in?
______Mom as single parent
______Dad as a single parent
______Both Mom and Dad
Question 6. Where do you get most or all of your information about current events in the nation
and the world? __Radio __Newspapers __Magazines __Internet
Question 7. Higher than single-family density is acceptable in order to make housing affordable.
Strongly Undecided Disagree Strongly Disagree
Agree Agree
1 2 3 4 5

3. FINALIZING A QUESTIONNAIRE
3.1. Sequence of the questionnaire
Two widely-used patterns of question sequence in questionnaire design have been found to
motivate respondents to co-operate and fully complete a questionnaire. They are called the
funnel sequence and the inverted funnel sequence.
The characteristic of the funnel sequence is that each question is related to the previous
question and has a progressively narrower scope. The first question can be either open format, or
multiple choice. It should be very broad, and is used to ascertain something about the
respondent’s frame of reference on a topic. This ordering pattern is particularly useful when
there is a need to prevent further specific questions from biasing the initial overall view of the
respondent.
• Would you say that the general quality of education provided by primary schools in
your community is:
o very good
o good
o uncertain
o bad
o very bad
 • How would you rate the overall quality of the primary school attended by your
child?
o very good
o good
o uncertain
o bad
o very bad
• Do you think your own child is receiving a good primary school education?
o Yes
o No
• Given the opportunity, would you have your child attend another school in your
area?
o Yes
o No

In the inverted funnel sequence, specific questions on a topic are asked first, and these
eventually lead to a more general question. This sequence requires the respondent to think
through his or her attitude before reaching an overall evaluation on the more general question.
Such a question order is particularly appropriate when there is reason to believe that respondents
have neither a strong feeling about a topic, nor a previously formulated view.
The placement of items in a questionnaire requires careful consideration. Good item
placement can increase the motivation of respondents – which in turn results in more valid data.
General guidelines for item placement
• Non-sensitive demographic questions should be placed at the beginning of the
questionnaire because they are easy to answer, non-threatening, and tend to put the respondent at
ease.
• Items of major interest to the research study should be placed next since there is greater
probability of the respondent answering or completing the first section of the questionnaire.
• Sensitive items that cover controversial topics should be placed last so that potential
resentment that may be provoked by these items does not influence responses to other questions.
• Items on the same topic should be grouped together. However, care should also be taken to
prevent one item influencing responses to later items.
• Items with similar response formats should be grouped together when several different
response formats are being used within a questionnaire.
• Section titles should be used to help the respondent focus on the area of interest.
3.2. Introductory paragraph
In all questionnaire, before any questions, it is necessary to enclose an introductory paragraph.
This paragraph should do the followings:
• Identify the organization conducting the study (for example, the Ministry of Education);
• Explain the purpose of the study;
• Assure the respondent that information provided will be managed in a strictly confidential
manner and that all respondents will remain unidentified;

• Expression of appreciation for the respondent’s help.

The following questionnaire is part of an International study of Reading Literacy and
attempts to identify differences in English instruction. It is recognized that teachers are
likely to respond quite differently to the enclosed questions.
Please answer all questions in such a way as to reflect most clearly your teaching practices.
Most questions will require you to circle your selected response. Others will require you to
write down a number. Do not leave blanks.
We thank you for your contribution to this important research.

3.3. Drafting instructions for answering questions

Writing instructions for answering questions is a very important part of the questionnaire
layout. If the questionnaire is to be administered by an interviewer, then the instructions will be
addressed to him or her. Such instructions are usually written in capital letters, as follows.
Who was your employer on your last job
(PROBE FOR CORRECT CATEGORY)
o Private
o National Government
o City
o Self-employed
o Public, non profit
o Others (please specify) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ..
o Doesn’t know

In a mailed or self-administered questionnaire, it is very important to provide clear

instructions because there is no resource person to help clarify respondents’ queries. Instructions
can be for a single question or for a set of questions.

INSTRUCTIONS TO A RESPONDENT FOR A SET OF QUESTIONS

INSTRUCTIONS: For each of the following questions, please mark the answer that comes
closest to the way you feel about learning mathematics. There is no right or wrong answer.
Answer the questions in the order in which they appear on the paper. Thank you for your
co-operation.
 INSTRUCTIONS TO A RESPONDENT FOR A SINGLE QUESTION
About how many different teaching positions have you held during your life? (Count only
those teaching positions that you have held for at least one full academic year)
The following examples provide illustrations of different instructions given for the same
question. In the first example, the instructions relate to an interview. In the second example the
instructions relate to a self-administered questionnaire. Note that the question is multiple choice,
followed by an open ended contingency question.

SELF-ADMINISTERED FORMAT
1. Thinking about the government facilities provided for schools, do you think your
neighborhood gets better, about the same, or worse facilities than most other parts of the
city?
o Better 1 (answer 1A below)
o About the same 2
o Worse 3 (answer 1A below)
o Don’t know 8
1A. If better or worse: In your opinion, what do you think is the main reason why your
neighbourhood gets (better/worse)
facilities? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.4. Pre-testing the questionnaire

Pre-testing the questionnaire is an essential step before its completion. The purpose of the
pretest is to check question wording, and to obtain information on open-ended questions with a
view to designing a multiple choice format in the final questionnaire. Pre-testing has a number of
very important advantages.
• Provides information on possible ethical problems overlooked previously.
• Helps determine if the research questions or hypotheses are appropriate.
• Helps determine if the levels of measurement are appropriate for the selected variables.
• Provides a check that the population is appropriately defined.
• Provides information on the feasibility and the appropriateness of the sampling method.
• Helps determine sample size by allowing estimation of variance from the pre-test sample.
• Provides additional training for interviewers, instrument administrators, experimenters,
coders, and data editors.
• Helps determine the length of the questionnaire.
 After training the interviewers and questionnaire administrators, the next step in pre-testing is
to select a small pilot sample of respondents that covers the full range of characteristics of the
target population. In the field of education this usually implies that the pilot sample includes
appropriate gender balance and covers a range of richer/poorer and rural/urban communities.
Pre-testing should never be carried out on a ‘convenience sample’, (for example, the
researcher’s friends or family, or schools in one neighbourhood of the Capital city). For
interview questionnaires 50 interviews will provide solid material for verifying question
wording, sequencing, instructions and general quality of the instrument. However, larger samples
of around 200 are required to calculate various statistics such as discrimination co-efficients.
Note that even questions ‘borrowed’ from existing questionnaires need to be pre-tested to
ensure that they will work as required with the ‘new’ respondents. This is particularly the case
with questionnaires administered to schoolchildren and with questions that are translated from
other languages.
The first version of the pre-test questionnaire often contains considerably more questions than
the final questionnaire. This can be upsetting for the respondents – especially if many questions
are asked in an unstructured and open form so that the amount of time required to complete the
questionnaire is considerable. If absolutely necessary, the questionnaire could be divided in two
or three parts (of equal length and answering time) for the first tryout, so that each respondent
answers only a fraction of the questions. For each form at least 50 respondents should be asked
to participate. The information collected in this first pre-test should provide sufficient
information to produce a second version of the questionnaire for final pre-testing.
This second version of the questionnaire will then be administered in one single form in order
to further verify the functioning of the items and answer categories, as well as that of the
questionnaire overall structure, layout, and accompanying instructions.
This process of pre-testing has a number of goals:
• To reformulate or eliminate ambiguous or superfluous questions
• To provide material to design the answer categories for open questions that need to be closed
• To determine whether the questionnaire is balanced in its structure, and to discover whether
instructions were properly followed
Basic steps in pre-testing
1. Select a sample similar in socio-economic background and geographic location to the one
that will be used in the main study. This sample will not be included in the final survey. Make
sure you have a sufficient number of copies of the questionnaire for the pre-test.
2. Instruct interviewers or questionnaire administrators to note all respondents’ remarks
regarding instructions or question wording.
3. Administer the questionnaires.
4. Debrief the interviewers and check the results:
• Is each item producing the kind of information needed?
• What role is the item going to play in the proposed analysis?
• Are the questions meaningful to the respondents?
• Are respondents easily able to understand the items?
• Can respondents use the response format for each item?
• Did the interviewers feel that they were receiving valid information?
• Was the question order logical and did the interview flow smoothly?
• Did some parts of the questionnaire arouse suspicion?
• Did other parts of the questionnaire seem repetitive or boring?
• Were interviewers able to read the questions without difficulty?
• Were respondents able to follow all instructions?
• Was the questionnaire too long?
Practice 2

1. Explain the difference between funnel and inverted funnel sequences.

2. List three aims that a good introductory paragraph of a questionnaire should address.
3. State the main objectives of a trial-testing programme.