RESEARCH METHODS

WEEK 7
Chapter 4: DATA ANALYSIS
4.1 How To Analyze Survey Data
4.2 Interpret Survey Data
ANALYZING SURVEY DATA

Once collection is completed, survey data will need to be organized into a format.
Coding, Entering, Cleaning Data From Paper-based Surveys
- Online survey tools will allow for data from electronic questionnaires to be directly imported
into analysis software.
- Data from paper surveys will need to be entered into software packages (like Excel, Access, SPSS,
SAS, etc.) by hand.
ANALYZING SURVEY DATA
1. Coding, entering, cleaning:
1. Write an identification number on each survey (e.g., in the upper right hand
corner).
2. Code data. Assign a numerical value to all textual data (except open-ended data).
For example:
Yes = 1
No = 2
Keep track of all codes either in a codebook or using a coding key
3. Create a database spreadsheet
Use survey question/item numbers as labels for columns
 Use survey identification codes as labels for rows
 ANALYZING SURVEY DATA
4. Enter data
- Type in the ID number and coded survey data from each questionnaire
(one row of data for each questionnaire)
- Transcribe/type open-ended data into single cell  Be careful to type in
data accurately
 Related issues:
 If a question is unanswered, leave the cell blank.
 If two answers are selected for a question when only one was
requested, the data are invalid; treat as unanswered.
 If there are irregular markings of rating scale items (e.g., choosing two scale
points; marking between scale points), (a) leave the cell blank, or (b) treat
all such instances the same, enter an average rating or round up/round
down.
ANALYZING SURVEY DATA
5. Clean data (= check data for accuracy)
- If practical, go through each survey again and compare survey responses
with spreadsheet data entries
- Scan the data for impossible entries (e.g., an entry of 6 when responses
should range from 1-5). Calculate frequencies for expected values to
reveal anomalous entries. If there are errors, check the particular survey and
re-enter correct value.
- For large databases, initially check five percent of entered data. If errors
are found, check the remainder of the data.
6. Save and back-up data (tip: store data in multiple formats and in
different locations)
 ANALYZING SURVEY DATA
Analyzing Data from Closed-response Items
- When choosing from possible procedures, think about (and/or discuss with
decision makers) what information is needed and organize data so that
relevant interpretations can be made.
 Follow the following Analysis Procedures for Closed-response Survey Items:
1. Response rate (ti le phan hoi)
2. Descriptive statistics (thong ke mot ta)
3. Cross tabulation (bang cheo)
4. Inferential statistics (thong ke suy luan)
5. Item reliability (internal consistency) (do tin cay cua cau hoi)
 ANALYZING SURVEY DATA
1. Response rate
- Response rate shows the proportion of respondents in the sample or census that
submitted surveys.
- Response rate is calculated by dividing the number of surveys completed by the
number of surveys administered:
- Surveys sent to respondents = 200 (either a sample or a census)
- Completed surveys returned = 60
- 60 / 200 => Response rate = 30%
 If evaluation uses of survey data require generalization, excessively low response
rates create the possibility of a biased sample or census.
- Recommendations for minimum response rates are conventionally between 60% to
80%.
ANALYZING SURVEY DATA
Analyzing Data from Open-response Items
- Analyzing open-ended responses involves organizing textual data (e.g.,
single words, brief phrases, sentences, etc.) so that the ideas in the
information can be used toward answering evaluation questions.
ANALYZING SURVEY DATA
1. Categorization (i.e., coding)
- For large amounts of data, an analytic process is commonly required to search through, reduce,
and organize the information for interpretation.
- The process typically involves identifying recurrent ideas and patterns, etc., and grouping
these into analytical categories to aid interpretation (= coding).
- The key task in this type of analysis is creating the analytical categories.
- It can be done in two ways: either by
(a) establishing preset categories
OR: (b) by looking for emerging categories in the data itself.
ANALYZING SURVEY DATA
2. Applying preset categories (deductive analysis)
- Start with a set of preset categories  then search through the data looking
for those topics.
- The categories can come from theoretical research or anticipated
issues the researcher want to know about.
ANALYZING SURVEY DATA
3. Steps in categorization/coding
To illustrate, below is a simplified inductive analytical procedure.
1. Arrange textual data in a spreadsheet
- Many methods (and tools) for organizing text exist. One option is to use a
spreadsheet.
- List all the data from an open-response question in a single column, with one
comment per cell.
- Next, number each row; then, create another adjacent column for
coding/labeling.
 ANALYZING SURVEY DATA
2. Initial read-through of all responses
- Read through each response to get an overall sense of the data.
 Look for patterns, groupings, common issues, recurring ideas, etc.

3. Create categories
- Create categories for patterns of concepts and recurrent ideas (e.g., ‘’more speaking in class’’,- ‘’more
extra-curricular activities’’).
- Assign an abbreviation or identifying numeral to each category for sorting later (e.g. ‘’more
speaking in class’’ = ‘’MSC’’)  List these in a coding key.

4. Label individual responses with a category (or code)

- Read through the data again and label each comment with one or more of the identified categories/codes
from step 3.
- Assign at least one category to each response.
ANALYZING SURVEY DATA
5. Presenting survey data
Survey data can be presented in various ways to aid interpretation.
 Use GRAPH/CHART or TABLE
 Focus on the following issues: What do the data show? What is the main
message?
=> The most appropriate format
a. Graph/Chart (e.g., pie, bar, line, scatter plot, box plot)
b. Table
c. Representative quotes, images, etc.
 PIE GRAPH
- Use pie graphs to show parts of a whole as per.
Note:
- Avoid pie graphs if there are numerous categories (use bar graphs instead)
- Avoid 3-D and elaborate Crosshatching
- Use black-and-white formats to show different parts clearly
PIE CHARTS
If the Line Graph focuses on the trend, PIE CHART (the chart of trigger) focuses on the
distribution.
Pie chart : compare the ratio of different components in an object, for example, different studies
in 1 school, different professions ... as the English name of the round chart has pointed out, the
round chart is a piece of pie (pie) that is divided into a variety of pieces. Comparison will come
from the difference between the pieces. Of course, we will have a bigger piece and a smallest
piece, along with other pieces in the middle. In the round graph, the way we use most of them: A
occupied ...% of the total x
Vocabulary
* Proportions/ The proportion of
* Percentages/ The percentage of
* A large number of people
* Over a quarter of people
* A small minority
* A significant number of people
* Less than a fifth
Percentage vs Fraction
80%= four-fifths
75% = three-quarters
70% = seven in ten
65% = two-thirds
60% = three-fifths
55% = more than half
50% = half
45% = more than two fifths
40% = two-fifths
35% = more than a third
30% = less than a third
25% = a quarter
20% = a fifth
15% = less than a fifth
10% = one in ten
5% = one in twenty
Percentage = Qualifier
77% = just over three quarters
77% = approximately three quarters
49% = just under a half
49% = nearly a half
32% = almost a third
Percentage/ proportion / number / amount / majority / minority
75% – 85% = a very large majority
65% – 75% = a significant proportion
10% – 15% = a minority
5% = a very small proportion
The results of a survey into the most popular leisure
activities in the United States of America in 1999 and
2009
Questions:
* How many activities in each chart
* In the 2nd charts, are the activities the same?
* Is there any change in the popularity of activities in these 2 years?
Answer:
* There are 8 activities in each year
•7 of 8 activities at 2 years are the same
•There are some change in the popularity level of the activities
Introduction:
These two pie charts show the changes in popularity of different leisure activities in the United
States of America between 1999 and 2009. It is clearly seen that the most popular leisure
activities were almost the same in both periods, but there were a number of differences in
popularity between the various activities.
=> Select and analyze the most important information and this is not an easy task. To do this, try
to find the following information:
* Largest figures
* The smallest data
* Which figures have increased
* Which figures have decreased
* Which figures do not change
* Any new figure?
And after answering these questions, we get some information as follows:
* Walking is the most popular activity in both years
* Yoga is replaced by weightlifting
* Aerobics, jogging and cycling decrease in popularity
* Soccer and camping have not changed
Arrange ideas into paragraphs
--> consists of two paragraphs. When writing these passages, remember three things:
The main ideas should be mentioned first
•Collection of data with the same characteristics
•Present ideas in a logical structure
Paragraph 1
walking is much the biggest slice of the pie in both years
weightlifting is new and yoga has gone
It is clear that walking was the most popular activity in both 1999 and 2009 with around 30%
of Americans saying that they preferred it. Also, yoga was no longer among the preferred
activities in 2009, but weightlifting was chosen by 10% of people.
Paragraph 2:
- refer to the other activities in order of popularity in 2009 – soccer and swimming first, jogging
and aerobics last
- note the biggest changes – cycling, jogging and aerobics all big fallers, swimming the big riser.
PIE GRAPH
BAR GRAPH
- Use bar graphs to compare values across categories.
Used for comparing individual pieces to each other.
 Add horizontal lines to help readers compare the values more easily
(though make these unobtrusive/a lighter color).
- If longer descriptive labels are needed, use horizontal bars.
BAR GRAPH
BAR GRAPH
TABLE
LINE GRAPH
- Use line charts for a time series (or some other sequence), or
relationships between variables.
 to show trends and patterns (not exact quantitative values).
LINE GRAPH
 INTERPRETING SURVEY DATA

- Like analysis, data interpretation is a subjective process.

- Where possible judgments and conclusions should be achieved via dialogue and
consensus (consider involving more than one decision maker/primary intended user
in the interpretation process to avoid willful, or politically-driven, misinterpretation).
INTERPRETING SURVEY DATA
 Focus on the following issues to interpret data

1. Will any conclusions be drawn about the data, or will just the data be
reported?
2. If the data are to be interpreted, who will do the interpreting?
3. Whose perspective matters most? Who should be involved in data
interpretation? Why?
4. How will interpretation be checked? Is it necessary to bring in other
perspectives?
INTERPRETING SURVEY DATA
5. Is triangulation necessary? (Triangulation = looking at additional sources of
information to confirm what the survey is telling us.)
6. Are patterns found from open-response items and closed-response items
complementary or divergent
7. Who will draw implications? Who gets to make recommendations?
8. Are the interpretations and recommendations cogently evident from the
data and clearly articulated in understandable language for intended users ?
 Languages used in
REPORTING RESULTS
Referring back to the research aims or procedures -
close
The first set of questions aimed to …
To compare the difference between …
The purpose of Experiment 3 was to …
Simple statistical analysis was used to …
The next question asked the informants …
To assess X, the Y questionnaire was used.
Changes in X and Y were compared using …
Languages used in
REPORTING RESULTS
Regression analysis was used to predict the …
To distinguish between these two possibilities, …
The first set of analyses examined the impact of …
The correlation between X and Y was tested using …
T-tests were used to analyze the relationship between …
The average scores of X and Y were compared in order to …
In order to assess Z, repeated-measures ANOVAs were used.
Nine items on the questionnaire measured the extent to which …
Languages used in
REPORTING RESULTS
Referring to data in a table or chart - close

an overview of …
shows the experimental data on X.
Table 1 compares the summary statistics for …
Figure 1 presents the breakdown of X according to …
provides the intercorrelations among the nine measures of X.
the results obtained from the preliminary analysis of X.

The table below illustrates

The pie chart above shows some of the main characteristics of the …
The top half of the table shows the breakdown of …
The bottom half of the table shows

As shown in Figure 1,
As can be seen from the table
(above),
the X group reported significantly more Y than the other
From the graph above we can see
two groups.
that
I t can be seen from the data in
Table 1 that

are shown
The results of the correlational analysis are set out
in Table 1.
The themes identified in these responses are presented
in Figure 1.
The results obtained from the preliminary analysis of X can be compared
are summarised
Languages used in
REPORTING RESULTS
Highlighting significant data in a table or chart - close
What stands out in the table is …
Closer inspection of the table shows …
It is apparent from this table that very few …
The most interesting aspect of this graph is …
In Fig.10 there is a clear trend of decreasing …
What is striking about the figures in this table is …
What is interesting about the data in this table is that …
Languages used in
REPORTING RESULTS
The differences between X and Y are highlighted in Table 4.
From the chart, it can be seen that by far the greatest demand is for …
From this data, we can see that Study 2 resulted in the lowest value of …
This table is quite revealing in several ways. First, unlike the other tables …
From the data in Figure 9, it is apparent that the length of time left between …
Data from this table can be compared with the data in Table 4.6 which shows …
As Table III shows, there is a significant difference (t = -2.15, p = 0.03) between the two groups.
Languages used in
REPORTING RESULTS
Stating a positive result - close
The mean score for X was …
Further analysis showed that …
Further statistical tests revealed …
A two-way ANOVA revealed that …
On average, Xs were shown to have …
Strong evidence of X was found when …
Languages used in
REPORTING RESULTS
This result is significant at the p = 0.05 level.
The results, as shown in Table 1, indicate that …
A positive correlation was found between X and Y.
There was a significant positive correlation between …
The difference between the X and Y groups was significant.
There was a significant difference between the two conditions …
Respondents who reported low levels of X also reported significantly lower levels of Y.
Languages used in
REPORTING RESULTS
Stating a negative result - close
No increase in X was detected.
No difference greater than X was observed.
No significant differences were found between …
None of these differences were statistically significant.
No significant difference between the two groups was evident.
No significant reduction in X was found compared with placebo.
No evidence was found for non-linear associations between X and Y.
No significant correlation was found between X scores and the Y scores (p = .274)
Languages used in
REPORTING RESULTS
X appeared to be unaffected by Y.
Only trace amounts of X were detected in …
There was no evidence that X has an influence on …
The Chi-square test did not show any significant differences between …
Overall, X did not affect males and females differently in these measures.
A clear benefit of X in the prevention of Y could not be identified in this analysis.
T-tests found no significant differences in mean scores on the X and Y subscales.
Languages used in
REPORTING RESULTS
Reporting positive and negative reactions - close
Stimulation of X cells with Y did not increase the …
With successive increases in intensity of the X, the Y moved further to …
Following the addition of X, a significant increase (P<0.05) in the Y was recorded.
When X cells were stimulated with Y, no significant difference in the number of Z was detected.
Languages used in
REPORTING RESULTS
Highlighting interesting or surprising results - close
I nterestingly, the X was observed to …
This result is somewhat counterintuitive.
I nterestingly, this correlation is related to …
The more surprising correlation is with the …
Surprisingly, only a minority of respondents …
The most surprising aspect of the data is in the …
The correlation between X and Y is interesting because …
The most striking result to emerge from the data is that …
I nterestingly, there were also differences in the ratios of …
The single most striking observation to emerge from the data comparison was …

surprising
significant
interesting result.
This is a/ an (rather)
remarkable outcome.
unexpected
disappointing
Languages used in
REPORTING RESULTS
Transition: moving to the next result - close
If we now turn to …
A comparison of the two results reveals …
Turning now to the experimental evidence on …
Comparing the two results, it can be seen that …
The next section of the survey was concerned with …
In the final part of the survey, respondents were asked …
Languages used in
REPORTING RESULTS
Summarising the results section - close
These results suggest that …
Overall, these results indicate that …
In summary, these results show that …
In summary, for the informants in this study, …
Together these results provide important insights into …
Taken together, these results suggest that there is an association between …
The results in this chapter indicate that … The next chapter, therefore, moves on to discuss the …
REQUIREMENTS FOR
CHAPTER 4
- In Chapter 4, analyze the data by taking the following steps as follow:
1. Group the themes stated in the Research Questions.
E.g: With the study investigating the “effects of the Internet on X…”, identify
the number of factors that have impact on X ( technological skill, teaching
methods, learning methods…)
2. Then, describe the data collected.  Focus on the most striking data.
3. Finally, relate to other studies for comparison in terms of research findings.
 Use scholarly support to back up your arguments. The arguments could
support or challenge ONE or a number of previously related studies.