509: DATA COLLECTION METHODS IN

THE FIELD OF EVALUATION

Topic: Unit 1: Approaches to Evaluation

Dennis K. Omvia
0752755276
[email protected]/[email protected]
 Overview
 This unit deals with the characteristic of
qualitative, quantitative and mixed methods.
Specifically the participants will learn about;
o The differences and similarities of these three
approaches
o The use of these approaches in evaluation study
o The research logic of these approaches
 Introduction
 The quality of an evaluation mainly depends on
the availability of exact information that is
adequate for answering the evaluation question.
Scientifically conducted evaluations make use of
social research methods.
In this context, there are different techniques
and tools available for collecting and analyzing
data for evaluation
 Introduction cont’
 Social research relies on different approaches to
gain relevant information:
1)Qualitative research approach
2)Quantitative research approach
o Note : that qualitative and quantitative research
methods do contradict each other and should be
seen as complementary, not antagonistic
 1. Qualitative Research
o Qualitative research is primarily exploratory
research. It is used to gain an understanding of
underlying reasons, opinions, and motivations. It
provides insight into the problem and helps to
develop ideas or hypotheses for potential
quantitative research.
o It is therefore a means of exploring and
understanding the meaning individuals or groups
ascribe to social or human problems
 1. Qualitative Research
o The process of research involves emerging
questions and procedures, data typically
collected in participants’ setting, data analysis
that inductively builds from particular to general
themes, and the research interpreting the data.
o The focus is on clarifying new phenomenon and
understanding context.
o Thus, qualitative research focuses on words and
uses open-ended questions and some what less
structured instruments for data collection.
 2 Quantitative Research
o Quantitative research is a means of testing
objectives theories by examining the
relationships between variables.
o These variables can be measured, typically on
instruments, so that numbered data can be
analyzed using statistical procedures.
o Quantitative research is used to quantify the
problem by a way of generating numerical data
or data that can be transformed into useable
statistics.
 2 Quantitative Research
o It is used to quantify attitudes, opinions,
behaviors, and other defined variables – and
generalize results from a larger sample
population.
o Quantitative research uses measurable data to
formulate facts and uncover patterns in research.
o Those who engage in this approach have
assumptions about testing theories deductively,
building in protections against bias, using control
mechanisms that allow for alternative
explanations, and being able to generalize and
 2 Quantitative Research
Replicate finding.
o Quantitative research focuses on numbers and
uses closed-ended questions and structured and
standardized data collection methods
 Conclusion: Quality Vs Quantity. E.g. qualitative
research referring to quality where problems are
answered without generally focusing on quantity, are
descriptions(in words) coming from interviews,
discussions or observations. However when words are
translated to quantity in order to describe or to
generalize, then the research is quantitative research
 Qualitative Versus Quantitative Research
Critical Qualitative Research Quantitative Research
1) Purpose To understand and interpret To test hypotheses, look at
social interaction cause and effect, and make
predictions

2) Group studied Smaller and not randomly Large and randomly selected
selected
3) Variables Study of the whole, not Specific variables studied
variables
4) Type of data Words, images or objects Numbers and statistics
collected
5) Form of data Qualitative data such as open- Quantitative data based on
collected ended responses, interviews, precise measurements using
participant observations, field structured and validated data
notes, and reflections collection instrument
6) Types of data analysis Identify patterns, patterns, Identify statistical relationships
features, themes
 Qualitative Versus Quantitative Research
Criteria Qualitative Research Quantitative Research
7) Objectivity & subjectivity Subjectivity is expected Objectivity is crucial
8) Result Particular or specialized Generalizable finding that can
findings that is less be applied to other
generalizable populations
9) Role of researcher Researcher & their biases may Researcher & their biases are
be known to participants in not known to participants in
the study, & participant the study, & participants
characteristics may be known characteristics are
to the researcher deliberately hidden from the
researcher
10) Scientific method Exploratory & bottom up: the Confirmatory or top-down:
researcher generates new the research tests the
hypotheses and theory from hypotheses and the theory
the data collected with the data
11) View of human behavior Dynamic, situational, social & Regular & predictable
personal
12)Most common research Explore, discover & construct Describe, explain & predict
objectives
Qualitative Versus Quantitative Research
Critical Qualitative Research Quantitative Research
13) Focus Wide-angle lens; examines Narrow-angle lens; tests a
the breadth and depth of specific hypotheses
phenomena

14) Nature of observation Study behavior in a natural Study behavior under

environment controlled conditions, isolate
causal effects

15) Nature of reality Multiple realities; subjective Single reality, objective

16) Final Report Narrative report with Statistical report with
contextual description & correlations, comparisons of
direct quotations from means and statistical
research participants significance of findings
 Logic of the Research Process
 both approaches follow a different logic of
research process. This explained in the following
1)Linear strategy:
o Quantitative research can be seen as linear, i.e.
taking one step after the other. The starting point
of a linear model lies in theories and models on
the basis of which cause and effect analysis are
conducted
 Logic of the Research Process
o The researcher forms hypotheses and operationalizes
indicators based on these theories. Afterwards, the
researcher chooses a sampling procedure and collect
data. In the context of linear research, it is important to
ensure representativeness, objectivity, and validity.
 Logic of the Research Process
2) Circular Research Strategy
o Qualitative research follows a circular strategy which
means that a particular succession of research steps is
repeated several times and that the current step always
depends on the results of the previous step.
o In most cases this means that there is only a vague.
understanding of the research object at the beginning of
the research process. Because of this, very few steps can
be planned in advance
Logic of the Research Process
 Advantages of Quantitative Research
o The advantage of quantitative approach is that it is
possible to measure the reactions of a large number of
people to a limited set of questions, thus facilitating the
comparison and statistical aggregation of the data.
o This provides a broad and generalizable set of findings
that can be presented in a succinct and economical way.
o Here validity depends on the careful construction of the
instrument, to ensure that the instrument measures
what it is supposed to measure
 Advantages of Qualitative Research
o The advantage of qualitative approach is that it
produces a wealth of detailed information about a much
smaller number of people. This increases the
understanding of cases and situation studied but also
reduces generalizability.
o Here, validity hinges to a great extent on the skills,
competence, and rigor of the person doing the field
research, as the researcher is seen as the actual
instrument
 3 Mixed Method
o The aim of every research study and evaluation is to
provide rich and unbiased information that can be
interpreted with comfortable degree of assurance.
o Researchers and evaluators strive for validity and
reliability, a comprehensive multi-perspective view, and
they look for procedures to decrease potential biases
within the research.
o This approach has different names, such as methods
triangulation or mixed method approach
o Methods can be combined into 2 exemplary ways; there
is within- method triangulation on one hand, and
between or across-method triangulation on the other.
 Mixed Methods
o Within-methods triangulation
o ………..means that the research/evaluation is based on at
least two data collection procedures from the same
research approach. From the quantitative perspective,
an evaluator can use a standard survey and pre existing
monitoring data. In qualitative approach, focus group
interviews can be combined with non participant
observation
 Mixed Methods
o Between – or across – methods triangulation
o ………refers to a combination of qualitative and
quantitative data collection methods in the same study
or evaluation. For example, one can use a standard
survey and conduct in depth interviews.
Rationale for mixed methods in
Evaluation
 Rationale for mixed methods in
Evaluation
o As shown above, qualitative and quantitative
approaches set different focuses. However, not
every evaluation question can be answered using
only one of these approaches.
o In other words, one has to apply different data
collection methods inorder to fully answer all
evaluation questions
 Rationale for mixed methods in
Evaluation
 Greene et al (1989) identified five reasons for
combining different approaches and or methods;
1)Triangulation. That is seeking convergence or
verification of results from different methods
studying the same phenomenon.
2)Complementarity. That is seeking elaboration,
enhancement, illustration, and clarification on
the results from one method with results from
the other methods
 Rationale for mixed methods in
Evaluation
3) Development: that is using the results from one
methods to help inform the other method.
4) Initiation: that is discovering paradoxes and
contradictions that lead to a reframing of the
evaluation question.
5) Expansion: that seeking to expand the breadth
and range of inquiry by using different methods
for different methods for different inquiry
component.
 Rationale for mixed methods in
Evaluation
o In conclusion, by mixing methods the
methodological weaknesses of one instrument
can be compensated by the strengths of another
instrument
End of Unit 1
 Questions 1
1) From the knowledge you have acquired in the field of
research and your own experience, explain the
following;
a) Qualitative research approach
b) Quantitative research approach
c) Mixed method
The END