GUIDANCE NOTE ON

QUALITATIVE RESEARCH
IN EDUCATION:
Considerations for Best Practice
In order to develop education policies, structures and systems that create sustainable development,
it becomes even more important than ever to understand what works, and in particular, in which
contexts. Qualitative research methods play an important role in program evaluation, especially
with a focus on research contextualization, but often they are considered “second class” when
compared to quantitative evaluations. When researchers want to know ‘what works’, quantitative
methods are commonly selected instead of qualitative methods. However, without good qualitative
data to contextualize these findings, ‘how or why things work’ can often remain obscured. This
guidance note addresses this gap to help commissioners of research and researchers design and
implement qualitative research that use a high level of rigor.

The BE2 Steering Committee*

February, 2020

Cover Design:

This document should be cited as follows:

DeJaeghere, J., Morrow, V., Richardson, D., Schowengerdt, B., Hinton, R., Muñoz Boudet, A. 2020. Guidance
Note on Qualitative Research in Education: Considerations for Best Practice. London, England: United Kingdom.
Department for International Development, prepared for Building Evidence in Education (BE2).

* The Building Evidence in Education (BE2) working group is led by a Steering Committee composed of the
Department for International Development (DFID), United States Agency for International Development (USAID),
The World Bank Group and a UN agency, currently the United Nations Educational, Scientific and Cultural
Organization (UNESCO).

ii
Guidance Note on Qualitative Research in
Education: Considerations for Best Practice

Foreword
The Building Evidence in Education (BE2) donor working group was launched in 2012 with the
aim to engage bilateral and multilateral donors and foundations committed to:
 Strengthening donor research collaboration and coordination;
 Encouraging higher standards of commissioned research; and
 Promoting the availability and access to rigorous evidence.

The working group is led by a Steering Committee composed of the Department for International
Development (DFID), United States Agency for International Development (USAID), The World
Bank Group and a rotating representative of the United Nations (UN) organizations, currently the
United Nations Educational, Scientific and Cultural Organization (UNESCO).

This series of Guidance Notes, prepared for the BE2 working group by its members, provides tools
and guidance for generating better evidence and leveraging existing evidence more effectively and
efficiently. These Guidance Notes have benefited from the advice of BE2 member organizations
and are intended to serve as tools for researchers and commissioners of research.

This guidance note has been authored by Joan DeJaeghere, Professor, University of Minnesota, Virginia Morrow,
Research Associate, University of Oxford, Dominic Richardson, Chief, Social Policy and Economic Analysis, and
formerly Senior Education Specialist, UNICEF Office of Research-Innocenti, Bethany Schowengerdt, University of
Minnesota, Rachel Hinton, Head of Education Research Team, DFID, and Ana María Muñoz Boudet, Senior Social
Scientist, The World Bank, for the BE2 working group.

Many organizations and practitioners have provided input, e.g., during working sessions at conferences or other
feedback opportunities. BE2 thanks all its members and other contributors for comments provided to drafts of this
guidance note, in particular Bina D’Costa (formerly UNICEF Office of Research – Innocenti), and the education unit
at UNICEF Office of Research – Innocenti, who provided comments on early drafts of the note.

iii
This page intentionally left blank.

iv
 Contents
Contents ........................................................................................................................................................ v
List of Tables .............................................................................................................................................. vii
List of Figures ............................................................................................................................................. vii
List of Boxes ............................................................................................................................................... vii
Introduction ................................................................................................................................................... 1
Section 1: A shared understanding of qualitative research ........................................................................... 4
1.1 Purposes and contributions of qualitative research ......................................................................... 4
1.2 Transparency of methodological choices........................................................................................ 7
Section 2: Processes for designing qualitative research ................................................................................ 8
2.1 Paradigms and methodologies ...................................................................................................... 12
2.1.1 Paradigms .................................................................................................................................. 12
2.2 Methodologies in qualitative research .......................................................................................... 13
2.2.1. Phenomenology and hermeneutics............................................................................................ 13
2.2.2. Ethnography .............................................................................................................................. 14
2.2.3. Qualitative longitudinal research (QLR) ................................................................................... 15
2.2.4. Life histories, oral histories, narratives ..................................................................................... 16
2.2.5. Case studies ............................................................................................................................... 17
2.2.6. Historical and policy analysis ................................................................................................... 17
2.2.7. Participatory approaches ........................................................................................................... 18
2.2.8. Critical discourse analysis ......................................................................................................... 19
2.2.9. Critical realist evaluation/research ............................................................................................ 20
Section 3: Methods in qualitative research ................................................................................................. 21
3.1. Interviews ..................................................................................................................................... 22
3.1.1 Individual and group interviews ................................................................................................ 22
3.1.2 Unstructured to structured interviews ........................................................................................ 25
3.1.3 Single or multiple interviews ..................................................................................................... 25
3.1.4 Informal to formal ...................................................................................................................... 26
3.2. Observations ................................................................................................................................ 26
3.2.1 Role of the observer ................................................................................................................... 27
3.2.2 Duration and frequency of the observations .............................................................................. 27
3.2.3 Focus of the observations........................................................................................................... 28

v
 3.3. Documents/artifacts ..................................................................................................................... 29
3.4. Participatory and arts-based research methods ............................................................................ 30
Section 4: Analysis approach—the final step when conceptualizing a study design.................................. 33
4.1 Content and thematic analysis ...................................................................................................... 33
4.2 Constant comparative analysis...................................................................................................... 34
4.3. Process tracing and contribution analysis .................................................................................... 35
4.4. Qualitative longitudinal analysis ................................................................................................. 36
4.5. Critical discourse analysis ........................................................................................................... 37
Section 5: Implementing rigorous qualitative research ............................................................................... 38
5.1. Designing protocols ..................................................................................................................... 39
5.2. Sources and sampling .................................................................................................................. 40
5.3. Data collection ............................................................................................................................. 46
5.4. Data management and transcription............................................................................................. 49
5.5. Data coding and analysis ............................................................................................................. 51
Section 6: Ethics related to qualitative research ......................................................................................... 55
6.1 The purpose and methods of the research ..................................................................................... 55
6.2 Assessing harms and benefits ....................................................................................................... 56
6.3 Privacy and confidentiality ........................................................................................................... 56
6.4 Selection and participation............................................................................................................ 57
6.5 Financial considerations relating to time, compensation .............................................................. 57
6.6 Reviewing the research—institutional review boards and research ethics committees ................ 57
6.7 Information ................................................................................................................................... 58
6.8 Consent ......................................................................................................................................... 58
6.9 Dissemination and implementation of findings ............................................................................ 59
6.10 Impact of the research on wider groups ...................................................................................... 59
Section 7: Identifying reliability and validity in qualitative research ......................................................... 61
References ................................................................................................................................................... 63
Appendix 1: Checklist to assess quality in qualitative research ................................................................. 75

vi
 List of Tables
Table 1: Contributions of qualitative research ................................................................................ 5
Table 2: Examples of questions that can be answered with qualitative methodologies ............... 20
Table 3: Distinctions in using group interviews and focus groups ............................................... 23
Table 4: Types of sampling in qualitative studies ........................................................................ 42

List of Figures
Figure 1: Process for designing rigorous qualitative studies .......................................................... 9
Figure 2: Theoretical aspects of methodology and method .......................................................... 12
Figure 3: Different paradigms and methodologies of qualitative research ................................... 13
Figure 4. Conceptualizing a study: example of the connection between the research question and
analysis approaches....................................................................................................................... 34
Figure 5: Iterations of constant comparative analysis................................................................... 35
Figure 6: From questions to categories using grounded theory. An example. ............................. 36
Figure 7: Processes for qualitative data gathering and analysis ................................................... 38
Figure 8: Processes for qualitative data gathering and analysis- protocols preparation ............... 39
Figure 9: Processes for qualitative data gathering and analysis - sources and sampling .............. 40
Figure 10: Processes for qualitative data gathering and analysis – data collection ...................... 46
Figure 11: Processes for qualitative data gathering and analysis – data management ................. 49
Figure 9: Processes for qualitative data gathering and analysis – data coding and analysis ........ 51

List of Boxes
Box 1: Thick Description: The value of details .............................................................................. 6
Box 2: Putting it all together: An example .................................................................................. 11
Box 3: Example of an observation description ............................................................................. 29
Box 4: A note on data saturation................................................................................................... 41
Box 5: Challenges of attrition ....................................................................................................... 46
Box 6: The Data Management Plan (DMP) Checklist of the Qualitative Data Repository ......... 51
Box 7: Systematic analysis of qualitative data: Young Lives....................................................... 53

vii
This page intentionally left blank.

viii
 Introduction
Current issues in education research
In recent years, the increased attention from both the international community and national actors
to solve the global learning crisis has been matched by a greater demand for research. This demand
has led to a wealth of studies that seek to understand “what works” in improving learning
outcomes. Qualitative research has been particularly important to shed light on “why,” and under
what circumstances, interventions or policies succeed or fail. But, with increased resources for
research comes the responsibility to ensure the research is of the highest quality, a “gold standard.”
To this end, this guidance note covers how to design and operationalize high-quality research using
qualitative methods.
The current state of evidence is mixed, and the proliferation of research has not always added up
to a body of knowledge, making systematic reviews difficult. An increasing number of high-
quality randomized control trials, which take an experimental approach to test whether an
intervention has an impact, is being produced. Yet, taken out of the specific context, the results are
rarely replicated. This “existential crisis” in the sector (George 2019) has led to a call for
qualitative research that can provide rich contextual data, including political economy analysis,
and shed light on the enabling environment and system in which interventions operate
successfully. It has also highlighted the wider system failures, which in themselves require the
toolkit of the qualitative researcher.
So, how can we fill the gap in high-quality qualitative research? The challenge is one of both
demand and supply. There is a growing recognition that country governments and stakeholders
facing challenges in delivering the education Sustainable Development Goals (SDGs) need to be
at the forefront of both demanding evidence and identifying research questions. In addition, those
commissioning research must make smart investments, which requires an understanding of the
main principles and processes for conducting diverse forms of qualitative and quantitative
research. One challenge is that researchers privilege methods dominant in their own disciplines;
another is that time and resources are needed for genuinely mixed-methods approaches. The
education sector has suffered from a lack of resources for academic capacity building in low-
income countries, leading to a scarcity of experts with contextual knowledge. In addition,
understanding the limitations and generalizability of studies will also help decision makers avoid
misinterpretations and improve the application of evidence. The Impact Initiative has examined
the use and uptake of knowledge across a range of country contexts and highlights the need for
ownership of the research by policy makers from the outset (Impact Initiative 2019). As the body
of knowledge grows, stakeholders (consumers, producers, and funders of such research) will need
the tools to evaluate the quality of evidence presented, as we have witnessed from the demand for
the Building Evidence in Education global group guidance on “Assessing the Strength of Evidence
in the Education Sector.” (BE2 2015) 1

1
Hinton, R. 2015. Assessing the Strength of Evidence in the Education Sector. London, England: United Kingdom
1
 About this guidance note
A core principle of this note is that research questions should guide the choice of research
methodology, not the other way around. Increasingly, mixed-methods research, in which both
qualitative and quantitative research methods work together, is considered the most appropriate
approach. If we understand quantitative and qualitative at opposite ends of a research continuum
(Creswell 2015), privileging either approach or discussing which one is better focuses on the
wrong question. As such, this note aims to equip those using, commissioning, and conducting
research in the education sector with the tools and information to assess when qualitative
methodologies can and should be used and how to evaluate the quality of such research from its
early design stages through implementation, data analysis, and dissemination. All stakeholders
need to be confident in their knowledge of the main features of a well-designed qualitative research
process.
For example, in the education sector, quantifiable differences in children's household economic
situation or parental attitudes regarding secondary school completion reveal only one part of a
complex picture. Qualitative studies can pose questions that challenge assumptions in a way that
allows researchers to arrive at identifiable differences or guide responses to these differences.
Rather than starting from parental attitudes or a household economic situation, a qualitative study
might ask, “What does secondary education mean for various people in this context?” or “What
experiences inside and outside of school seem to affect completion and why?” Findings from these
(and other) lines of questioning can inform the design of policies and programs that acknowledge
and work within complex systems.
Throughout this note, several examples of qualitative studies will be presented. The Young Lives
study 2 by Oxford University and Voices of the Poor by the World Bank represent two of the
largest-scale efforts to collect qualitative data (alongside survey data) to inform policies at a cross-
national level. 3 In the case of “Young Lives,” which focuses on reducing child poverty, qualitative
data gathered with children, parents, and teachers have provided critical insights on their
experiences, perceptions, and behaviors with regards to education in the countries of study.
Qualitative insights around experiences, perceptions, and behaviors are important and valuable
evidence for informing policy decisions in education and, therefore, should be derived from high-
quality studies. To help build high-quality qualitative evidence, this note is a resource to be used
when discussing and designing qualitative research. It contains and explains key terminology,
approaches, and methodologies used in qualitative research, as well as the fundamentals of the
underlying criteria of high-quality studies.

Department for International Development, prepared for Building Evidence in Education (BE2). Accessible here:
https://www.usaid.gov/documents/1865/assessing-strength-evidence-education-sector.
2
For further information see www.younglives.org.uk.
3
Other large qualitative projects that had an impact on the understanding of poverty include “Voices of the Poor” by
the World Bank (see D. Naraya, et al. 2000).

2
This guidance note presents examples of qualitative research on education, privileging those from
low and middle-income countries when available; however, the content of the note is not exclusive
to research in one sector or income level, and the main principles and processes outlined apply
across research topics and locations. 4 The research examples provided in this note meet the criteria
for strong evidence (BE2 2015) and present evidence-based guidance based on qualitative research.
These studies included at minimum: a solid explanation of the study purpose and the methodology
used; a detailed account of the data collection process, including sampling and/or selection of
participants; an explanation of the analysis process and how data were analyzed to arrive at
findings; and findings that are supported by detailed description of data.
The guidance note is organized as follows:
• Section 1 aims to set a common understanding of what qualitative research is and when it can
provide the most value. It highlights the importance of being transparent about the choice of a
methodology.
• Section 2 outlines the process for the design phase of various types of qualitative research.
• Section 3 describes different qualitative research methods.
• Section 4 outlines the analysis approach.
• Section 5 describes the implementation phase of conducting the study, covering the main areas of
protocol design, sampling, and data analysis.
• Section 6 discusses critical issues related to guidelines and ethics for qualitative research.
• Section 7 summarizes considerations of reliability and validity interpreted for a qualitative context.
• The appendix includes a checklist to assess the quality of qualitative research summarizing many
of the aspects covered at length in the note (see Appendix 1).

4
Most of the aspects of high-quality research discussed in the note align with BE2’s guidance note on Assessing the
Strength of Evidence in the Education Sector, and the seven principles to assess evidence are included (the seven
principles include: conceptual framing, openness and transparency, robustness of methodology, cultural
appropriateness/sensitivity, validity [or in the case of qualitative studies: credibility], reliability [replicability] and
cogency).

3
 Section 1: A shared understanding of qualitative research
1.1 Purposes and contributions of qualitative research

What comes to mind when many people think of qualitative research is the type of data it generates:
narratives, quotes, life histories, group discussion notes, or detailed ethnographic accounts of the
life and behaviors of a specific community or group. These are powerful products to communicate
the problems and solutions for education delivery in discussion with policy makers. Personal
accounts may resonate at a psychological level with individual values, increasing the likelihood of
evidence uptake. Profound behavioral changes are needed, beyond the nudge approach (Thaler and
Sunstein 2009), to achieve shifts in government outcomes to deliver the educational SDGs.
Building on insights from cognitive and behavioral sciences, the increasingly influential field of
behavioral economics stresses the importance of identities, narratives, and norms in explaining
human behavior that departs from standard economic “rationality” (Collier 2016); qualitative
research is needed to gather the required data.
Qualitative research is critical to understanding the enabling environment for education services.
Specifically, it can help us understand the gap between policy and practice. Strong systems
diagnostics and an understanding of the socio-cultural context can increase both the choice and
fidelity of interventions. With the increased concern about isomorphic mimicry—or the tendency
of governments to mimic other governments' successes—qualitative evidence is needed to reduce
the current failures in replication and scaling. As accelerating social change increases the
complexity of development contexts, the “ground-truthing” of realities through qualitative
methods becomes increasingly important (Chambers 2017, p. 156). A qualitative lens can expose
the differences between how things are supposed to work de jure rather than the de facto
structures—which is crucial for implementation at scale. One stark example is the continued
significant investment in training that is measured by trainee attendance or, at best, the skills
learned rather than the skills applied once back in the classroom. A comparative examination of
investment in training by Popova, et al. (2018) shows that the majority of training does not an
impact. Yet a systematic body of knowledge of “why” teachers remain poorly motivated and how
best to support the acquisition and application of skills has yet to be gathered. Without the deep
dives into the cultural and political context, interventions continue as if the de facto structures
deliver as planned, ignoring what happens in reality.
Whether qualitative research in education focuses on educational resources or language of
instruction in the classroom or draws on interview or observational data, what unifies such research
is its attention to how humans relate to and make sense of their social and material world. Hence,
qualitative research engages with people’s experiences and views of the world in a given context
and moment in time. The table below summarizes areas in which qualitative research can
contribute both, as it is used alone or combined with quantitative research.

4
Table 1: Contributions of qualitative research

Qualitative research is useful to better Combined with quantitative research: 5

understand:

• Behaviors, emotions, experiences, and • Provides meaning and context to data.

one’s explanations of these.
• Can explain quantitative results
• Attitudes, opinions, and actions and beyond the numbers.
how people justify them.
• Helps to develop surveys, intervention
• Perceptions of phenomena. materials, and processes.
• Identifying unanticipated influences • Helps to develop new hypotheses and
and outcomes. theories when there are none or the
existing ones are inappropriate or
• Culturally shared meaning.
inaccurate.
• Social structures and relationships.
• Provides causal explanations (beyond
• Processes and relations within systems proven causality)

When it comes to education, the purposes described above can be applied to a range of research
questions. These questions can be applied at different levels of educational systems—from the
macro-level of policy-making and data use to the meso-level of schools and how they function or
produce learning and the micro-level of students, teachers, and parents and how learning occurs
through individual interactions and engagement. Qualitative research can also be used at different
points in the policy process, such as understanding the antecedents and underlying causes of an
educational problem before it is addressed, or capturing how the implementation process is going
or understanding why an intervention did or did not work the way it was planned.
Much of qualitative research explores and generates, rather than tests, hypotheses; as such, it will
often depart from a pre-identified set of assumptions or models of reality. Rather, it tends to
identify and generate explanations from the data and the data collection process in a flexible and
iterative process, as findings can and do affect the research design and direction and can challenge
initial research assumptions or ideas. 6 Even in cross-country or longitudinal research, when

5
This guidance note does not go into detail about mixed-methods, though the reader should note that many of the
approaches discussed throughout the note can be used in mixed-methods studies. For more in-depth discussions on
mixed-methods approaches, see Creswell and Clark 2007; Teddlie and Tashakkori 2009; and Bamberger 2000.
6
Iteration—revising and revisiting as research progresses—is central to all exploratory analysis. This does not mean
that iteration is unique to qualitative research, nor that there is no qualitative research that departs from a priori
concepts or theories. The distinction between quantitative and qualitative is more of a continuum than a binary, with
5
comparability of data is expected, a qualitative approach will encourage degrees of freedom to
respond to priorities and findings in each location or moment that lead to locally relevant
explanations and theories (see for example the Young Lives approach to qualitative research in
four countries described in Crivello, Morrow, and Wilson 2013).
Similarly, qualitative research’s attention to contextually specific knowledge calls for research to
be situated within the social, political, cultural, and economic context, and to be conducted in a
culturally appropriate way. As such, researchers must be knowledgeable about and aware of their
situation within the context of the research. For ethnographic research, this would demand fluency
in the local language and the adoption of core cultural dress and conduct. The ethnographer would
be required to build trust and integrate sufficiently to become a participant observer (Brewer 2000).
The range of qualitative research methods available provides great flexibility in the type of
evidence that can be gathered. Thick description, for example, provides valuable detail (see Box
1). In general, the emphasis in qualitative research on processes over measurement and analysis of
causal relationships (Maxwell 2013) can be helpful.

Box 1: Thick Description: The value of details

Although most commonly found in ethnography, thick description is a feature of much of

qualitative research. Thick description involves accurately describing and interpreting social
actions within the context in which the social action took place (Ponterotto 2016).

The term was first used by Ryle (1949), who distinguished between “thin” and “thick
description,” with thin being more like a photograph and thick being akin to the photographer
narrating the context and content of the image. The anthropologist Clifford Geertz (1973)
applied Ryle’s “thick description” to describe his own work, which used participant observation,
saying: “really our own constructions of other people’s constructions of what they and their
compatriots are up to” (p. 9). Since Geertz, the concept has been applied as a tool for qualitative
research in sociology, psychology, and education studies.
Sociologist Norman K. Denzin (1989) highlights the following features of thick description, in
contrast to thin description: “(1) It gives the context of an act; (2) it states the intentions and
meanings that organize the action; (3) it traces the evolution and development of the act; (4) it
presents the action as a text that can then be interpreted. A thin description simply reports facts,
independent of intentions or the circumstances that surround an action” (p. 33). Thick
descriptions also enable the reader to evaluate the extent to which the conclusions of a study are
drawn from context-specific aspects of a situation or whether they are transferable to other times,
settings, situations, and people (Lincoln and Guba 1985, p 124-5).

each research methodology displaying, on occasion, characteristics of the other (e.g., qualitative research can be
quantified or use more standardized instruments, and quantitative research can include flexible data collection
processes and rely on bottom-up explanations) (See Brady 2004, Mahoney and Goertz 2006, and Ragin 2014 for more
details on the distinctions between qualitative and quantitative research).

6
 1.2 Transparency of methodological choices

It is important to be clear about what qualitative research can and cannot do. Qualitative
researchers do not generalize particular findings, which can never be fully extricated from their
place and time, but they do develop theoretical and practical insights into how the social world
may work under particular conditions. In contrast, quantitative researchers seek “power” in large
sample sizes, aiming to generalize by controlling for factors that differ across contexts. This
enables the study of far greater extents of space and time. The failure to replicate quantitative
studies in different contexts, however, suggests a tendency to underestimate the complexity of how
contextual factors can influence measured variables. 7 Reliance on “hard” quantitative methods
and the subsequent failure to achieve externally valid results has contributed to the “existential
crisis” noted in the introduction.
Another feature of qualitative research is the degree to which the interpretation and analysis of
data depend on the particular theory the researcher has chosen to use. This means the model of
testing hypotheses against data, common in quantitative research, may not be an appropriate
measure of rigor in qualitative research. Although professional judgment is often relied on more
heavily, there usually exists a far greater opportunity to test theoretical assumptions because the
density of evidence in qualitative settings provides greater scope for falsification. Experienced
qualitative researchers commonly report having to significantly alter their theoretical assumptions
having spent time in the field. 8

7
See Pritchett (2017) for a discussion of these issues.
8
See Section 5 below for approaches to rigor specific to qualitative research. See Building Evidence in Education
(BE2) (2015) for how the quality of both qualitative and quantitative evidence may be assessed in terms of validity,
reliability, and cogency.

7
 Section 2: Processes for designing qualitative research
A common mistake when thinking of conducting qualitative research is starting with ideas about
methods (e.g., focus groups, interviews, or classroom observations) before discussing and deciding
on the overall purpose and related assumptions of the research. The purpose and assumptions
should inform the selection of methodologies and methods. Identifying and agreeing on
assumptions, purposes, and research goals before and while deciding on the data collection tools
are good practices in qualitative research. Qualitative research design is a process of
conceptualization that flows from the research purpose to its findings. We use the term
“conceptualize” because a design requires organization and justification to establish coherence
between the elements of the study and its relevance to the purpose and context. Researchers can
learn a great deal from existing studies, but conceptualization goes beyond picking out and
applying designs from a catalogue of options. Judging the quality of a qualitative study is, in part,
about evaluating how rigorously researchers conceptualized a design that establishes coherence
between research purpose, context, and assumptions about the world.
Figure 1 shows the process of designing a rigorous qualitative study, including 1) establishing a
purpose, 2) framing, 3) situating the questions and methodology, and 4) selecting methods and
planning data collection. The alignment of the purpose with a framework and methodology is
always situated within assumptions and knowledge about the world. Therefore, we need to be
reflexive about our ontological and epistemological assumptions before and during the study. As
shown in the figure, these assumptions undergird our framing and design. It is necessary to frame
a study before designing it, but that does not mean the work of framing is necessarily finished once
the design is chosen. Qualitative research is generally an iterative process, and the research design
can change in response to the data. Such adaptation is a sign of rigor if it allows us to gather more
meaningful data to answer our questions. When making design choices, it is necessary to consider
the assumptions that inform such choices. For example, if struggling between two methodologies,
it is wise to revisit key concepts in the frame to assess the merit of the methodologies in addressing
those concepts.
Ontology is the study of the kinds of entities that constitute reality. Debates between different
traditions of qualitative research often have an ontological dimension. For example, positivists 9
assert reality is made of entities that exist objectively as facts beyond the minds of actors. Social
constructivists take reality to be socially constructed, dependent on the meanings developed
intersubjectively within a specific culture. Critical realists argue for a “stratified” ontology that
enables causal claims to be made on behalf of structures and powers that may not be empirically
observed.

9
Positivism in the social sciences involves the claim that society, like nature, is governed by laws and that uncovering
these laws should be the goal of social research. This uncovering has primarily involved searching for statistical
regularities between variables. Many quantitative researchers, following Karl Popper, hold “postpositivist” views that
acknowledge the theory-dependence of their findings while upholding the scientificity of their approach.

8
 Figure 1: Process for designing rigorous qualitative studies

Establish purpose
Frame
Situate
What is it we
aim to learn or What do we not already know (from
do through existing research, practical experiences)? Which questions can we ask given our Plan
research? frame?
Which concepts or concerns do we focus
on in furthering our understanding (e.g., Which methodology (toolbox) works
power dynamics, teacher identity, Which methods (tools) can we use
with my questions to produce the
academic achievement) ? to acquire knowledge towards
knowledge we need to produce?
answering our research questions?

Paradigm/Theoretical Perspective

Ontology Epistemology
Theories and beliefs about reality: Theories and beliefs about how we know what we know:
- Is there a reality external to actors’ beliefs? - Is access to reality always mediated by theory?
- Do events have meaning independent of any - Are facts separable from values?
consciousness of them? - Does knowledge involve beliefs corresponding to independently existing
facts or to other beliefs?

9
Epistemology is the study of what knowledge is and how it is achieved. Claims about knowledge
are related to claims about reality. For example, positivism can claim to achieve objective and
theory-independent knowledge only because it assumes that the reality it investigates exists
beyond the concepts and meanings (theory) of the researcher. Interpretative/constructionist,
critical, and deconstructive approaches deny that reality can be accessed except within the confines
of a particular theory and that it is impossible to extricate the facts described in research from
theory. This makes the researcher’s knowledge claims relative to his or her particular theoretical
perspective and, as such, not “objective” in the sense of being universally valid.
A paradigm or theoretical perspective is a tradition of research commonly identified by a set of
ontological and epistemological commitments and a shared sense of what research is for and what
it can achieve. Quantitative research tends to operate within a positivist paradigm, in which
research aims to uncover statistical regularities that correspond to causal laws. A phenomenon is
said to be explained here if its relationship to other phenomena can be reliably and validly
described. Qualitative research, by contrast, primarily aims at understanding the human actors
under investigation in terms of their motivations and the meanings they give to their reality.
Different qualitative paradigms may have additional goals, as described below.
Methodologies involve applying our theoretically informed approach to gain knowledge about the
world, including a coherent set of methods and processes for analysis. Methodological analysis
aims to ensure consistency between the research question, theoretical assumptions of the study,
and the toolkit of methods chosen for collecting and analyzing data.
The theoretical issues discussed above are inescapably bound up with any scientific endeavor. Yet
self-reflection regarding ontology, epistemology, and methodology is typically a feature of
qualitative rather than quantitative research. The adoption of the positivist paradigm from the
natural sciences has enabled quantitative research to unify its methods and make incremental
progress without calling into question its theoretical assumptions. In contrast, questioning the
nature of social reality, our access to it, and the status of our knowledge has led to a branching of
approaches within qualitative research, emphasizing different questions relating to the social world
and deriving from distinct philosophical traditions. To a far greater extent than quantitative
research, changes in qualitative theoretical approaches and methodologies have tended to be part
of broader social and intellectual changes, such as poststructuralism, postcolonialism, and
feminism. Many qualitative researchers self-consciously position themselves within these
movements. Theoretical debates are therefore close to the surface in many areas of qualitative
research.
For more details on the topics discussed above, please see Box 2 and Figure 2 below.

10
Box 2: Putting it all together: An example

Teacher absenteeism is a challenge in the education sector of many countries in sub-Saharan

Africa. Suppose the goal of a research project is to better understand teacher absenteeism in order
to inform policymaking. This study could be conceptualized and carried out in different ways
based on different ontologies, epistemologies, and, thus, paradigms in doing research.
A quantitative approach may start by reviewing data from other studies of teacher absenteeism in
order to identify measurable factors that may be statistically related to teacher absenteeism.
Formative fieldwork may also be used to identify the factors to be measured. Researchers may
then develop a survey to collect the data deemed relevant, such as teachers’ health, marital status,
level of independent means, the distance between home and place of work, and gender. The
ontological assumptions behind this approach are that social facts (teacher absenteeism) are
correlated with other facts (gender, marital status) in regular and predictable ways. Social reality,
like physical reality, is assumed to be governed by invariant laws. Epistemically, an explanation
is deemed to have been provided if we can say that, given a more or less restricted set of
background conditions, when one or more variables are present, the phenomenon to be explained
is also present. Deciding to use a survey as a particular method, in this case, brings with it further
epistemic assumptions, namely that the variables will be known to the respondents and will be
reported reliably. Statistical analysis of the survey data could then explain teacher absenteeism in
terms of the factors related to it, which may then help to inform the policymaking process.
A qualitative approach may attempt to understand the phenomenon of teacher absenteeism in
terms of the beliefs and motivations that lead to this behavior. Researchers may attempt to build
up a detailed picture of the social world in which the teachers live and work. This may use the
ethnographic method of participant observation, situated in the interpretivist paradigm. Areas of
investigation may include the personal, social, and economic circumstances of teachers, how their
work fits with their other roles and identities, how they see their work in relation to the community,
or how they experience the institutional structures of the education system. This approach may be
grounded in a social constructivist set of ontological and epistemic assumptions. Reality here
would be conceived as constructed from the meanings generated by social actors. What “going to
work” means will vary from one culture to another, including with respect to the obligation to
attend. Epistemically, researchers assume they can access these meanings through extended
exposure to and participation in local social practices. This is by no means straightforward, as the
researcher may start with very different concepts and meanings from the interviewee, posing a
significant hermeneutic challenge. However, by exploring how the research subjects connect their
concepts to other concepts, practices, and institutions the researcher attempts to translate the
subjects’ meanings into terms familiar to the researcher. The kind of explanation aimed at here is
based on teachers’ beliefs and motivations—and, once these are known in enough detail, teachers’
decisions not to attend become “understandable.” The level of detail gained from this approach
may help to determine how teachers are likely to respond to changes made to their circumstances
by policymakers.

11
 The box below indicates how theoretical and methodological assumptions are related to the choice
of research methods. This is not a linear process, but demands reflexive reconsideration of each
element, including the validity of the assumptions, as the research progresses.

Figure 2: Theoretical aspects of methodology and method

Theoretical perspective or Methodology Methods

paradigm
Quantitative: statistical Quantitative: survey
Quantitative: positivism analysis of variables Qualitative: participant
Qualitative: social Qualitative: interpretivist observation
constructinism

Epistemology
Ontology Quantitative: Statistical relations between
Quantitative: Facts in the social world are facts in the social world can be uncovered
corralated with other facts in regular ways. through observation.
Qualitative: Reality is constructed from Qualitative: Social actors' meanings can be
meanings generated by social actors. understood and translated by researchers
from a different culture.

2.1 Paradigms and methodologies

The nine qualitative methodologies this note deals with may be grouped into three broad
paradigms. There are significant overlaps, and room for disagreement regarding exact placement
exists, but as Figure 3 below shows, each methodology can be said to correspond to one of three
aims: (1) understanding social phenomena in their context; (2) creating change and empowerment;
and (3) challenging and countering dominant discourses.
2.1.1 Paradigms

An interpretive or constructionist paradigm seeks to understand human behavior and social

contexts by understanding the meanings actors give to their actions, goals, roles, identities, and all
aspects of the world they experience. Meanings are commonly said to emerge “intersubjectively,”
as the outcome of communicative action between subjects. Common techniques include
‘Verstehen,’ whereby the researcher attempts to see the world through the eyes of the person or
group under investigation, and ‘thick description’ (see Box 1).
For the critical paradigm, the point is not merely to understand the social world but to change it.
Research into the underlying causes of discrimination, domination, or injustice is seen as a
prerequisite and motivator for social change, and it often involves the inclusion of research
participants who might otherwise be underrepresented or silenced.

12
Deconstructive approaches aim primarily at analyzing forms of knowledge (discourses) that are
argued to configure power relations in which certain groups are subordinated. By revealing and
criticizing (deconstructing) the historical formations of existing discourses, this approach aims to
create space for different discourses corresponding to more equitable configurations of power.

Figure 3: Different paradigms and methodologies of qualitative research

Paradigm Methodology
• Phenomology/hermeneutics and grounded theory (§1)
Interpretive/constructionist • Historical analysis and policy analysis (§6)
Aim: Understand, interpret, • Ethnography (§2)
construct meaning • Qualitative longitudinal research (QLR) (§3)
• Case studies (§5)

Critical • Communityengaged and participatory approaches (§7)

Aim: Create change and reveal • Critical ethnography, critical realist evaluation (§9)
hidden perspectives and causes • Qualitative longitudinal research (QLR) (§3)
alternative and silenced voices • Life histories, oral narratives (§4)

Deconstructive • Post-structural analysis - narratives

Aim: Deconstruct, challenge, • Postcolonial analysis - historiography (§6)
replace assumptions • Critical discourse analysis (§8)

2.2 Methodologies in qualitative research

This note covers nine main methodologies that can be used to guide qualitative research in
education. As noted earlier, these are the research and analytical approaches that will guide the
choice of methods. Each of them is described in detail below, and relevant examples are presented
for each of them. These are not mutually exclusive—for example, it is possible to do case studies
in participatory ways or to apply critical discourse analysis to a policy study.

2.2.1. Phenomenology and hermeneutics

As methodological approaches, both phenomenology and hermeneutics are primarily concerned
with how humans live in, experience, and interpret specific phenomena in their world (van Manen
1990). 10 Phenomenology is particularly useful when studying groups of individuals’ experiences

10
These are also philosophies that have underpinned several methodological approaches in the
13
and interpretations.
For example, Sharma-Brymer and Fox (2008) use this methodology to understand what it means
to be an educated woman in India. They explore women's narratives of their experiences and
identify patterns and differences in the narratives and women’s tensions and conflicts. They show
how recipients of education regard the value of education in their own lives and what that education
means within their socio-economic groups and in different circumstances. Another study, Vagle
(2009), looks at how teachers experience moments of recognition and response when their students
are not understanding something. This study captures teachers’ own thinking, feelings, and actions
as they see and experience them in relation to students in the classroom. The study also identifies,
for example, how teachers recognize students’ body language and when, and if, that recognition
prompts actions by the teachers to address what they are perceiving. In the first case, understanding
education’s meaning can support the design of policies aimed at increasing school attendance, for
example, by presenting information on the value of education. In the second case, understanding
the nuanced processes of teaching can be used to design support materials for teachers or those
working with them.
Related to phenomenology and hermeneutics is grounded theory, which aims to generate
emerging theories from in-depth qualitative data by interpreting key themes across multiple data
points or participants (Corbin and Strauss 1990). 11 This type of study is particularly useful when
existing theories are not appropriate to the question or context under study. Such a study can help
generate more relevant theories that can then be further tested on larger samples of participants or
in other contexts.
An example of the use of grounded theory is provided by Kirchoff and Lawrenz (2011) who study
teachers’ career paths in high-need contexts. The analysis of teachers’ interviews reveals a
theoretical model of career paths informed by factors such as the support provided by the education
programs and the preparation teachers received for working in high-need settings (in the analysis
section later in this paper, additional details are included to show how the authors undertook their
analysis).
2.2.2. Ethnography
Ethnography encompasses a diverse set of approaches and methods that primarily aim to
understand participants in their social and cultural context in order to provide social and cultural
explanations of a phenomenon (Anderson-Levitt 2012). In this quest, ethnography engages with
cultural contexts and communities over a longer time frame, and it tends to involve the researcher
as a participant observer in the context over a period of time, often acquiring the local language
and using multiple data collection tools.
For example, Magee and Pherali (2017) observe a non-formal educational program supporting
Jordanian and Syrian refugee youth. They also report how participatory pedagogies engage out-

interpretive/constructionist tradition, including grounded theory, ethnography, and general interpretive

methodologies, such as semiotics.
11
Grounded theory uses a sequential iteration—analyze, revisit, revise—to identify categories and concepts within
the data that are then linked into formal theoretical models in a systematic manner. It does so by using a constant
comparison method (Glaser 1965).

14
of-school refugees in host communities and describe how a series of intersecting factors, such as
the value or desire for certification, affect the use of such pedagogies. Similarly, Rodriguez Gomez
(2017) reports findings from 12 months of research in an Ecuadorian school, including how being
in an environment of protracted armed conflict permeates everyday life. Social linkages with the
conflict affect relationships within the school community, including between the school and
children.
While ethnography has mainly been used to study groups of people, it has also been applied to the
study of policy as enacted in education (see Sutton and Levinson 2001). This approach is more in
line with the critical ethnography of education and development. Critical ethnography aims to
reveal hidden assumptions underlying education systems, development aid, or learning.
Ethnographies stemming from the critical paradigm question taken-for-granted assumptions or
ideas (even when expressed by the research participants, who are asked to consider
counterfactuals). In this way, critical ethnography aims to reveal and question the assumptions of
the system or program (Levinson, Foley, and Holland 1996). 12
In an example of critical ethnography, Jeffrey, et al. (2008) studied unemployment and
underemployment among young Muslim men in Uttar Pradesh, India. The study reveals “school‐
educated” men and “madrasah‐educated” (religious school-educated) men’s perceptions of their
education and how these perceptions related to their search to become “respectable men,”
including their views of what constitutes good work. The study also reveals how uneducated men
have different views about education and job search strategies than their educated peers and how
critical they are of them. As another example, Bartlett and Garcia (2011) document a four-year
research engagement in Gregorio Luperon High School in New York City with a majority of
Dominican immigrant students and provide accounts of curriculum, teacher practices, and the
school community that challenge the view that schools serving such populations are failing to
educate students. Moreover, the study also provides an account of the extent of positive situations
and outcomes that challenge assumptions about the school’s achievements.
2.2.3. Qualitative longitudinal research (QLR)
Qualitative longitudinal research is primarily concerned with understanding how time can affect
individuals, groups, or a given phenomenon, and it analyzes the dynamics of change and continuity
in relation to time or across generations (Saldana 2003). It includes a number of approaches and
methods, including ethnographies that extend over years, repeat interviews, observations, or life
histories.

12
This includes international aid in education where researchers such as Mosse (2005) suggest that low-impact
projects might be because of faulty assumptions about educational policies and practices.

15
For example, the Young Lives qualitative study follows children over a seven-year period to
understand their educational experiences and life trajectories. The qualitative approach was
designed to gain an in-depth understanding of children’s views and opinions and to document
change and continuity in their lives. (Crivello, Morrow, and Wilson 2013). Given the richness of
the dataset, several papers and analyses have been produced. One example is the research that
Thuc Du and Minh Tam (2013) do using the third wave of Young Lives qualitative data (combined
with analysis of Young Lives survey data) to look at school dropout in Vietnam. They find, from
the explanations of the children who left school early, that children’s reasons for doing so were
commonly related to the need to contribute to income generation. For example, for one child whose
household was chronically poor, the authors found that this child had previously devoted most of
her time to work. Time spent at work had affected her ability to study, leading to poor performance,
low expectations in future exam performance, and finally, her mother’s decision to pull her out of
school even before the exam was to be sat. In the case of a second child, who passed the exam and
had a family with a good economic status and that valued education, the study recorded his
experience of being the only one from his ethnic group in his class and how this affected his later
decision to drop out.
Camfield and Roelen (2011) also used the Young Lives data to look at a group of Ethiopian
children and their experiences in transitioning in or out of poverty over time. They also looked at
how some positive changes (e.g., the acquisition of livestock) were not unequivocally beneficial
to children, as some children dropped out of school to herd the new animals.
Another example of qualitative longitudinal research is a five-year study of the dynamics and
impacts of a livelihood program for youth in East Africa. Lefebvre, et al. (2018) studied how youth
were able to “get ahead” as a result of the program, as well as how those “getting by” experienced
challenges. “Getting ahead” and “getting by” were defined from the data as the research team
examined how their livelihood trajectories developed. For example, fewer young women were
getting ahead, in part because they faced exploitation and harassment in securing employment.
Furthermore, the sectors in which they were often employed or self-employed were less secure or
lower paid. 13 This longitudinal qualitative study added to an analysis of findings of short-term,
post-training outcomes (Krause, et al. 2016), and it provided a complex picture of youth earnings,
livelihood, and wellbeing.
2.2.4. Life histories, oral histories, narratives
Life histories, oral histories, and narratives are inquiry approaches that tend to focus on individual
experiences and how individuals construct important events in their lives (see Clandinin and
Connelly 2000). Narrative inquiry follows the idea that experiences and memory are organized as
a narrative or account of events partly along a timeline, but that the timeline is both socially and
experience-dependent (Clandinin 2006, Bruner 1991). 14 Oral histories are a useful method to

13
Researchers collected annual interview data for five years from of 230 youth, asking about their participation in
further education/training, working for pay, savings patterns, goals, and challenges and opportunities in attempting to
create a sustainable livelihood.
14
As with any account of something that happened in the past, salient pieces of a process or event are captured in a
narrative, and the time span between these events might vary.

16
document lives and experiences underrepresented in mainstream research or experiences not
represented in aggregate data. These narrations can be done individually, or they can be used as a
form of community-engaged research 15 by having the participants’ construct and share their oral
histories in an effort to change commonly accepted knowledge about history. Life histories and
narratives can be particularly helpful in understanding how the lives of specific individuals are
affected by certain events, as well as how different decisions, actions, and external influences
concatenate in a person’s narrative of their own life.
One example of narratives related to teachers and teaching practices is Vance, Pendergast, and
Garvis’ (2015) study of teacher resilience, including skills such as persistence, problem solving,
and confidence. This research asked about teachers’ perceptions of their own resilience while they
were involved in a program to teach socio-emotional skills and promote resilience among students.
Teachers were interviewed repeated times and engaged in conversations where emotions, feelings,
and practices in the classrooms were detailed. Findings show how teachers reflect on their own
views of resilience—something they said they had not given a lot of thought to—as they teach the
content of the curriculum. It also showed how these teachers link resilience to stories of their
personal life rather than their professional role, despite the program including features of resilience
in the curriculum.
2.2.5. Case studies
In case studies, the focus is not on individuals (as with previously discussed methodologies) but
on cases defined by specific boundaries related to the phenomena or setting. Cases include both
individual experiences as well as contextual information from multiple forms of data. Case studies
can be single or multiple (comparative) and can refer to a program, site, or group of individuals
(Bartlett and Vavrus 2017). Irrespective of the type, case studies use a variety of methods to more
fully describe and explain the phenomenon of interest (Yin 2017, George and Bennett 2005). This
is done by triangulating the evidence from different data sources or by data proceeding from an
inquiry on different aspects or participants involved.
2.2.6. Historical and policy analysis
Historical and policy analysis is one of the approaches that can take different forms depending on
the paradigm framing the study. An interpretive approach will focus on how problems addressed
by policy come to be, how policy options are formed, and how policies are then implemented. This
approach tends to examine political processes, policies as texts, and policies as practices (Dunn
2015). Critical approaches examine who is advantaged or disadvantaged by a policy. For instance,
a critical sociological approach proceeds by situating policy problems, texts, and practices within
a social context of considering what constitutes “policy problems” as defined by different groups
(Gale 2007; Ball 1993). Also, a critical approach seeks to understand the policy discourses and
practices related to the policy problem under study; this approach can focus on examining public
issues and how they came to be addressed through policy, both in the past and in the present (Gale

15
Community-engaged research encompasses a set of approaches that situate the participants of a study as co-inquirers
in the process of producing that study.

17
2007). Historical analysis grounded in a postcolonial perspective, the third paradigm, examines
issues or policies in education in relation to colonial or various imperialist influences and aims to
deconstruct past practices and taken-for-granted current practices (see Tikly and Bond 2013 for
more on postcolonial research and ethics).
For example, Dreyden-Peterson and Mulimbi (2017) provide a rich example of historical policy
analysis of education policies in Botswana starting in 1966. They draw on documents, in-depth
interviews, and survey data to present the analysis of a set of moments in history that defines
critical periods of negotiation between the two main opposing ideas permeating Botswana’s
education system, national unity and ethnic diversity.
Seth’s (2007) examination of the British colonial system of education in India is an example of
historical analysis from a postcolonial perspective. The research focuses on the period between
1835 and 1930 and the British colonial authorities’ decision on whether to base the official
education format on the English language and “western” knowledge or Sanskrit and Arabic and
“oriental” knowledge. The research offers a historical analysis of how a “western” knowledge
approach was enacted and perceived over time and how this choice came to be seen as the only
knowledge system (rather than one of many). The research presents how “western”-oriented
education ideas and practices continue today.
2.2.7. Participatory approaches
Participatory and community-engaged research encompasses a set of approaches that situate the
participants of a study as co-inquirers in the process of producing that study, as opposed to simply
being sources of data. Robert Chambers (1983) developed a range of participatory methods to
overcome the barriers to understanding the local, diverse, and often complex and dynamic realities
of people in poverty created by asymmetric North-South and urban-rural power relations. “Putting
the last first” (i.e. starting with local realities) aims to disrupt the dominance of northern knowledge
in defining development possibilities, enabling “otherwise marginalised people to exercise greater
voice and agency, and to work to transform social and power relations in the process” (Gaventa
and Cornwall 2015, p. 468). Thomas and Narayanan (2015) distinguish the disruptive, subversive
potential of participatory methods from the participatory components found in many large-scale
interventions. Johnson and West (2018) have pioneered new approaches for the participation of
children, as applied to understanding education-related contexts. Teamey and Hinton (2014)
explore the transformative potential of participation and learning.
There is a broad continuum of what constitutes participation in these approaches, from the use of
engaging and creative methods so that participants decide on how and what they will reveal about
a phenomenon, to participants deciding on the research questions and how they will be studied.
Participatory methods can also be used in collecting quantitative data (Holland 2013). A key
criterion of the quality of such an approach is the extent to which it is responsive to, and
incorporates, the active participation of those who are affected by and involved in the issue under
study. The field of evaluation has developed many variations of participatory evaluation, starting
with the early work of Robert Chambers (1990) and evolving to include more recent forms of
democratic evaluation, appreciative inquiry, and others. Norton, et al. (2001) offer advice on how
to conduct participatory assessments. Researchers working from a decolonizing perspective of
knowledge production (see, for example, Tuhiwai Smith 2013) have developed principles and
processes for undertaking community-engaged research that are relevant to the needs of local

18
communities without imposing dominant knowledge structures on them.
Ravitch, et al. (2017) provide an example of a participatory action research project and the
principles that underlie its community-school partnership program in Nicaragua. Participatory
action research “seeks to understand and improve the world by changing it. At its heart is
collective, self-reflective inquiry that researchers and participants undertake, so they can
understand and improve upon the practices in which they participate and the situations in which
they find themselves. The reflective process is directly linked to action, influenced by
understanding of history, culture, and local context and embedded in social
relationships.” 16 Drawing on the involvement of different stakeholder groups in the program, the
authors reflect on the process participants use to develop and operationalize the program; the
challenges they experience when co-constructing capacity; and how designing and implementing
the program was a learning experience for participants. Setlhare, Wood, and Meyer (2017)
describe using participatory action research to understand and support the career development of
teachers in under-resourced settings in South Africa. In this case, they analyzed teachers’ written
reflections during the group discussions of a career counseling program, transcripts of those
discussions, and teacher journals produced during the training. Findings suggest that as the
program progressed, teachers identified personal and professional assets and further developed
strategies to stay motivated during challenging times. The research also showed how participants
developed greater agency for career and personal goals, as they designed future life maps and
explored pathways for collectively improving their support to learners.
2.2.8. Critical discourse analysis
Critical discourse analysis corresponds to the linguistic analysis of documents and texts with a
focus on the discourses—the ideas and language used to shape meaning and practices. Critical
discourse analysis is particularly useful in examining taken-for-granted or everyday ideas as they
take on new and different meanings across groups, time, and spaces (see Gee 2014; Fairclough
2013). Using this methodology, a policy as a discourse can be analyzed in terms of how it is
intended to be implemented and its desirable effects, as well as how it is implemented, and the
effects it generates.
Vavrus and Segher’s (2010) analysis of meanings of participation as used in government policies
in Tanzania is an example of critical discourse analysis. The authors use a semantic (text and
meaning) analysis and apply it to official documents related to poverty reduction strategies in
different moments of time. They comb the texts for specific keywords (“partnership” being the
main one) and document how these words appear (the semantic relation), for example, in relation
to a problem/solution sentence. They compare if the terms are ascribed the same meaning across
documents in order to reflect on common or divergent understanding of what partnership means.
Vavrus and Seghers (2010) found that partnership is used to refer to external partners but not to
communities and that when communities are mentioned, they are mentioned in a passive and less
equal role (e.g., “communities participate”).

16
Retrieved from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2566051/.

19
 2.2.9. Critical realist evaluation/research
Finally, a critical realist evaluation methodology focuses on causation and outcomes. But it does
so from an acknowledgment of both objective and subjective perspectives of reality, aiming to
account for both social structural forces and human constructions as they influence outcomes (see
Pawson 2006, 2013; Maxwell 2012; Fletcher 2016). From an interpretivist epistemology, this
approach recognizes that causation—for example, the link between an anti-poverty policy and the
achieved outcomes of that policy (lower poverty rates)—is constructed by individuals and
communities. From a realist perspective, beliefs, values, and reasons have causal consequences
(Maxwell 2012). Critical realism is particularly useful in studying complex educational programs
or outcomes as it pays attention to the complexity of these processes within particular contexts. It
focuses on identifying the mechanisms that connect a program with an observed outcome and
which mechanisms help or hinder that outcome.
Before moving to a description of methods for qualitative data collection, Table 2 provides a
general overview of the types of questions qualitative research approaches can be used for,
covering what, how, and why questions (as the where is always present in qualitative research
given its context-focused stance). It shows the different methodologies discussed here that can be
used to answer these questions.
Table 2: Examples of questions that can be answered with qualitative methodologies

What we need to know Why we need to know this Examples of methodologies and
approaches that can be used

What are the Without knowing the antecedents and  ethnography/critical

antecedents or underlying causes or mechanisms that ethnography
underlying causes of give rise to an educational problem, the  community-based
the educational policies or interventions may not address participatory
problem? (Why is them, address them only partially, or approaches
there a problem?) address the symptoms but not the  historical and policy
conditions. analysis
 critical discourse
analysis

What are the Educational program effectiveness is  historical and policy

contextual conditions often assessed based on assumptions of analysis
that affected the the fidelity of implementation. Research  qualitative longitudinal
implementation of into the contextual conditions and research
the educational implementation processes can reveal  case studies
program? whether and how this has been done.  ethnography

What unexpected Beyond the stated or pre-defined  qualitative longitudinal

outcomes occurred outcomes of educational research
as a result of the programs/interventions, and assumed  community-based
education program? spillovers, additional positive and participatory
negative unexpected outcomes can be approaches

20
 What we need to know Why we need to know this Examples of methodologies and
approaches that can be used

present.  case studies

How do participants Even if a new approach is found to  phenomenology

(e.g., teachers) improve learning, we may not  narratives
understand/enact a understand how it is regarded by those  qualitative longitudinal
new teaching who implement it or how they analysis
approach? implement it to achieve the results it has.

How do different Even if we research the impact of an  case studies

contextual, educational program/intervention  ethnography
implementation, through quantitative methods, we do not  qualitative longitudinal
and/or individual often have sufficient data to explain the research
factors affect processes that affected the outcomes.  critical realist
learning outcomes? evaluation

How does the new Because educational processes and  qualitative longitudinal
education program learning are dynamic and contingent, research
affect students students are affected differently by  ethnography/life
differently over interventions or practices at different histories
time? And when? times. Longitudinal/time-series
quantitative methods can provide
answers to this question, in part, but the
nuances of when and how students are
affected by a program can also be
revealed through qualitative data.

Why is an With policy and program borrowing a  critical realist

educational program common practice in education, it is evaluation
effective in one important to understand under what  case studies
context and not conditions programs have been effective  ethnography
another? or when/how they are not.

Section 3: Methods in qualitative research

A range of methods is used in qualitative research, and these methods are usually categorized as
interviews, documents/artifacts, and observations (Patton 2002). More recent research has also
included social media artifacts. In addition, products of participatory, visual, and art-based
methods can also be used for qualitative research. In all cases, these methods can be used to

21
produce either quantitative or qualitative data, but they are distinctively qualitative when they are
designed, following the purpose of our research questions, in a semi- or unstructured manner and
produce open-ended forms of data. Moreover, the methods can be used individually or in different
combinations, depending on the methodology guiding the study.
More detailed descriptions of these methods are presented in the following pages, including the
type of data associated with them. However, as before, these descriptions are not exhaustive and
instead focus on the attributes of these methods that can help guide decisions about when to use
them in qualitative education research. 17
The four categories of methods discussed here can be used within the different approaches and
methodologies for data collection (previously discussed), and the data they produced can fit
different analysis approaches discussed below. The types of methods used to answer the research
questions vary in terms of the key participants in the study, the time required for data collection,
and the depth of interaction in the field, among other factors.
3.1. Interviews
Interviews include a broad set of methods used to gather data through a conversation about
people’s lives, experiences, perceptions, opinions, feeling, and knowledge. They produce data in
the form of conversation transcripts.
In general, the main goal of interviews is to capture the individual experience and perspectives on
events. The assumption is that the participant is knowledgeable and his/her perspectives
meaningful. There is also an assumption the participant can make articulate and make explicit his
or her perspectives. Interviews are particularly appropriate when the subject matter is highly
complex or sensitive, when detailed information is needed, and when a process of progressive
exploration may be beneficial. However, it is important to keep in mind that the interviewing
process is one that is always mediated by the interviewer/researcher, which can affect what people
choose to talk about and share.
There are several different types of interviews, and they align with different purposes and
methodologies. Of these types, the most frequently encountered types include individual and group
interviews (these latter include focus groups); unstructured-structured; single-multiple; and
informal-formal (see Patton 2002).
3.1.1 Individual and group interviews
Interviews can be conducted with individuals, small groups, or larger groups. One-on-one
interviews may make people more comfortable sharing their thoughts or ideas, especially on
sensitive topics, e.g., violence or gender discrimination. They are also appropriate when the
presence of others might affect the answers provided. Interviews can elicit deep narratives about
the phenomenon being studied.
Small group interviews may be useful when it is either a common practice to sit with someone else
you know when being with an unfamiliar person (e.g., a researcher) or when there is a relationship

17
Several guides explain in more detail the uses of these methods in qualitative studies: Mack, et al. 2005 and Hassler,
et al. 2008 for the RECOUP manual on qualitative research skills workshop, among others.

22
or shared experience between members of the small group that can prompt responses to questions.
Large-group interviews tend to focus on getting a group’s shared thoughts or reactions to
questions; the individual perspectives are less the focus of analysis.
Focus groups or focus group discussions (FGDs) are a related but distinct form of interviewing a
group of individuals. In terms of their use vis-à-vis group interviews, researchers suggest a couple
of situations in which focus groups make sense: when the research aims to gather information from
a group of people that share some common characteristics and seeks consensus about a particular
topic (Krueger and Casey 2014); or to foster a dialogue that allows for diverse perspectives from
individuals and the collective to be revealed (Barbour and Kitzinger 1998). In general, focus
groups capitalize on group dynamics. They are best used when a single subject area is being
examined in-depth and strings of behaviors/thoughts are less relevant. Table 3 elaborates on the
differences between group interviews and focus groups.
Group interviews, irrespective of their form, are affected by interpersonal dynamics, including
power relations between participants, regardless of their previous knowledge of each other. The
degree to which these dynamics are a logistical issue or a focus of analysis depends on the
epistemological stance taken in the research.
Table 3: Distinctions in using group interviews and focus groups

Group interviews (small) Focus groups

Purpose Practical: Ease of interviewing more Benefit: To have a discussion around

than one person or because it is a common topic where group
culturally appropriate to not be alone participants have a relationship or
in an interview. experience with the topic; used to
Benefit: The purpose is to analyze the gather views and opinions (similar
group’s conversation and how and different) on the topic.
participants relate to each other and to
gather multiple perspectives on the
issues discussed.

Role of the The interviewer directs questions to The interviewer facilitates the
interviewer each person and seeks their responses. discussion (presents a question to the
group and ensures voice to all
participants).

Participants Two to four participants, usually 6-10 participants from a homogenous

friends or related members (such as a group related to the topic under study
young girl and a sibling or classmates). (e.g., teachers in a school).

Sample Based on the total number of Based on the number of groups

individuals to be interviewed (doing conducted (as they are organized in
group interviews does not necessarily relation to specific shared
23
 Group interviews (small) Focus groups
reduce the number of participants to be characteristics). 18
interviewed).

Recording Data can be audio recorded or notes Data may be audio/video recorded,
and taken; care should be given to who is though this can inhibit comfort in
analyzing, speaking and answering each question sharing views in a public space.
quality of so analysis of individual responses can Audio recording is more acceptable
data be done, or conversational between when verbatim transcripts are
individuals can be analyzed. needed. Video recording is often not
encouraged due to confidentiality
The quality of the data depends on the
and privacy concerns related to the
interviewer’s focus on each member
individual’s participation. Notes are
and also on how they direct the flow of
taken by a notetaker, but it can be
conversation to respond to the
difficult to distinguish who the
questions.
different speakers are. The analysis
focuses less on what specific people
say than the overall opinions or ideas
shared as a group.
The quality of the data may be lower
than individual or small group
interviews as individuals may speak
less in this format.

Organization, Small group interviews may slightly Focus groups should be organized in
time, and reduce time and cost, but the focus is relation to homogeneous groups
costs on identifying relevant participants and within the community. While many
having them respond to questions. researchers perceive they can gather
Small groups may facilitate some more data in a smaller amount of
participants to feel more comfortable time and less cost, focus groups can
or respond more readily. take considerable time if all group
members are facilitated to
participate.

18
As a rule of thumb, less than two focus groups for each category identified as needed in the sample (e.g., primary
educated adults, secondary educated adults) is not advisable.

24
 3.1.2 Unstructured to structured interviews
Interviews can take three forms: (1) unstructured, (2) semi-structured, or (3) structured, which
differ in the extent to which the questions are unplanned, focused around key themes or questions,
or fixed in advance. This choice will be defined by the research methodology and purpose.
Generally, qualitative interviews always include open-ended questions (not closed-ended
responses that are then categorized or quantified, though researchers may decide to quantify some
qualitative data later).
If a study’s purpose is to do a comparative analysis of participant experiences in a program, then
the questions should be consistent (structured) across participants. But if the study’s purpose is to
understand why a program succeeded in one site but not in another (comparative case study), then
different questions may be posed to understand the various processes or causes related to the
program’s success or lack thereof.
Ethnographic and life history/narrative studies tend to be more unstructured as they follow
emerging themes relevant to the study. Qualitative longitudinal research (using repeat interviews
over time) may require a combination of more and less-structured interviews, in which similar
questions are asked of all participants in order to follow changes in the phenomenon over time.
Different questions may also be asked of participants when following their narratives development
overtime to capture change or continuity.
In general terms, a structured interview will follow a protocol to ensure all questions are answered
(and asked in a similar way). Semi-structured interviews will follow a looser protocol in which
themes and sub-themes are identified, but where probing—encouraging respondents to elaborate
on a topic—might or might not be needed to elicit all data and where there is the flexibility to
inquire about new topics or issues. 19 Unstructured interviews depart from a general plan regarding
the focus and goal of the interview; but, instead of relying on an organized interview guide, they
are based on open-ended guiding themes.
3.1.3 Single or multiple interviews
Another consideration when conducting interviews is the number of interviews/conversations that
will be carried out with the same participants in order to answer the research questions. Single
interviews are best used to gather data about a phenomenon that is limited in scope and time, such
as a participant’s perceptions of an educational program at a specific time. Qualitative longitudinal
studies are designed to carry out multiple waves of data collection, and the number of interviews
and the length of time between them depend on the kind of change that is being explained. For
example, if a study seeks to understand language development among young children, more
frequent forms of interviews/data gathering may be needed to see the nuances in the change in

19
Kvale 2008; Rubin and Rubin 2012; Seidman 2013 all are helpful guides to developing interview processes,
including the stance of the interviewer, developing questions, and considering probing questions.

25
language. If a study is trying to understand secondary students’ aspirations post-schooling, then
the timing of these interviews may be spaced across the secondary years and several years post-
schooling. Ethnographies tend to involve multiple iterations of interviews/conversations in part to
home in on the explanatory nature of what is being studied. Similarly, life histories may also
involve multiple interviews.
In any study using interviewing, it is a good practice to plan for follow-up
interviews/conversations, particularly if there is an incomplete understanding of what is being
studied or a need to clarify contradictory statements or evidence. Follow-up interviews also allow
for longer conversations.
3.1.4 Informal to formal
A common way to distinguish the types of interviewing is along a continuum from informal to
formal. This continuum refers to how the researcher positions her/himself in relation to the
participants, and thus it affects how questions are asked and the non-verbal communication in the
conversation. This continuum also reflects different epistemological views (about objectivity,
subjectivity, and inter-subjectivity) about data gathering.
Informal interviews tend to be spontaneous and akin to a conversation between two (or more)
people. They are less rigid and can, therefore, be helpful in building rapport with respondents and
capturing quick impressions on a phenomenon (e.g., opinions of adults when seeing small children
out of school on a school day). Formal interviews, on the other hand, are planned and prepared,
with clearly defined roles between researcher and participants. These can follow informal
interviews or come before them. For example, interviews conducted in ethnographies—where
relationships are established in the context—can vary in their degree of formality depending on
the specific discussion at hand. Life histories may depart from a more formal setting, but as the
conversation progresses, informality may be necessary to explore the intricate details or develop
as the participant and the researcher repeatedly interact and converse.
The degree of formality might also be related to the power the interviewer has in a given social
structure or institution. For instance, interviewing a government official, someone who has a
leadership role, or is in some way considered “elite” may require some level of formality (e.g.,
setting appointments, sharing the interview questions or research purpose beforehand, or
undertaking more detailed preparation). Significantly, the type of responses provided might be
affected by such power, and interviewees at both ends (the elite and non-elite) might have different
considerations when providing answers to interviews that can be recorded, published, or
considered part of the public record.
3.2. Observations
Another common method for collecting qualitative data consists of direct observations and the
collection of visual data. Observational research captures interactions within a context (e.g.,
among people; between people and their environment; people engaged in a process), allowing the
researcher to learn about things people may be unaware of or might be unwilling/unable to discuss
in an interview or focus group. Observations are a tool to collect information about the setting and
physical environment of a program and to document how participants (such as teachers and
students) interact with each other or toward a process (e.g., how are teachers implementing a new
curriculum in the classroom).

26
Observations consist of recordings and images of activities, behaviors, actions, interpersonal
interactions, and processes as they are observed. These data may take the form of written field
notes with rich descriptions and/or drawings, images, and photos, including video recordings (see
Box 3 below for an example of an observation description). Representations of these data in reports
include detailed descriptions and visuals that illustrate the actions and interactions, as well as the
contexts of the study. Observational data can also be categorized along a series of continua, again
depending on the purpose of the study and the methodology used. Several continua to consider are
the role of the observer, the duration of observations, and the focus of observations (Patton 2002).
3.2.1 Role of the observer
The role of the observer varies, ranging from an active participant in the observed activity/event
to a remote observer. For example, in ethnographic studies the researcher is involved as a
participant observer, meaning he or she is engaged in the activity or setting being observed (to an
extent). Typically in such a case, the research speaks the local language, reducing the
complications of translation. An additional benefit of participant observation is direct ‘first-hand’
experience for the observer, as well as the flexibility to engage in other forms of data collection,
such as conversations with participants to clarify or better understand what is being observed.
Long-term residence in the community being studied leads to deeper relationships and trust, which
are likely to lead to more open communication (Brewer 2000). There are, however, risks involved.
First, the study focus can become limited to what the researcher is able to take in as a participant
(e.g., optimal locations for observing may not be optimal for participating, it may be more difficult
to take notes, the researcher’s attention is on respectful participation so phenomena may be
missed). Second, the presence of the researcher might alter the interactions that take place,
potentially rendering the observation less useful. An experienced anthropologist will report on the
potential distortions and limitations of their viewpoint through a process known as reflexivity
(Brewer 2000).
On the other hand, non-participant observations, where the researcher is present but not interacting
with the environment or situation, are frequently used in case studies or when documenting the
implementation of a policy or program. In education research, video or audio recording activities
and conversations in a setting are commonly used so that researchers do not have to be physically
present to observe. Video recordings are less disruptive and mitigate the disadvantage of
participants adjusting their behavior due to the presence of a human observer, though the presence
of a video camera can still change interactions and what is said. Furthermore, recording devices
are unable to capture contextual information that might be relevant when analyzing the data
collected. A given qualitative study may use participant and non-participant observation (or a
combination) and potentially in conjunction with other methods.
3.2.2 Duration and frequency of the observations
The duration of observations (the time spent observing) and the number of observations depend
on the purpose of the study. For a specific case study focusing on, for example, the enrolment
process in a school, short-term observation of that process or even a one-time observation might
be sufficient. For a case study focusing on the differences in the enrolment process for students
from different ethnicities in a given school district, multiple observations (across schools) or longer

27
observations (as parents from different ethnicities appear to enroll their children) might be better.
There is no given rule on either the duration or number of observations. Studies spanning a long
time period, such as ethnographic studies or qualitative longitudinal studies, are more likely to
involve multiple or repeat observations, though, again, it depends on the research question.
3.2.3 Focus of the observations
Finally, the focus of the observation needs to be considered in line with the research purposes,
especially since sites observed in educational settings (e.g., classrooms) are frequently busy and
unpredictable. Defining the focus ex-ante in line with the research objectives and the chosen
methodology will tend to produce observations with a very specific focus. For example, in the
OECD video study of teacher pedagogy, the focus is narrowly defined to capture the teaching
practices and students’ classroom learning in a single type of lesson across school systems that
vary in their classroom settings, pedagogical traditions, system-level policies, and student
achievement level (OECD 2018).
On the other hand, a more holistic approach allows the research to follow emergent issues of
relevance. Barbra Bruns has developed methods of classroom observation to capture a wide range
of data, enabling investigation into different aspects of teaching and learning (Bruns, et al. 2016).
This makes an iterative approach possible, where issues not thought to be of significance at the
outset may be analyzed as a clearer picture of the areas of interest emerges. This is in strong
contrast to many quantitative approaches, where departing from predefined research parameters
and selected informants is seen to compromise validity.

28
Box 3: Example of an observation description

Source: Turner, et al. (2002) pp. 100

3.3. Documents/artifacts
A further type of qualitative data are documents and other textual data and artifacts. Documents
consist of both written and digital materials, including policies, official publications and reports of
programs and organizations, and personal written works of participants, including educational
tasks, written exams/essays, diaries, and stories. Artifacts are material objects that help explain the
phenomenon under study, including tools or materials used in teaching, works created by
students/participants, or household or work/livelihood objects that are being studied.
The main goal of using documents and artifacts as data sources is to gain information from
naturally occurring, accessible data that have real effects in the world. Artifacts represent reality
beyond what people say about it. Documents and artifacts are particularly advantageous in that
they are generally readily available, unobtrusive to obtain, and stable. At the same time, they have
a disadvantage in that they may lack sufficient detail to address the research questions, and there
is selectivity in terms of what is being put into a document and what is being left out (Bowen
2009). Document selectivity might stem from considerations such as length and type of language
used) as well as active decisions regarding exclusion or lower visibility of some groups or issues.

29
Artifacts add a dimension to understanding a phenomenon, and they can be particularly useful
when used in relation to what people say about the artifacts and their meanings and uses.
In recent years, documents and artifacts have become easier to use and access, as documents are
digitized and artifacts are documented using digital means (e.g., photo or video recordings in
online archives or on social media). In addition, documents or artifacts produced by participants
of a study are increasingly used as an additional source of data to complement interview or
observation data in an effort to triangulate (cross-verify findings using data from two or more
sources).
Document and artifact analysis is the primary method for undertaking historical and policy
analysis, although other methodologies also rely on documents to provide additional evidence. For
example in Dryden-Peterson and Mulimbi's (2017) study of Botswana’s use of educational policies
to address (or not) ethnic conflict and how policy documents reflect those discourses, the authors
analyze a set of Botswana’s post-independence policy frameworks, including the National Policy
on Education, a report of the National Commission on Education, and the National Development
Plan its midterm review, among other documents. The study also looks at school syllabi at different
moments in time to understand the implementation of the discourse ideas and statements and their
translation into educational content.
One example of using documents produced by the participants is found in Bajaj’s (2009) study of
students who attended different kinds of secondary schools in Zambia. Bajaj used student diaries
along with group and individual interviews. Participants’ entries were structured with different
assignments, such as writing about their families, communities, and aspirations, with the objective
of capturing how an alternative pedagogy and school structure affected students’ sense of agency.
The diary analysis provides the main categories for the report (e.g., receiving advice from teachers,
affective relations in the school), which are then complemented with information from interviews.
Bajaj (2009) finds that the alternative school structure and pedagogy enabled transformative
agency, but that students had to renegotiate their agency upon graduation given structural
constrains in broader society.
3.4. Participatory and arts-based research methods
The main goal of participatory methods is to involve the participants as co-inquirers or co-
researchers, allowing their voices to be the dominant ones and enabling them to reflect on the
process of generating data on their experiences and their everyday knowledge (Reason and
Bradbury 2008). These methods are particularly useful in life histories/narratives and critical
ethnographies. Researchers who take a critical and decolonizing approach to knowledge
production use these methods to elucidate knowledge from perspectives that might not be visible
through other methods.

30
Based on the participatory goal of the study, participatory methods are flexible in how data are
produced and collected, as they need to be appropriate to the concrete research situation and
goals. Generally, participatory methods involve the co-researchers (or participants) in actively
generating and using visual and performative methods of data collection. Data may be visual
products created during data collection activities (e.g. drawings) or performances (e.g. skits
acting out personal experiences or beliefs).They are also commonly used to engage children and
youth who can be less inclined to talk in interviews or participants with limited verbal expression
skills or when trauma or sensitive topics mean that an interview would be uncomfortable or even
unethical. 20

Photovoice methods, where participants use cameras to take photographs of people, contexts, or
situations they consider representative of the aspect of their life (or the life of their community)
related to a research question, allow participants to define where attention is to be placed and what
is to be observed and interpreted. One example of the use of such a method is Ciolan and Manasia’s
(2017) research on undergraduates’ understanding of learning and learning strategies. In their case,
photovoice methods are used to explore the daily experience of learning. A group of volunteer
undergraduates across fields of study and universities received training on how to use photovoice
methods ethically to photograph objects, people, and situations that represented the way they learn
during one semester; they also wrote short narratives to accompany the learning experience
represented in the picture. Training participants on the photovoice methodology led to stronger
awareness and self-reflection of the students’ own learning processes. 21 Participants were involved
in analyzing data, deciding which photos taken by the group were most representative. Findings
point to the importance of students’ own experiences due to qualitative and hierarchical differences
in learning processes that the prevailing theories did not predict. The study also highlighted the
role of digital devices and distractions in re-defining what self-regulated learning can be. 22
Arts-based methods employ creativity to encourage the expression and sharing of perspectives,
emotions, and ideas. Such methods draw on literary/creative writing, performances (theatre, dance)
and visual art. These methods ask participants to draw, write, or act out a story, either alone or in
a group, that relates to the phenomena or question under study. Participatory and arts-based
methods both can be used within a range of paradigms, including for purposes such as
understanding participants’ interpretations of the world or the phenomenon under study
(interpretive); including perspectives which are usually excluded from research (critical); and
challenging and critiquing assumptions and norms of the phenomenon under study

20
Crivello, Morrow, and Wilson (2013) document a large set of tools used with children during the Young Lives
qualitative data collection, as well as tools used by other researchers, ranging from community maps where children
could draw or mark specific issues (e.g., where do they feel at risk) to drawings on specific issues, activities
representing their use of time, etc.
21
Training is a frequent element of participatory methods, and it is seen as a capacity-building and development part
of the empowerment process related to the production of data.
22
Wang and Burris (1997) provide additional information regarding photovoice methods, and Mark and Boulton
(2017) provide an example of adapting photovoice so Maori participants to share how they experience health services.

31
(deconstructive) (Leavy 2015).
One of these methods, popular or participatory theatre, calls for participants to be both the writers
and actors of a play, providing insight into their knowledge and experiences. Popular theatre
engages participants in generating, interpreting, and representing their ideas and views of the
world. For example, Mabala and Allen (2002) relate the process and outcomes of participatory
theatre research in four districts of Tanzania, as part of an effort to reduce youth’s risk of HIV
infection. A male and female artist in each district were trained by the research team, affiliated
with an arts college and theatre company. These artists, in turn, trained and supervised small groups
of youth in community theatre. They collected observations and interview data on problems related
to sexuality and HIV/AIDS and other social problems community members that contribute to HIV
transmission. The teams reflected on this data to produce dramatic performances, which were
presented to the surrounding community during rehearsals and at a district theatre festival. The
themes highlighted and structured by the performances, as well as audience input, generated frank
and sometimes contentious discussions. Community theatre specialists, along with district officials
carried out follow-up meetings and evaluations which guided the development of new programs
and policies.

32
 Section 4: Analysis approach—the final step when conceptualizing
a study design
Once a researcher has identified the research purpose, tradition, or paradigm, the research
questions, and the context or community in which the study is to be carried out, he or she can
define the methodology and methods and fine-tune the entire design. As discussed in previous
sections, the purpose, framing (within a paradigm), research questions, methodology, and methods
are foundational elements of high-quality research design. Operationalizing this design requires
thinking about the logistics of data collection protocols, sampling, and data storage. Both logistical
considerations and a study’s design elements critically shape how the analysis approach is
defined.
Analysis approaches are, simply put, the processes for making sense of the data in order to answer
the research questions. They should be aligned with the purpose, methodology, and methods. As
an example, Figure 4 provides a flow chart of the connection between a study purpose and analysis
approaches.
This note discusses five analysis approaches that align with different methodologies discussed
earlier. These approaches include (1) content and thematic analysis, (2) constant comparative
and/or qualitative comparative analysis, (3) process modeling and contribution analysis, (4)
qualitative longitudinal analysis, and (5) critical discourse analysis.
4.1 Content and thematic analysis

Content and thematic analysis are among the most frequently found analysis in qualitative
research. Both content and thematic analysis approach an investigation as a discovery of trends
and patterns. Content analysis focuses on terms, words, phrases, or concepts and their frequency
and relationships—that is, what is said and how is it said. Content analysis is often used as a
quantitative approach to qualitative data, which involves coding the frequency of words, phrases,
or concepts. A qualitative approach to content analysis aims to identify the sub-categories or
meanings associated with categories of words/concepts (for additional details on the analysis [Elo
and Kygnas 2007 and Fereday and Muir-Cochrane 2006]). Thematic analysis aims to identify
patterns within the data and create from these patterns categories of analysis (Braun and Clarke
2006 and Bowen 2009). The content analysis approach is also one of the initial steps of discourse
analysis, in which researchers identify the frequency of words used across textual sources and
analyze meanings associated with these words or concepts.
Thematic analysis is used to identify patterns within the data, with emerging themes becoming the
categories for analysis (see Bowen 2009). In the process of analyzing and coding, the researcher
asks the question: How are the themes in this text similar to, or different from, preceding texts?
The process involves a careful, focused reading of the data. The researcher carefully applies codes
to the data. Bowen (2009) provides a detailed description of using thematic analysis of documents.

33
Figure 4. Conceptualizing a study: example of the connection between the research question and analysis
approaches

Source: R. Bamattre, 2018, University of Minnesota

4.2 Constant comparative analysis

Constant comparative analysis is most often associated with grounded theory, and it is also used
in ethnographic or case studies (see Corbin and Strauss 1990). In grounded theory, a constant
comparison of concepts in the data is undertaken to generate an emerging theory. For example, in
case studies, constant comparison, or the related approach of qualitative comparative analysis, may
be used to provide explanations within and between cases. Grounded theory is a process where
there is a constant iteration to compare and contrast, so the theory emerges from the data, not the
other way around. Please see Figure 5 for more details.

34
Figure 5: Iterations of constant comparative analysis

Researcher Concepts are Concepts and

Similar concepts
identifies compared and categories are
are grouped A theory is
concepts in the tested with new compared and
together to form created
data (e.g., concepts/other tested as new
"categories"
words, events) concepts data is added

Source: Based on the process described by Frost, et al. (2010)

This type of analysis can help answer questions about the context and implementation of education
programs; it can also be useful in comparing antecedents and outcomes in different contexts. For
example, Wang (2011) identifies gaps in the evidence base on the difficulties of implementing
student-centered teaching reforms. While many studies focus on materials, teacher training, or
assessment, this study examines how a reform in rural China may have impacted teachers’
experiences of lesson time and their pedagogical decisions. Through unstructured interviews,
lesson observations, and participant observation, Wang (2011) began with teachers’ perceived
problems of teaching rural students and identified contradictions between curricular expectations
and the preparedness of students, which led to lecturing, not as reluctance or failure to adopt new
methods but as a strategic form of “defensive teaching” (p. 5). In another example, Kirchoff and
Lawrenz (2011) provide a detailed description of their coding and analysis within a grounded
theory approach, in which they iteratively sought additional participants and data to confirm (or
disaffirm) the model of teacher retention in high-need contexts. Figure 6 below shows how they
create the “support” category, based on the initial questions, and the codes used to identify answers
to those questions in the text of the interviews.
4.3. Process tracing and contribution analysis

Process tracing and contribution analysis approaches are not as commonly used in education
research but are gaining recognition—particularly as approaches that aim to show correlation in
qualitative and mixed-methods studies (see Ragin 2014 and Maxwell 2012). Process tracing and
contribution analysis are methods designed to explain the mechanisms, or interacting factors, in a
given situation that can result in particular outcomes. Both approaches require a theory of change
to explain the processes and desired outcomes. The aim is to look for evidence that increases our
confidence in the existence of causal mechanisms by comparing the evidence for the theory of
change with evidence of the existence of alternative mechanisms (see Befani and Mayne 2014 for
details on both contribution analysis and process tracing; see also Bennett 2016).
These analyses can be used to answer questions related to the conditions and mechanisms that
affect implementation and outcomes; they can also help answer questions about whether an
education program can be effective in different contexts. Process tracing and contribution analysis
can be particularly useful in historical and policy analysis, critical realist evaluations, case studies,
and qualitative longitudinal research. For example, Dryden-Peterson and Mulimbi (2017) describe
how process tracing was used to analyze policy documents, among other data, over time in order
to explain whether and how a specific set of educational policies resulted in a reduction of conflict

35
or whether there were alternative explanations for this outcome. Le Fanu (2013) also uses this
analysis when looking at the rollout of a new inclusive education curriculum in Papua New Guinea,
by tracing the elements that generate a disconnection between the plan and the practice. Le Fanu
looks at different elements (e.g., perceptions of the nature of the education system) that may create
challenges in implementing the curriculum and its inadequacy for the Papuan context.
Figure 6: From questions to categories using grounded theory. An example.

Source: Kirchoff and Lawrenz (2011)

4.4. Qualitative longitudinal analysis

Qualitative longitudinal analysis considers how time relates to data and how the data show change
or continuity of the phenomenon under study. Saldana (2003) suggests that the key framing
questions for such analysis are: When do changes occur through time? What contextual and
intervening conditions appear to influence and affect participant changes through time? And what
are the dynamics of change through time? A critical question in longitudinal analysis is to decide
the length of time and the number of waves of data collected in order to answer questions of how
time matters for a particular phenomenon. This analytical approach is particularly useful in
answering questions related to unpredicted outcomes from an educational program (spillover
effects), and whether and how those outcomes affect students over time and for which students.
For example, Morrow and Crivello (2013) provide a rich discussion of analyzing qualitative
longitudinal data for the Young Lives study over four waves of data collection, following two
cohorts of youth in four countries. In one example, they follow the story of a girl leading up to her
early marriage in Ethiopia, who was interviewed four times between 2007 and 2014, and find that
continuous and compounding experiences of food scarcity, illness (personal and family), and the
difficulties of balancing work and school are connected to her early marriage and her school

36
dropout. 23

4.5. Critical discourse analysis

Critical discourse analysis is an analytical approach that aims to deconstruct language and how it
is used to perpetuate or create existing social and political structures of power. There are different
approaches to critical discourse analysis (see Gee 2014 or Fairclough 2013), but they all focus on
conducting close analysis of the linguistic features of discourse or documents, and how these
discourses are taken up and used by others, as well as how these discourses are connected to larger
social and political practices that reproduce power inequalities. Critical discourse analysis can be
particularly useful with critical policy analyses; it also can be used with analyses of narratives or
in historiography.
For example, DeJaeghere, et al. (2013) conducted a review of policies and texts related to ethnic
minorities and education, using selected policies and governmental decrees written and enacted
from 1945 onward in China and Vietnam. The analysis looks at both the content as well as the
intention of the documents and the contexts within which the policy documents were written and
used. The authors find that ethnic “differences” and inequality are constructed similarly in these
two societies in relation to both Marxist-socialist ideology and a market-economic ideology. An
example of meanings associated with ethnic minorities is illustrated in their analysis of the
emergence of the 56 ethnic categories recognized by the Chinese government. They find that these
categories are a result of a centralized process taken by the government starting in the 1950s, which
used specific criteria to merge the more than 400 ethnic groups into an official list of 54 (which
then was added to, taking the total number to 56, including the Han). They also find that policy
texts through time refer to ethnic minorities as backward or un-modern, which leads to policies
focused on economic redistribution (e.g., boarding schools) but silent on status recognition.
Educational initiatives such as bilingual education are also linked with a discourse that positions
ethnic groups’ social status as inferior. 24

23
Other examples of longitudinal qualitative research include McLeod and Yates’ (2006) book Making Modern Lives:
Subjectivity, Schooling, and Social Change, which reports results of a study that followed secondary students through
their schooling, and into their first year after school, to examine their trajectories and biographical development, and
Bamattre, et al.’s (2017) study on the Learn, Earn, and Save Initiative.
24
Vavrus and Seghers (2010) example presented earlier in this note is also for an example of using critical discourse
analysis with policy texts, when they analyze the use of the term partnership in public documents in Tanzania. Souto-
Manning (2007, 2014. among others) has focused on critical discourse analysis in education, with a particular focus
on ethnicity and inclusion.

37
 Section 5: Implementing rigorous qualitative research
As with any data collection effort, good implementation of qualitative fieldwork and sound data
management are critical to producing a high-quality study. Regardless of the amount of data to be
collected, careful planning is central to qualitative research. As previously discussed, qualitative
research design is an iterative process that allows for adaptability to the context and, as such,
challenges researchers’ planning. It is important to consider the necessary time and money to
collect data, especially when the qualitative designs needed to best answer research questions are
resource intense. While the image of a lone researcher conducting qualitative or ethnographic
studies may come to mind, qualitative studies are, in fact, rarely carried out alone. Normally, a
research team is involved, including, for example, local and international researchers. Planning
requires sorting out all kinds of details, from training the research team to accessing field sites,
translating and transcribing data, and storing data.
This section covers in more detail some of the critical steps in successfully implementing a
qualitative research design. These steps 25 are summarized in Figure 9 and described in more detail
below.
Figure 7: Processes for qualitative data gathering and analysis

25
Note: These processes can be iterative, and some of these processes may exceed the length of the study period (e.g.
data management).

38
 5.1. Designing protocols

Figure 8: Processes for qualitative data gathering and analysis- protocols preparation

Data collection protocols follow from a clear description of the study purpose and the main
questions and frameworks guiding the study. Whether for interviews, observations, or
participatory methods, these protocols should be designed in relation to the research questions and
methodology guiding it. For example, if a study uses a critical realist approach to answer questions
about the mechanisms or contextual factors that affect outcomes of an education program, then the
protocols need to allow for questions related to both the mechanisms and the contextual factors
theoretically assumed to affect the outcomes. In addition, the methods should include questions or
observations that allow for unknown mechanisms or contextual factors to be revealed.
In emergent designs—when data collection and analysis procedures are designed to be flexible
enough to adapt to new ideas, concepts, or findings that arise while conducting the research—such
flexibility is to be included in the protocol. This means that the protocols, such as the interview
questions, need to be designed to allow for revisions, such as including new questions related to
the emerging themes. For example, life histories and qualitative longitudinal studies might want
to ask how the participant has changed regarding the phenomenon under study, and, when doing
so, be flexible enough to follow the answer given. For example, the Young Lives qualitative
longitudinal study (Crivello and Wilson 2016) starts each round by asking children about major
changes in their families in the years between interviews (“Can you tell me about major changes
that have taken place in the home in the last three years?”) and follows with how the past years
have been for the child (e.g., happy or sad).
Protocols should be piloted on a similar group of participants to refine how questions are asked
and observations made. When working with an international team (i.e., a research project that
includes people representing multiple national and language groups), the protocols will need to be
developed in one language and translated to another. A good practice in these cases is to translate

39
the protocols from the original language into the language of data collection and back. If for
example, a term does not exist or has a different meaning in the language of data collection,
comparison of the original and “back-translated” versions will highlight points to address.
One example of a highly structured protocol aiming at a high consistency of information themes
across different field teams is the one used for the World Bank’s qualitative assessment on gender
across countries (Turk, et al. 2010). The protocol, which was implemented in 10 countries,
provides guidance to field teams on the study’s main concepts and rationale and the sampling
selection, as well as details on how to translate and adapt the data collection tools, how to
implement the data collection tools, and the protocols for each tool (questions for interviews and
focus groups). It also covers how to document data and report on completed fieldwork. It is
important to note that the way one conducts a focus group discussion or chooses sites, for instance,
reflects the purpose of the study and how key concepts are defined, which point to epistemological
assumptions. In this example, “inequitable gender power relations and social norms” (p. 4) were
assumed to be important concepts and inform how data should be collected (e.g., through the types
of case studies the protocol specified). In designing a protocol such as this one, researchers should
carefully explain their study’s conceptualization to inform decisions by field teams. They should
also decide how to manage the tension between gathering comparable data about known or
theorized phenomena and attention to contextually specific phenomena which may challenge or
counter assumptions.
5.2. Sources and sampling

Figure 9: Processes for qualitative data gathering and analysis - sources and sampling

Sampling is the process of identifying participants or sources of data. Most often sampling is at
the participant level, but it could also be at the case/site level, depending on the methodology.
Qualitative research operates from a different principle in relation to sampling than quantitative
data. Some common features of qualitative sampling include:
• Qualitative sampling is generally purposive, meaning that participants are selected because
they are likely to generate useful data for the project, with a focus on the quality of the data,
which is more important than the total sample size.
• Sampling can be an iterative process in that researchers may seek more data to answer the
research questions and reach saturation (see Box 4), as well as to confirm or disconfirm ideas

40
 and topics that arise from the data as it is being gathered.
• The total number of participants or cases does not indicate whether the data are representative.
Qualitative research does not aim for empirical generalizability but can produce theoretical
generalizability, which means theories developed for the phenomenon and situation
investigated can be applied to other contexts or similar issues.
• Sampling may be informed by a principle of comparability if the research design and study
questions are seeking explanations across groups or sites. 26
Irrespective of the strategy chosen for sampling (these are detailed in Table 4), the sampling
strategy, as with every other aspect of qualitative research, should be related to and adequate for
the study purpose and design. Sampling approaches are not necessarily used discretely, meaning
that more than one type can be used in a study, particularly if qualitative approaches are used in
combination with quantitative methods and sampling.

Box 4: A note on data saturation

A frequently asked question among those commissioning or using the results of qualitative
research is, “How many [interviews, observations, etc.] are good/needed/sufficient/adequate?”
In the case of qualitative research, the answer to this question has to do with the notion of
saturation. Saturation occurs when enough data is collected to determine key themes and
provide sufficient explanation of the phenomenon being investigated, or when new data provides
no new information.
Sample sizes are partly drawn based on an expected saturation point (see the section on sampling
for more details). But sample sizes might also vary depending on the purpose of the study and
its theoretical assumptions. As with other aspects of qualitative research, flexibility is required,
as saturation is best assessed during or after completing the fieldwork process.
Saunders, et al. (2018) review the notion of saturation and identify four models summarized in
the table below:

Model What is it When does it

apply

Theoretical saturation Related to grounded theory methodology, Sampling

refers to no new information for the creation
of theoretical categories

Inductive thematic No new information for new codes or Analysis

saturation themes

A priori thematic Data covers identified codes or themes Sampling

26
See Patton 2005; Teddlie and Yu 2007.

41
 saturation

Data saturation New data repeats what is already in existing Data collection
data

Source: Saunders, et al (2018).

Based on their table, three types of saturation are relevant when deciding sample sizes. A priori
thematic saturation is the closest to defining a fixed set of data (a sample) before fieldwork.
Theoretical saturation is where an initial idea of a sample is refined using the discovery process
as data are analyzed (Are new things mentioned that suggest new categories? Are these
categories sufficiently informed?). And data saturation is based on what happens when data is
being collected—for example, when new interviews are not providing new insights or
information compared to the ones already conducted.
Some attempts to systematize saturation points across qualitative research (such as Guest, et al.
2006, Hagaman and Wutich 2016, and Namey, et al. 2016) have come up with a proposed rule
for thinking of sampling. They suggest a minimum of four to six interviews and a maximum of
12-20 and between three and six focus groups. These proposed sample sizes are not for the
whole study but for the identified relevant categories of subjects within the study. This means,
if a study aims at capturing the differences in experiences between young women and young
men in an urban and a rural school, the number of interviews will need to cover each of all four
groups.

Table 4: Types of sampling in qualitative studies

Type of Purpose Example

sampling 27

Criterion Identifying and selecting cases based on a set of E.g., New teachers who are first
pre-determined criteria; this could include time teaching in rural schools.
selecting cases that are homogenous.

Maximum Documenting diverse variations to identify E.g., Rural teachers across

variation common patterns that cut across variations. income levels or locations, both
old and new to rural schools,
both new and old to teaching

27
See Palinkas, et al. 2015 for more types and details on qualitative sampling

42
 Type of Purpose Example
sampling 27

Theory-driven Selecting participants based on the theory of E.g., Newly graduated teachers
change and seeking confirmatory and that are originally from rural
disconfirmatory evidence; this may include a areas and go to teach there are
form of maximum variation to select participants better at incentivizing students
with a wide variety of experiences to help inform than teachers of non-rural
the development of a theory. origin.

Iterative, Selecting participants based on emergent theory E.g., Teachers that were rural
snowballing, or findings, also facilitating the identification of teachers in their early career and
and emergent hard-to-find cases. stopped teaching. Starting from
a reference from a given school.

Extreme, Selecting participants to explain unique or E.g., Long-term rural teachers

unique, outlier cases in the phenomenon being studied. that have good results that
deviant, or Learning from highly unusual manifestations of deviate from others’
outlier case the phenomenon in question or adding detail to performance.
the average findings from quantitative-based
research.

Stratified or Selecting participants for comparability among E.g., As a first step, cluster rural
cluster sites/cases or linked to a mixed-methods study, schools by performance in
which may have a representative sample and standardized tests, then look at
participants are selected within that sample. rates of new teachers and select
schools/teachers based on such
indicators.

Intensity Providing rich information form a few select E.g., The five new teachers that
cases that manifest the phenomenon intensely arrived at a rural school in a
but not extreme cases. batch-allocation of rural
teachers.

The most frequently used sampling strategies are criterion, maximum variation, and theory-driven.
As noted in the table, criterion sampling identifies participants based on criteria set to answer the
purpose of the research questions. This form of sampling may overlap with theory-driven sampling
if the criteria relate to specific theoretical constructs being explored, such as a criterion of teachers
engaging with tutoring as a construct related to teacher absenteeism. More generally, criteria to
consider for case, site, or participant selection include being relevant to the research question,
accessible, and able to test tentative and contrasting explanations (Rubin and Rubin 2012).
Maximum variation sampling is based on heterogeneity, with the goal of capturing different
experiences or perspectives on a given phenomenon. This sampling strategy involves selecting key
43
demographic or other variables likely to have an impact on participants’ view of the topic or issue
under investigation. Based on these variables, a sampling “grid” is created, which will guide the
sampling, with participants selected to reflect various combinations of variables. Following on the
above example, we might think that years of experience, frequency of engagement in tutoring, and
level of absenteeism in the school might give us the maximum variation we require, hence there
would be nine groups of teachers to be interviewed, covering the different combinations of these
categories.
Theory-driven sampling involves selecting cases, individuals, or documents based on their ability
to address the theoretical concepts under study. The number of participants (or cases, etc.) is
selected using an iterative process until the data have reached theoretical saturation. If there is
considerable heterogeneity in the participants, or in the data, then a larger sample may be required.
For example, if the theory indicates that caste and age matter in explaining learning outcomes in
primary school, then a theory-driven sample will require participants who represent the relevant
caste and age groups. Similarly, if the data do not reveal patterns related to the phenomenon being
explained, additional participants will be sought to seek confirmatory or disconfirmatory evidence.
Moreover, if findings are to be compared across groups or sites, a stratified or cluster sampling
may be needed to produce sufficient evidence from each group or site.
Iterative, emergent, and snowballing approaches are common in ethnographic studies and life
histories. In these approaches, certain themes or participants become important in answering
emerging questions and findings. This means, for the case of iterative sampling, for example, that
researchers sample cases for data collection following information that emerges from the initial
data of each case sampled. Emergent sampling is also a good tool for exploratory research. Using
emergent sampling, the researcher decides on the sample as the fieldwork proceeds, as more
knowledge of the problem is gained (e.g., when using documents, as more documents are revised,
potentially a decision is to move from large policy documents to their operational manuals, as it
appears that those documents contain more detailed information), or when an opportunity arises
(e.g., if a new authority decides to re-draft the manuals that the research has collected).
Snowball sampling, as its name indicates, goes from one sampled individual or group to the next
one, picking up cases as it goes along with the assistance of participants. This approach may also
be familiar to quantitative researchers. In practice, each informant provides the contact information
of other informants, who then provide the contact information of additional ones. Snowball
sampling is useful when the research aims to sample difficult-to-find cases (e.g., school dropout
children in a context where non-attendance is punishable), cases that are difficult to access without
a reference (e.g., high-income groups), or cases that relate to sensitive, private, or stigmatized
situations (e.g., abuse victims).
Identifying extreme, unique, or outlier cases or participants is another type of qualitative sampling,
especially in case studies or life histories, where the purpose of the study may be to understand
certain positive or negative outcomes among particular cases or to understand unique phenomenon
among individuals. Bajaj (2009) offers an example of sampling unique cases. In this study, siblings
comprised the sample with one sibling participating in a unique school with a transformative
curricular approach and the other sibling attending a regular government school.
No matter the sampling strategy, qualitative (and mixed methods) studies should include
disconfirming cases or data. For example, if the research study aims to understand the processes

44
and mechanisms that resulted in high learning outcomes, the sample should also include cases
where the learning outcomes were low in order to be able to examine if the processes and
mechanisms are indeed different for high and low learning outcomes.
Attrition can be a challenge across these sampling strategies, and mitigation plans should
be put in place (see Box 5).

45
 Box 5: Challenges of attrition

Qualitative sampling is not immune to attrition, non-response, or take-up or participation

problems. Participants can decline participation, decide to interrupt or not complete data
collection or participate in one round of data collection and not the next. This is particularly
common in longitudinal qualitative studies that take place over the long term, where participants
might move, change contact information, or decide to not participate in consecutive rounds of
data collection. If attrition is a concern, studies should consider initial over-sampling, meaning
they should identify a larger set of participants to compensate for future losses.
People drop out of studies for all sorts of reasons, including respondent fatigue, migration, etc.
Strategies to deal with attrition include:
- In a multi-round study, track research participants between rounds to ensure attrition rates are
kept low.
- In longitudinal research, there is always a risk that the data gathering methods (e.g., interviews)
get progressively longer, as researchers will want to ask about change since the previous visit,
as well as potential new questions. Piloting research instruments can help to keep the research
encounter as brief as possible.
- Forms of “community” reciprocity, in which research teams return to communities and
individuals with summaries of their research findings, can help to keep participants engaged in
a study.
- Researchers should be prepared to answer questions from participants about the likelihood of
change/improvement in people’s lives as a result of the research, as well as requests for
compensation.

5.3. Data collection

Figure 10: Processes for qualitative data gathering and analysis – data collection

46
Data collection requires both a clear protocol and identification of those who will provide data and
an organized data collection effort and a well-trained team (if the data is not to be collected by the
primary researcher).
Managing primary data collection for qualitative research, similar to other forms of data collection,
requires a clear set of procedures and related documents (protocols for data collection),
including for sampling, note-taking, and managing potential questions or issues that arise (e.g.,
regarding the use of the data, potential payments for answering, etc.). The other three main
elements of quality data collection are trained personnel (previous experience is desirable but
does not obviate the need for training on the materials and protocols of the specific research), as
qualitative studies might require more expertise to carry out the data collection, particularly with
less-structured protocols; an adequate time-frame, including time to identify participants and
collect data with them (depending on the research purpose and thus the participants involved,
considerable time might be needed to identify or contact research participants); and sufficient
funding to cover for all fieldwork-related costs (including transport and lodging of the research
team when needed).
Who gathers the data is a critical question to consider when designing and carrying out qualitative
fieldwork. As noted above, when discussing participatory methods, data collectors could involve
the participants themselves, who may gather photos for a photovoice method or create a popular
theatre script, as guided by the field questions and/or a researcher or facilitator. For video
observations, participants involved in the study or a trained data collector may take videos. In
many observations, as in interviewing, the interviewer is the “tool” as is the protocol used, hence
who accesses the participants, develops rapport, and elicits meaningful information is not a trivial
choice.
The ideal scenario is that those involved in gathering data are also involved in designing the study,
but this is not always the case. Whether the data collection team is involved in the study design or
not, field team training is critical. Ideally, the team identified already has experience and
knowledge of qualitative approaches. Even if they do, to ensure quality data collection, all
members of the team should receive training that includes an introduction to qualitative
approaches; the overall study purpose, design, and qualitative methodology and approach; and a
thorough review of the instruments, including considerable practice applying the methods with
mentoring, debriefing, and support by their peers and the researchers that designed the study. 28
It is important that the field research team has a clear, shared understanding of the study’s purpose
and line of questioning, so they do not miss opportunities to pursue relevant themes. This
understanding also ensures the field research team does not ask questions that may be irrelevant
or highly sensitive. For example, if the research is to explore how parents see early childhood
learning and assess their children’s progress, the team should be aware of the possible relationship
between parents’ views on talking to a child that cannot yet speak and parents’ behavior as well as
how they perceive the relationship between nutrition and development if these are important to the

28
For more details, see the RECOUP manual on qualitative research skills workshop, which focuses on training
researchers in these different methods (Hassler, et al. 2008)

47
theoretical framing of the study. Sensitivity to issues around comparing children, asking about
specific milestones, or asking parents with low literacy about their reading habits is necessary,
whatever the framing. In addition, if the data collection is taking place in a place where qualitative
data collection is less accepted or prompts questions from the community or authorities, a clear
and good explanation of what is being done, why, and its value are important.
Training of researchers is vital here—in listening skills, context-adequacy, understanding what
constitutes a good question, and understanding how researchers’ own normative assumptions and
experiences may influence responses to questions, as well as the researchers’ presentation
(including dress, body language, terms of address, and the sharing of personal information).
The field team needs to discuss and be aware of their reflexivity (or awareness of how, as a
researcher, one can affect both the research processes and outcomes, given that the researcher can
and is influenced by the topic of the research and the participants’ answers). They must also be
aware of their positionality (how the researcher positions her/himself in relation to the researched
(e.g., as a friend) and how the research participants position the researcher) and how it can affect
interviewer-interviewee interactions and thus influence the nature and quality of the data produced.
These concerns extend to all field team members—including translators, note takers, and research
assistants—as their presence in the knowledge/data production process can introduce bias. 29 The
aim of any research encounter is to quickly set up mutual respect, trust, and rapport. Interviewers
can, for example, use sympathetic techniques by sitting at eye level, sitting neither too close nor
too far apart (respecting local norms), letting people hear their voices on the recording if they wish,
looking and sounding interested, listening carefully, gently reflecting back by repeating points
interviewees have made to affirm them, and checking verbally and through observing body
language whether people are comfortable to continue talking (Alderson and Morrow 2011).
Finally, training needs to cover practical issues, from seeking permission to record and taking
notes, to determining an appropriate time and duration, location, and other elements of the data
collection process.
The time needed for quality data gathering and the demands on participants’ time also need to be
considered in the process of gathering data. For example, there are no precise guidelines on how
long an interview or an observation should take—it depends how the method used produces the
data to answer the research questions, what is culturally appropriate for engaging in a conversation
or observation, as well as the time a person can dedicate to the process. 30 An assessment of the
quality of the data being provided is important. For instance, participants may become tired or
distracted given the time of day or their obligations. A final time-related element useful to include
in the training and guidance documents is how to write good field notes, including when to write
them and what should be included in these notes and other means of debriefing. 31
Logistical arrangements should take into account the level of volatility in the context (e.g., related
to climate fluctuations, security, or conflict situations) (see Campbell 2017, Berman, et al. 2016),

29
For more information see Bujra (2006) and Naveed, et al. (2017).
30
As a general rule, interviews—individuals or group—should not be planned to last more than one to two hours, and
that information needs to be provided beforehand to the participants.
31
See Emerson, Fretz, and Shaw (2011) for guidance on writing field notes.

48
and training should include these issues. Practical matters such as safe transport and supplies,
including electricity for computers in areas with intermittent energy supply, are also important to
consider. In-depth ethnographic research, for example, may require longer-term periods of
fieldwork or even permanence in a selected community, making contingency planning important.
Further, for educational research taking place in schools, researchers need to be aware of holidays
and festivals in the school calendar, as well as exam periods, when teachers and pupils are under
pressure, among other considerations.

5.4. Data management and transcription

Figure 11: Processes for qualitative data gathering and analysis – data management

Qualitative studies can produce large amounts of data and, as such, it is important to plan for its
use (both for immediate and later analysis). Data, depending on the method used, can include
recordings or notes of interviews or conversations, notes and images from observations, a large set
of documents or notes based on documents, videos, etc., as well as the field notes from the research
team. As such, an adequate data management plan is required, from the point the data is collected
through its handling, storage, and transformation (e.g., from recording to transcript).
Researchers should keep a thorough data log that includes all types of data and related information,
such as the date and location where data were collected, format of the data, duration, researcher
responsible, note taker, original language, and critical identifiers from the respondent, among other
pieces of information. This allows for data tracing and organization. Ideally, this data log
preparation should be done before any data transcription or coding. For studies that involve
multiple waves of data (ethnographic, longitudinal) or multiple sources, the organization of data
files by time and group is critical for finding data later. As with quantitative surveys, the originals
need to be properly stored and organized such that they can be accessed as needed. Plans for their
future destruction or safe storage are also required.
Transcribing recorded data—audio or video—and organizing field notes and documents or
observations should be done in accordance with clearly organized protocols and under supervision.
A similar process is needed when transcripts or information are to be translated, for example, by
49
having more than one translator go over the same recording or document and having the field team
read through translations to discuss consistent word usage. The research team will need to make
conscious decisions about the stage at which translation will occur. Given qualitative research’s
emphasis on capturing perspectives authentically, there are advantages to analyzing data in the
language of the participants. However, much depends on the language capacity of the researchers
using the data and on storage, access, and reporting requirements. All of these processes are
important considerations for translating data, as translation in qualitative data is itself an
interpretive act, closer to analysis than mere data handling. 32 Time estimates and budgets should
be adequately considered for transcription and analysis. A one-hour interview can take two to four
hours to transcribe (depending on the transcription conventions that the study uses). Video
observations or group interviews (classroom recordings) can take longer because they involve
transcribing what multiple people say and do.
In terms of data storage, all data should be stored in a way that protects those involved in the
research in accordance with ethical protocols. Often, data are stored anonymously and in a manner
that restricts access to the research team (encrypted and/or password protected), and back-up files
should be included. 33
There is increasing interest in sharing and publishing data for review and use by others. While
there are ethical considerations in doing so with qualitative research, how data are shared and
published should be considered by the research as well as those commissioning the research, and
awareness of the potential impossibility of making the data public is important. 34 While most
qualitative data is not suitable for public access because of ethical concerns around anonymity,
there are some efforts to share data in an ethical way (see one example in Box 6).

32
See, for example, Lapadat 2000; Temple and Young 2004; Rubenstein- Ávila 2013.
33
Storage systems, from Dropbox, Box, or other cloud-based systems have become more common recently, as well
as centralized data storage severs.
34
See the ROER4D website for information on data sharing and open data practices: http://roer4d.org/3575

50
 Box 6: The Data Management Plan (DMP) Checklist of the Qualitative Data Repository

The checklist aims for a clear and detailed reporting of procedures to be followed throughout
the research process with regards to the data, some of the issues (not exhaustive list) include:
- Personnel that will work on the project and have data access: What specific responsibilities and
levels of access to the data (if applicable) will each individual or organization have?
- Formats in which the data will be collected and used: Examples: WAV audio files (e.g., of
recorded interviews); scans (e.g., of archival documents) in PDF format; photographed images
as JPEG files; typed transcripts of focus group conversations in .docx format; paper-based
handwritten notes;
- Data organization: Qualitative data example: paper copies of archival documents will be filed
alphabetically in a standard archival box.
- Processing and transformation of data: For example, auto-coding in qualitative data analysis
software.
- Handling of the pre-processed data sources (original versions) to guarantee security: All files
will be encrypted.
- Storage and preservation of data after project completion: e.g., after anonymization, the original
data sources will be discarded/erased, and the processed data sources will be archived at
repository X.
- Plans for data sharing: e.g., Data will be deposited with access restrictions with a data repository.
- Conditions for access and re-use: feasibility, metadata, anonymity, partial or full sharing.
Source: www.qdr.org. Center for Qualitative and Multi-Method Inquiry, Maxwell School,
Syracuse University

5.5. Data coding and analysis

Figure 12: Processes for qualitative data gathering and analysis – data coding and analysis

51
Data coding, the first step in data analysis, must also be clearly planned and outlined. Qualitative
data can produce a lot of data, which can be complicated to manipulate and analyze. Teams and
studies can benefit from focusing on a clear, limited number of questions or themes to start the
analysis.
The analysis of qualitative data requires time, as it is an iterative process of coding, note making
(known as memoing, meaning notes linking data to concepts, constructs, and theories), analyzing,
and reviewing the data. There are many different approaches to qualitative data analysis—
following the diverse methodologies discussed above—but there are also some common features.
Miles, Huberman, and Saldana (2014) offer some common features to the many different types
and steps of analysis; the ideas below are adapted from their list.
The analysis process should follow the purpose of the study and the research questions. For
instance, if the purpose is to identify new contextual factors that are not accounted for in existing
literature, then the purpose is exploratory and aims to identify these factors.
Analysis often begins during the data gathering phase, when researchers are already considering
the data and how they answer the research questions. This initial analysis can take the form of
memos for each type of data gathered or overall research notes for the project.
Analysis usually includes coding the data. The purpose of coding is to reduce large amounts of
data (sometimes hundreds of pages of words from interviews or text, or images) to more
meaningful parts that are conceptually/theoretically important. Coding involves looking for
patterns among small amounts of data and associating data with common themes or concepts and
looking for variations. The codes can emerge from the data (inductive) or they can come from the
conceptual framework that informs the study (deductive). Usually, qualitative coding is a
combination of both emergent and a priori defined codes 35. (See Box 7 for an example).
Coding is only a first-order analysis or an initial step in the process. The second step in analysis
requires making sense of the coding through identifying patterns or differences (or disconfirming
cases); making comparisons within or between individuals, groups, or cases; and testing alternative
hypotheses. These steps in analysis often involve relating the data to the conceptual framework
and the relevant literature to extend, negate, or propose new conceptual ideas.
While traditionally coding was done manually, a number of computer-aided data analysis packages
exist (approximately 25, the most common being ATLAS-ti, NVIVO, MaxQDA, and Transana). 36
These are software packages that help researchers organize and extract qualitative data. The
important point to note is that they do not automatically analyze the data for the researcher. The
researcher/research team must decide how to code the data, what lines of analysis/themes to follow,
and so on. Computer-aided packages are particularly useful with large research studies or large
teams to organize the data. Analysis software is important for storing coding and analysis so that

35
See Saldana (2014) for more on developing codes, a codebook, and coding practices, including inter-coder
reliability, for data.
36
All of these programs have online tutorials and guides for users. Some packages are useful for smaller studies;
others are better for mixed-methods and large datasets (Nvivo, MaxQDA); Transana is useful for analysis of digital
media, particularly videos.

52
the researchers or others wishing to analyze the data or refer back to it at later points in time.
Coding the data, however, does not mean the analysis is complete. Second-order analysis (or third
or more) is done to explain patterns, comparisons, relationships, and disconfirming evidence
among the themes to answer the research questions. These steps of coding and analyzing data vary
based on the methodological and related analysis approach taken.
When working across national contexts and languages, practical steps of data coding or
organization in relation to themes and topics can be defined. There is no prescribed best format or
gold standard; instead, several options can be considered. Local research teams can conduct the
analysis, with training provided to ensure all study members understand the overall study purposes,
questions, and frameworks to analyze the data accordingly. Local and international teams work
collaboratively to analyze data—working through multilingual researchers as well as translators
to discuss interpretations, meanings, and analysis. Or a stepped approach can be taken, whereby
local teams go over the data first, followed by the international team.
Secondary analysis of data where the data are collected and transcribed/translated and then
analyzed by another team is the most challenging as the analysis is often removed from the cultural
and linguistic context. To minimize the loss of contextual richness, country teams can produce in-
depth data gathering reports (Crivello, et al. 2013 and Turk, et al. 2010). Data can be reviewed
after each round of data generation or after the first set of interviews or observations.

Box 7: Systematic analysis of qualitative data: Young Lives

The analysis of qualitative data, and in this case longitudinal data, is a multi-stage process.
Rather than focusing solely on either cross-sectional findings or differences between two points
in time, such analysis aims to illuminate the trajectories or pathways individual children are
following and to understand how different factors interact to shape them.
Most Young Lives qualitative data take the form of text, although they also include pictures,
diagrams, and photographs. The majority of the text comprises transcripts of interviews, though
it also includes reports on group activities and texts such as diaries produced by the children
themselves. Each text file is transcribed and recorded in a common, collectively agreed format,
which allows it to be identified by date, research method, and participants.
The principal method for organizing these data to facilitate systematic longitudinal analysis is
coding and categorizing text sections according to a mutually agreed framework. Once the
framework for each round is finalized, each section of the text is coded according to its content
using the qualitative data analysis software Atlas.ti. Once coded, the whole dataset can be
searched according to different codes and intersections of code, facilitating systematic analysis.
The first stage of analysis in each round tends to be descriptive, summarizing the data and
highlighting emerging themes, patterns, and connections, while subsequent stages reflect the
specific focus and research questions developed by each country team.
The principal divisions in this framework, known as “super-families,” follow the core themes
of the longitudinal qualitative research component, which are services, transitions and changes,
and child well-being. Each superfamily is subdivided into “families,” topics that are derived

53
from the main category. In the case of the transitions and changes super-family, for example,
the family categories are expectations, relationships, educational transitions, and other
transitions.
All country teams use the same framework of super-families and families, but a third level of
the framework, comprising detailed sets of codes within each family, is unique to each country.
In Peru, for example, the educational transitions family was further sub-divided into
communications and exchange between teachers, communications and exchange between
teachers and parents, school organizational arrangements, school material resources, teacher
training, adaptation, attitudes, expectations, support, children’s readiness for school, children’s
feelings, importance of preschool, and difficulties (Crivello, Morrow, and Wilson 2013).
Analysis can be approached either by theme or by individual:
• A ‘horizontal’ reading of the data is undertaken through the themes and sub-themes identified
in the coding framework, enabling the identification of trends, similarities, and differences
between and within the research sites.
• A ‘vertical’ reading of the data entails starting with selected individual children and gathering
all the available information about them in order to construct a detailed, composite image of
their lives, and to triangulate data collected at different times.
Making such horizontal and vertical readings was relatively straightforward with a single round
of data but has become successively more complex as subsequent rounds of data have been
collected. Research teams agree on strategies for summarizing longitudinal data. Tables and
matrices constructed to track changes in the lives of case study children over time are an
important tool for condensing data, guiding researchers to extract coded material from previous
rounds. Case histories are constructed by examining all interviews with one child over
successive data collection rounds, dividing the data into different domains such as education,
work or risk, and creating a narrative or storyline for each domain.
Source: Young Lives 2017a, 2017b.

54
 Section 6: Ethics related to qualitative research
Ethics in research refers to the moral principles guiding research, from its inception to completion
and publication of the results and beyond. Research ethics guidance ensures that the principles of
justice, respect, and harm prevention are upheld. The past 25 years have seen an expansion of
research governance globally, and now research ethics approval systems and frameworks exist, to
a degree, in all parts of the world. Qualitative research is often assumed to raise greater ethics
concerns than survey-based research because of the co-production/face-to-face nature of the
research and the interaction/relational nature of the research process. But this perception is very
misleading—survey enumerators also co-construct data in gathering survey data. Although ethics
approval systems may seem burdensome, it is helpful to note the positive benefits of the systems,
notably that they challenge researchers to be clear about the aims and purposes of their research
and what participants are expected to do. 37 One way to think positively about ethics in social
research is to bear in mind that ethical research involves 5Rs—relationships, respect, relevance,
responsibility, and reciprocity (adapted from Markiewicz 2012) 38.
Ethical guidelines for educational research such as BERA (2011) and AERA (2011) can guide
research processes for studies in other countries. In addition, countries and institutions may have
their own research guidelines and processes, and these should be followed.
Ethics questions in research can be usefully discussed under a series of 10 topics (adapted from
Alderson and Morrow 2011). These can be framed as questions to address during the lifetime of a
research study, such as, is the research worth doing? Can the investigators explain the research
clearly enough so that anyone invited to participate can give informed consent?
The 10 topics are set out below: the first six topics involve the planning stages; topics seven and
eight involve the data-collection stage; and the final two involve the writing, reporting, and follow-
up stages.
6.1 The purpose and methods of the research

Are the research questions worth asking, and why? In whose interests are the questions being
asked? Do the methods fit the aims of the research? If children are involved, what “standpoint”
are researchers taking when they study children? How are children’s competencies respected?

37
Researchers are increasingly giving attention to the ethics of educational research involving children in low- and
middle-income settings—see Holliday 2013, Honan, et al. 2012, Jadue Roa 2017, Obaidu Hamid 2010, Pillay 2014,
Robinson-Pant and Singhal 2013, Sikes 2013, and Sinha 2017. See also Mikesell, et al. 2013 and Friedman Ross, et
al. 2010 for useful overviews of ethics considerations in relation to participatory approaches and community-engaged
research.
38
Markiewicz’s article develops a number of principles for evaluators to follow in evaluating programs. These
include: “having respect for the importance of historical, socioeconomic and psychological context; commitment to
ensuring relevance in methodologies and approaches used; reciprocity in considering the benefits for participating
Indigenous communities; and responsibility in undertaking effective communication and consultation”.

55
 6.2 Assessing harms and benefits

What contributions are people asked to make to the study? What are the risks or costs, such as the
use of their time, embarrassment, and intrusion? What are the benefits of taking part in the
research, such as satisfaction, increased knowledge, and time to talk to an attentive listener? (The
principle of beneficence means that research participants should benefit from the findings of a
study. 39) If children are involved, what contributions are they asked to make to the research? Will
they be tested, observed, or recorded? Are there risks if the research is not carried out? Does the
research address sensitive questions about teacher performance/testing teachers, absenteeism, use
of corporal punishment? Does the research use surveillance methods, such as using observations
from video/CCTV?
6.3 Privacy and confidentiality

What are the limits to privacy? Are people able to opt in to the research? Is it coercive to send
reminders? Where will the research be conducted, in a quiet, private place? How realistic is it to
achieve privacy in many settings, especially in research directly with children? Will names be
changed in reports of the research to protect people’s identity? How will data be protected, i.e. will
research records, notes, films, digital material be secured and de-identified, and in what way? Will
researchers check back with participants about the use of long quotes, etc.? Will research records
be destroyed at the end of the study? 40
Research in educational settings/schools inevitably involves children, so thought needs to be given
to how children are involved. Is it really necessary to include children? If yes, can children opt in
to the research? Can they opt out of the research? 41 In undertaking research in schools, children’s
voluntariness is questionable and must be handled with care. Thought also needs to be given to
how to handle privacy in the classroom.
Ethics questions arise in relation to breaching confidentiality in case of child protection concerns.
If researchers think that a child is at risk of harm or in danger, and they feel they must report a
child’s confidences, do they try to discuss this first with the child? Is a referral system in place to
support vulnerable children, and in the absence of any formal support systems, have researchers
considered what to do? What do researchers do, for example, when they observe corporal
punishment in a school? Further, in some countries, researchers are required to have a police check

39
For example, if a study informs a new curriculum, the researchers should ensure that the new curriculum makes its
way back to the schools that participated in the research.
40
In relation to data protection, new regulations came into effect in 2018 relating to the processing of personal data
and the free movement of data between countries under the General Data Protection Regulation (GDPR). Governments
globally are adjusting legislation accordingly. Researchers are advised to check national legislation and seek
clarification from with local data protection officers about the implications of this, and research with children in
particular, noting the special provisions relating to children’s data.
41
It is important to differentiate between consent and assent. Consent may only be given by individuals who have
reached the legal age of consent. Assent is the agreement of someone not able to give legal consent to participate in
the research but deemed old enough to understand the proposed research in general. Work with children or adults not
capable of giving consent requires the consent of the parent or legal guardian and in most cases the assent of the
subject.

56
before undertaking research in schools.
6.4 Selection and participation

Why have the individuals concerned been selected to take part in the research? Are the research
findings intended to be representative or typical of a certain group of people? Who is left out
of/excluded from the research? In research with children, have some children been excluded
because they have speech or learning difficulties, for example? Can such exclusion be justified?
Are the research findings then representative of certain groups of children?
6.5 Financial considerations relating to time, compensation

Does funding allow for time and resources to collect and analyze data respectfully and accurately?
Should participants be compensated for their time? Should this be a reward upon the completion
of the data-gathering process to avoid “incentivization”? What constitutes a fair return for help
and services? How does this differ by context? In relation to research with children and young
people, consideration needs to be given to how to compensate children and young people for their
time. 42
6.6 Reviewing the research—institutional review boards and research ethics
committees

Research ethics committees are now commonplace in high-income countries, especially in UK and
USA universities, and are increasingly available in low and middle-income countries, and there
are often mechanisms in place for in-country ethics review. These committees differ widely, and
the onus is on researchers to check—regularly—what is required in each context. 43 Often, there
will be research ethics committees available to review public health-related research (health
defined in the broadest sense), and educational researchers may feel their study falls outside the
bio-medical model. Yet approval is still needed. Advisory groups/boards and national steering
committees that monitor and guide the research process can be very helpful here—and in some
cases, seeking ethics approval in study countries may result in greater uptake or “buy-in” from
stakeholders, as well allow for constructive feedback on the research design/process.

42
Payment—and, indeed, any form of “research reciprocity”—can lead to misunderstandings and difficulties within
communities, especially if some people/children are included in research and others are not, and especially in poverty
settings (see Morrow 2009, 2013).
43
E.g., national guidelines in Ethiopia enable waivers of informed consent if the research project “carries no more
than minimal risk, if the research… is to be conducted or approved by federal or regional government and is designed
to study, evaluate, or otherwise explore public benefit or service programs….” On the other hand, in Vietnam,
increased concerns about over-testing of children in the formal education system have meant that it may be difficult
to obtain approval to undertake research with children that involves them taking further tests.

57
There are further questions about review to be considered, including the following: have
participants helped to plan the research or been invited to comment on it? Is the design unhelpful
to participants in any way, e.g., are they asked to talk about matters that might embarrass them?
Are children being asked to answer questions they cannot know the answer to—not because they
are incompetent but because they have not been taught or because the questions/tests are irrelevant
or completely meaningless to them? How do researchers manage situations in which some children
in a particular class cannot read or write, so cannot complete a test or questionnaire? Are questions
deceptive? Can the research design be improved? Are there ways of dealing with complaints? Are
local understandings of, for example, informed consent, taken into consideration?
6.7 Information

It is vital to inform people involved in research, and researchers usually use a combination of
written and oral/spoken information in order to do this. An information leaflet helps ensure that
accurate, standardized information is given. It is important that researchers spend time orally
explaining the research and check people’s understanding, as well as answer any questions.
If research is taking place in a school, are adults (teachers, parents/caregivers) and children
provided with details about the purpose and nature of the research? Do the researchers explain the
study and encourage children/parents to ask questions? If parents and children are not informed,
how is this justified?
Researchers working in educational settings should ensure that children know and understand that
participation in a research study is separate from their usual lessons, and that refusal to
participate/withdrawal from participation will not affect their participation in any standard
educational activities, nor their grade or class position. Further, care should be taken not to disrupt
children’s schooling or other regular activities such as extra-curricular activities and part-time
employment.
In any context, parents/caregivers and others in the community and children themselves should be
informed about the research at the very least, even if the requirement for “informed consent” has
been waived by a research ethics committee. People are highly likely to see researchers in the
community and wonder what they are doing there, and any concerns can be allayed by careful
explanation. 44
6.8 Consent

Consent—in other words, saying yes or agreeing to participate—is the central element of research
ethics. While most guidance on consent derives from medical research and treatment, the values
of respect, trust, clear information, and good communication apply to consent to any kind of
research. Covert and/or deceptive research is seen as unethical.
As noted above, consent needs to be informed and freely given. People should be given time to
decide, they should be able to ask questions and talk to other people before they decide, and they

44
See also Shamin and Qureshi 2013.

58
should be able to refuse or drop out at any time without giving a reason. People also need to be
reassured that if they decline, it will not be held against them, and the services they receive will
not be affected in any way. Consent may be spoken or written on a consent form. Researchers may
audio-record oral consent.
Most research guidance now insists that children and young people give informed consent to
participate in research. This, then, begs the question of the age at which children can be deemed
competent to give informed consent, and this varies between countries. 45 In general, researchers
are advised to seek the consent of both parents and children to participate in research, unless
researchers can show that the risks are small enough to rely on children’s consent alone.
Researchers in schools need to seek a range of permissions—from education ministries to regional
and district education offices, headteachers, and teachers—and may find it difficult to insist on
requesting parents’ and children’s consent when they depend on the goodwill of staff at the school.
As already noted, guidance is that ethical research involves informing and respecting everyone
involved. There are creative ways of informing parents/caregivers and communities about
research, for example by researchers holding meetings at the school to explain their research and
answer concerns. Children’s refusal to participate must be respected.
6.9 Dissemination and implementation of findings

Do research reports show a balance and range of evidence? Qualitative research is sometimes
criticized for “cherry-picking” quotations that illustrate the findings researchers want to highlight,
so it is important to use and discuss the full range of evidence to show balance. Will participants
(including children) be invited to comment on the preliminary analysis of findings, to check the
researchers’ interpretation? Will participants be sent short reports of findings? Not least, this is a
useful way of saying “thank you,” and it is particularly important in longitudinal research in which
a long-term relationship between participants and research teams necessarily develops. Finally,
how can the research be made relevant and usable to country/local communities and participants?
6.10 Impact of the research on wider groups

What collective effect can research findings have on attitudes and policies toward particular
groups, and how might the conclusions of one study affect wider groups of similar participants?
What ethical questions are raised by news reporting of research findings that may be
sensationalized? In relation to teachers, see the blog: http://www.younglives.org.uk/node/8329.
Most ethics guidance focuses on the data collection stage, but there are ethical questions raised by
the likely effects of published reports, both on people who participated in the research (who may
feel let down or angry when they see how the data they have produced has been used to draw
conclusions) or on wider groups as policies, practices, and public opinion may be influenced by
the conclusions drawn from research findings. This is particularly the case for children and young

45
E.g., in US the concepts of “assent” and “mature minor” are used in relation to informed consent, but these are
difficult to translate and confusing for researchers in practice.

59
people who lack the power, status, and resources to challenge research findings about them—but
also for low-status teachers and others who work in schools around the world.

60
 Section 7: Identifying reliability and validity in qualitative research
The reliability and validity of any research are crucial aspects of its quality. 46 While these concepts
have distinct meanings in a quantitative context, where reliability concerns the collection and
processing of data and validity the truthfulness of the conclusion drawn from the data, in a
qualitative context such a distinction may not be appropriate. Instead, a number of techniques may
be used to ensure both reliability and validity of qualitative research. 47 These techniques (adapted
from Hadi and Closs 2016) include:
• Reflexivity and positionality: Quality research openly and transparently presents and discusses
assumptions and biases and how they affect the data gathering and analysis process. By keeping
field notes or journals of how the researcher’s role and assumptions may have influenced the data
gathering and analysis and then discussing examples of this, the researcher aims to illustrate the
credibility of the analysis.
• Triangulation: This involves checking inferences from data across different sources and methods.
Triangulation should not be used to check the accuracy or “truth” of one source of data against
another but rather the overall inferences and where biases might emerge.
• Prolonged engagement: Prolonged engagement in the context and with the participants enables
trust and relationships to form, which are helpful in obtaining dependable and credible data.
• Thick description: Thick description addresses the principles of openness and transparency.
Researchers provide adequate details of the study setting, participants, data collection processes,
and data to allow the reader to understand the claims made.
• Audit trail: Like thick description, the audit trails provide adequate details about the study data
collection processes, decisions, and analysis. The reader can judge the strength of the evidence
based on these details as well as the adequate use of data.
• Peer debriefing: This involves sharing emerging hypotheses, analysis, and findings with other
researchers to seek feedback and alternative perspectives on the analysis.
• Member checking: Like peer debriefing, but here participants review the data and findings from
the study as a way to help ensure greater dependability and credibility.
Many of these checks can be assessed in the writing and presentation of qualitative data analysis.
A high-quality report, as noted earlier clearly articulates the qualitative design, data gathering
processes, and analysis processes, as well as the approach used in the research design.
Conducting sound and rigorous qualitative research that can have an impact on the education sector
means being clear on the purpose of the study. It is the study’s purpose that guides the myriad
choices that face researchers, and which we have addressed in this note. These choices start in the

46
See Appendix 1 for a full checklist of considerations relevant to assessing research quality.
47
Some qualitative researchers (Lincoln and Guba 1985, p. 290) argue for abandoning the concepts of reliability and
validity for qualitative research, and instead use “trustworthiness.” The approach here retains reliability and validity
as dimensions of both qualitative and quantitative research, consistent with Building Evidence in Education (BE2)
(2015). Assessing the strength of evidence in the education sector.

61
design phase, in which researchers must reflect on and describe their context and their ontological
and epistemological assumptions to appropriately frame their study. These assumptions shape
research questions and methodologies, which in turn influence the choice of methods, sampling
strategies, and analysis approaches. A strength of qualitative research is that studies can adapt to
changes in unpredictable or evolving situations that affect education, such as the life of a child, a
curriculum reform, or a policy change in ways that generate important insights. Rigorous
qualitative research, as illustrated in the many examples cited throughout the note, can affect (and
has affected) decision-making within education. There is a clear need and demand for extending
the base of qualitative evidence in education. This note, and related open-access videos, aim to
help funders, researchers, and users of qualitative research assess how such studies are ethically
and methodologically rigorous so that they can contribute to and impact educational policy and
practice.

62
 References
“AERA Code of Ethics: American Educational Research Association Approved by the AERA
Council February 2011.” Educational Researcher 40, no. 3 (April 2011): 145–56.
https://doi.org/10.3102/0013189X11410403.
Alderson, Priscilla. Listening to Children: Children, Ethics and Social Research. Ilford:
Barnardos, 1995.
Alderson, Priscilla, Virginia Morrow, and Priscilla Alderson. The Ethics of Research with
Children and Young People: A Practical Handbook. 2nd ed. London: SAGE, 2011.
Altinyelken, Hulya Kosar. “Curriculum Change in Uganda: Teacher Perspectives on the New
Thematic Curriculum.” International Journal of Educational Development 30, no. 2 (March
2010): 151–61. https://doi.org/10.1016/j.ijedudev.2009.03.004.
Anderson-Levitt, Kathryn M. “Complicating the Concept of Culture.” Comparative Education
48, no. 4 (November 2012): 441–54. https://doi.org/10.1080/03050068.2011.634285.
Annas, George J, and Michael A. Grodin. “The Nuremberg Code.” In The Oxford Textbook of
Clinical Research Ethics, edited by Ezekiel J. Emanuel, 136-140. New York: Oxford University
Press, 2008.
Bagamoyo College of Arts, Tanzania Theatre Centre, Richard Mabala, and Karen B Allen.
“Participatory Action Research on HIV/AIDS through a Popular Theatre Approach in Tanzania.”
Evaluation and Program Planning 25, no. 4 (November 2002): 333–39.
https://doi.org/10.1016/S0149-7189(02)00044-7.
Bajaj, Monisha. “‘I Have Big Things Planned for My Future’: The Limits and Possibilities of
Transformative Agency in Zambian Schools.” Compare: A Journal of Comparative and
International Education 39, no. 4 (July 2009): 551–68.
https://doi.org/10.1080/03057920701844503.
Ball, Stephen J. “WHAT IS POLICY? TEXTS, TRAJECTORIES AND TOOLBOXES.”
Discourse: Studies in the Cultural Politics of Education 13, no. 2 (April 1993): 10–17.
https://doi.org/10.1080/0159630930130203.
Bamberger, Michael, ed. Integrating Quantitative and Qualitative Research in Development
Projects. Directions in Development. Washington, D.C: World Bank, 2000.
Bartlett, Lesley, and Ofilia Garcia. Additive Schooling in Subtractive Times: Bilingual
Education and Dominican Immigrant Youth in the Heights. Nashville: Vanderbilt University
Press, 2011.
Bartlett, Lesley, and Frances Vavrus. “Comparative Case Studies: An Innovative Approach.”
Nordic Journal of Comparative and International Education (NJCIE) 1, no. 1 (July 11, 2017).
https://doi.org/10.7577/njcie.1929.
Befani, Barbara, and John Mayne. “Process Tracing and Contribution Analysis: A Combined
Approach to Generative Causal Inference for Impact Evaluation.” IDS Bulletin 45, no. 6

63
(November 2014): 17–36. https://doi.org/10.1111/1759-5436.12110.
Bennett, Andrew. “Process Tracing and Causal Inference.” In Rethinking Social Inquiry: Diverse
Tools, Shared Standards (2nd ed.), edited by Henry E. Brady and David Collier. Lanham,
Maryland: Rowman & Littlefield Publishers, 2010.
Berman, Jason Hart, Dónal O'Mathúna, Erica Mattellone, Alina Potts, Claire O'Kane, Jeremy
Shusterman, and Thomas Tanner. “What We Know about Ethical Research Involving Children
in Humanitarian Settings: An overview of principles, the literature and case studies.” Innocenti
Working Paper No. 2016-18, UNICEF Office of Research, Florence, 2016.
Bowen, Glenn A. “Document Analysis as a Qualitative Research Method.” Qualitative Research
Journal 9, no. 2 (August 3, 2009): 27–40. https://doi.org/10.3316/QRJ0902027.
Brady, Henry E. “Doing good and doing better: How far does the quantitative template get us.”
In Rethinking social inquiry: Diverse tools, shared standards, edited by H.E. Brady and D.
Collier, 53-67. Lanham: Rowman and Littlefield, 2004.
Braun, Virginia, and Victoria Clarke. “Using Thematic Analysis in Psychology.” Qualitative
Research in Psychology 3, no. 2 (January 2006): 77–101.
https://doi.org/10.1191/1478088706qp063oa.
Brewer, John. Ethnography. McGraw-Hill Education (UK), 2000.
British Educational Research Association. Revised Ethical Guidelines for Educational Research
(2011). Birmingham: British Educational Research Association, 2011.
Bruns, Barbara, Soledad De Gregorio, and Sandy Taut. “Measures of effective teaching in
developing countries.” Research on Improving Systems of Education (RISE) Working Paper,
16(009), Blavatnik School of Government, University of Oxford, Center for Global
Development, and Oxford Policy Management, 2016.
Building Evidence in Education (BE2). “Assessing the Strength of Evidence in the Education
Sector,” 2015.
https://www.usaid.gov/sites/default/files/documents/1865/BE2_Guidance_Note_ASE.pdf.
Bujra, Janet. “Lost in translation? The use of interpreters in fieldwork.” In Doing development
research, edited by Vandana Desi and Robert B. Potter. London: Sage, 2006.
Camfield, Laura, and Keetie Roelen. “Household Trajectories in Rural Ethiopia: What Can a
Mixed Method Approach Tell Us About the Impact of Poverty on Children?” Social Indicators
Research 113, no. 2 (September 2013): 729–49. https://doi.org/10.1007/s11205-013-0298-7.
Campbell, Susanna P. “Ethics of Research in Conflict Environments.” Journal of Global
Security Studies 2, no. 1 (January 2017): 89–101. https://doi.org/10.1093/jogss/ogw024.
Center for Global Development. “‘The Evidence’ About ‘What Works’ in Education: Graphs to
Illustrate External Validity and Construct Validity.” Accessed March 3, 2020.
https://www.cgdev.org/publication/evidence-about-what-works-education-graphs-illustrate-
external-validity.
Chambers, Robert. Rural Development: Putting the Last First. Prentice-Hall, 1983.
Chambers, Robert. “Rapid and Participatory Rural Appraisal.” Appropriate Technology 16, no. 1
64
(March 1990): 15–16.
Chambers, Robert. Can We Know Better?: Reflections for Development, 2017.
https://doi.org/10.3362/9781780449449.
Ciolan, Lucian, and Loredana Manasia. “Reframing Photovoice to Boost Its Potential for
Learning Research.” International Journal of Qualitative Methods 16, no. 1 (December 2017):
160940691770290. https://doi.org/10.1177/1609406917702909.
Clandinin, D. Jean, and F. Michael Connelly. Narrative Inquiry: Experience and Story in
Qualitative Research. San Francisco, Calif: Jossey-Bass, 2000.
Corbin, Juliet M., and Anselm Strauss. “Grounded Theory Research: Procedures, Canons, and
Evaluative Criteria.” Qualitative Sociology 13, no. 1 (1990): 3–21.
https://doi.org/10.1007/BF00988593.
Collier, Paul. “The Cultural Foundations of Economic Failure: A Conceptual Toolkit.” Journal
of Economic Behavior & Organization 126 (June 2016): 5–24.
https://doi.org/10.1016/j.jebo.2015.10.017.
Creswell, John W., and Vicki L. Plano Clark. Designing and Conducting Mixed Methods
Research. Thousand Oaks, Calif: SAGE Publications, 2007.
Creswell, John W., and J. David Creswell. Research Design: Qualitative, Quantitative, and
Mixed Methods Approaches. Fifth edition. Los Angeles: SAGE, 2018.
Crivello, Gina, Virginia Morrow, Emma Wilson. Young Lives longitudinal qualitative research:
A guide for researchers. Oxford: Young Lives (2013).
Crivello, Gina, and Emma Wilson. Young Lives Qualitative Fieldwork Guide: Round Four
(2013/2014). Young Lives Technical Note 34. Oxford: Young Lives, 2016.
DeJaeghere, Joan, Xinyi Wu, and Lisa Vu. “Ethnicity and Education in China and Vietnam:
Discursive Formations of Inequality.” Compare: A Journal of Comparative and International
Education 45, no. 1 (January 2, 2015): 118–40. https://doi.org/10.1080/03057925.2013.841034.
Denzin, Norman K. Interpretive Interactionism. Applied Social Research Methods Series, v. 16.
Newbury Park, Calif: Sage Publications, 1989.
Dryden-Peterson, Sarah, and Bethany Mulimbi. “Pathways toward Peace: Negotiating National
Unity and Ethnic Diversity through Education in Botswana.” Comparative Education Review 61,
no. 1 (February 2017): 58–82. https://doi.org/10.1086/689614.
Dudwick, Nora, Kathleen Kuehnast, Veronica Nyhan Jones, and Michael Woolcock. Analyzing
Social Capital in Context: A Guide to Using Qualitative Methods and Data, World Bank
Institute, Washington (2006).
Dunn, Julie, Penny Bundy, and Nina Woodrow. “Combining Drama Pedagogy with Digital
Technologies to Support the Language Learning Needs of Newly Arrived Refugee Children: A
Classroom Case Study.” Research in Drama Education: The Journal of Applied Theatre and
Performance 17, no. 4 (November 2012): 477–99.

65
https://doi.org/10.1080/13569783.2012.727622.
Dunn, William N. Public Policy Analysis. Routledge, 2015.
Elo, Satu, and Helvi Kyngäs. “The Qualitative Content Analysis Process.” Journal of Advanced
Nursing 62, no. 1 (April 2008): 107–15. https://doi.org/10.1111/j.1365-2648.2007.04569.x.
Emerson, Robert M, Rachel I Fretz, and Linda L Shaw. Writing Ethnographic Fieldnotes, 2011.
http://public.ebookcentral.proquest.com/choice/publicfullrecord.aspx?p=3563012.
Fairclough, Norman. Critical Discourse Analysis: The Critical Study of Language. 2. ed.,
[Nachdr.]. London: Routledge, 2013.
Fereday, Jennifer, and Eimear Muir-Cochrane. “Demonstrating Rigor Using Thematic Analysis:
A Hybrid Approach of Inductive and Deductive Coding and Theme Development.” International
Journal of Qualitative Methods 5, no. 1 (March 2006): 80–92.
https://doi.org/10.1177/160940690600500107.
Fletcher, Amber J. “Applying Critical Realism in Qualitative Research: Methodology Meets
Method.” International Journal of Social Research Methodology 20, no. 2 (March 4, 2017):
181–94. https://doi.org/10.1080/13645579.2016.1144401.
Frost, Nollaig, Sevasti Melissa Nolas, Belinda Brooks-Gordon, Cigdem Esin, Amanda Holt,
Leila Mehdizadeh, and Pnina Shinebourne. “Pluralism in Qualitative Research: The Impact of
Different Researchers and Qualitative Approaches on the Analysis of Qualitative Data.”
Qualitative Research 10, no. 4 (August 2010): 441–60.
https://doi.org/10.1177/1468794110366802.
Gale, Trevor. “Critical policy sociology: Historiography, archaeology and genealogy as methods
of policy analysis.” In Education, Globalisation and New Times: 21 Years of the Journal of
Education Policy, edited by Stephen J. Ball, (pp. 148–166). 1st ed. Routledge, 2007.
https://doi.org/10.4324/9780203963999.
Gaventa, John and Andrea Cornwall. “Power and Knowledge.” In The SAGE Handbook of
Action Research: Edited by Hilary Bradbury, 465-71. Third edition. Los Angeles: SAGE
Publications, 2015.
Gee, James Paul. An Introduction to Discourse Analysis: Theory and Method. Fourth edition.
New York: Routledge, 2014.
Geertz, Clifford. The Interpretation of Cultures: Selected Essays. New York: Basic Books, 1973.
George, Alexander L., and Andrew Bennett. Case Studies and Theory Development in the Social
Sciences. BCSIA Studies in International Security. Cambridge, Mass: MIT Press, 2005.
George, Martin. TES. “Exclusive: Education Research’s ‘Terrifying Existential Crisis.’”
Accessed March 2, 2020. https://www.tes.com/news/exclusive-education-researchs-terrifying-
existential-crisis.
Glaser, Barney G. “The Constant Comparative Method of Qualitative Analysis.” Social
Problems 12, no. 4 (April 1965): 436–45. https://doi.org/10.2307/798843.
Goodburn, Charlotte. “Learning from Migrant Education: A Case Study of the Schooling of
Rural Migrant Children in Beijing.” International Journal of Educational Development 29, no. 5
66
(September 2009): 495–504. https://doi.org/10.1016/j.ijedudev.2009.04.005.
Hadi, Muhammad Abdul, and S. José Closs. “Ensuring Rigour and Trustworthiness of
Qualitative Research in Clinical Pharmacy.” International Journal of Clinical Pharmacy,
December 14, 2015. https://doi.org/10.1007/s11096-015-0237-6.
Hamid, M. Obaidul. “Fieldwork for Language Education Research in Rural Bangladesh: Ethical
Issues and Dilemmas.” International Journal of Research & Method in Education 33, no. 3
(November 2010): 259–71. https://doi.org/10.1080/1743727X.2010.511714.
Hassler, B., R. Jeffrey, A. Sriprakash, and N. Singal. “Qualitative Research Skills Workshop: A
Facilitators Reference Manual.” University of Cambridge, 2008.
http://ceid.educ.cam.ac.uk/publications/RECOUP_Manual.pdf.
Holland, Jeremy, ed. Who Counts? The Power of Participatory Statistics. Rugby, Warwickshire,
UK: Practical Action Pub, 2013.
Holliday, Adrian. “The Politics of Ethics in Diverse Cultural Settings: Colonising the Centre
Stage.” Compare: A Journal of Comparative and International Education 43, no. 4 (June 27,
2013): 537–54. https://doi.org/10.1080/03057925.2013.797775.
Holloway, Immy. Basic Concepts for Qualitative Research. London ; Malden, MA, USA:
Blackwell Science, 1997.
Honan, Eileen, M. Obaidul Hamid, Bandar Alhamdan, Phouvanh Phommalangsy, and Bob
Lingard. “Ethical Issues in Cross-Cultural Research.” International Journal of Research &
Method in Education 36, no. 4 (November 2013): 386–99.
https://doi.org/10.1080/1743727X.2012.705275.
Impact Initiative. “The Delta of Impact: Mapping a meta-impact pathway to uncover six
stepping stones of research impact.” Impact Initiative Working Paper,
https://www.theimpactinitiative.net/resources/delta-impact-mapping-meta-impact-pathway-
uncover-six-stepping-stones-research-impact, 2019.
Jacoby, Hanan G., and Ghazala Mansuri. Crossing Boundaries: Gender, Caste and Schooling in
Rural Pakistan. Policy Research Working Papers. The World Bank, 2011.
https://doi.org/10.1596/1813-9450-5710.
Jadue Roa, Daniela S. “Ethical Issues in Listening to Young Children in Visual Participatory
Research.” International Journal of Inclusive Education 21, no. 3 (March 4, 2017): 332–45.
https://doi.org/10.1080/13603116.2016.1260829.
Jeffrey, Craig, Roger Jeffery, and Patricia Jeffery. “School and Madrasah Education: Gender and
the Strategies of Muslim Young Men in Rural North India.” Compare: A Journal of
Comparative and International Education 38, no. 5 (October 2008): 581–93.
https://doi.org/10.1080/03057920802351358.
Johnson, Vicky, and Andy West. Children’s Participation in Global Contexts: Going beyond
Voice, 2018.
Kirchhoff, Allison, and Frances Lawrenz. “The Use of Grounded Theory to Investigate the Role
67
of Teacher Education on STEM Teachers’ Career Paths in High-Need Schools.” Journal of
Teacher Education 62, no. 3 (May 2011): 246–59. https://doi.org/10.1177/0022487110397840.
Kitzinger, Jenny, and Rosaline S. Barbour, eds. Developing Focus Group Research: Politics,
Theory, and Practice. London ; Thousand Oaks: SAGE Publications, 1999.
Krause, Brooke L., Aine Seitz McCarthy, and David Chapman. “Fuelling Financial Literacy:
Estimating the Impact of Youth Entrepreneurship Training in Tanzania.” Journal of
Development Effectiveness 8, no. 2 (April 2, 2016): 234–56.
https://doi.org/10.1080/19439342.2015.1092463.
Krueger, Richard A. Focus Groups: A Practical Guide for Applied Research. SAGE
Publications, 2014.
Kvale, Steinar. Doing Interviews. SAGE, 2008.
Lapadat, Judith C. “Problematizing Transcription: Purpose, Paradigm and Quality.”
International Journal of Social Research Methodology 3, no. 3 (January 2000): 203–19.
https://doi.org/10.1080/13645570050083698.
Le Fanu, Guy. “The Inclusion of Inclusive Education in International Development: Lessons
from Papua New Guinea.” International Journal of Educational Development 33, no. 2 (March
2013): 139–48. https://doi.org/10.1016/j.ijedudev.2012.03.006.
Le, Thuc Duc. Why Children in Vietnam Drop out of School and What They Do after That.
Working Paper 102. Oxford, UK: Young Lives, 2013.
Leavy, Patricia. Method Meets Art: Arts-Based Research Practice. Second edition. New York ;
London: The Guilford Press, 2015.
Lefebvre, Elisabeth, Acacia Nikoi, Richard Bamattre, Amina Jafaar, Emily Morris, David
Chapman, D. and Joan DeJaeghere. “Getting ahead and getting by in youth livelihoods and
employment.” Toronto, CA: The Mastercard Foundation, 2018.

Levinson, Bradley A., Douglas E. Foley, and Dorothy C. Holland, eds. The Cultural Production
of the Educated Person: Critical Ethnographies of Schooling and Local Practice. SUNY Series,
Power, Social Identity, and Education. Albany, NY: State University of New York Press, 1996.

Lincoln, Yvonna S., and Egon G. Guba. Naturalistic Inquiry. Beverly Hills, Calif: Sage
Publications, 1985.
Mack, Natasha. Family Health International, United States Agency for International
Development. Qualitative Research Methods: A Data Collector’s Field Guide, 2005.
http://www.fhi360.org/sites/default/files/media/documents/Qualitative%20Research%20Methods
%20-%20A%20Data%20Collector’s%20Field%20Guide.pdf.
Magee, Arran, and Tejendra Pherali. “Freirean Critical Consciousness in a Refugee Context: A
Case Study of Syrian Refugees in Jordan.” Compare: A Journal of Comparative and
International Education 49, no. 2 (March 4, 2019): 266–82.
https://doi.org/10.1080/03057925.2017.1403312.
68
Mahoney, James, and Gary Goertz. “A Tale of Two Cultures: Contrasting Quantitative and
Qualitative Research.” Political Analysis 14, no. 3 (2006): 227–49.
https://doi.org/10.1093/pan/mpj017.
Mark, Glenis, and Amohia Boulton. “Indigenising Photovoice: Putting Māori Cultural Values
Into a Research Method.” Forum Qualitative Sozialforschung / Forum: Qualitative Social
Research 18, no. 3 (September 28, 2017). https://doi.org/10.17169/fqs-18.3.2827.
Markiewicz, Anne. “Closing the Gap through Respect, Relevance, Reciprocity and
Responsibility: Issues in the Evaluation of Programs for Indigenous Communities in Australia.”
Evaluation Journal of Australasia 12, no. 1 (March 2012): 19–25.
https://doi.org/10.1177/1035719X1201200103.
Maxwell, Joseph Alex. A Realist Approach for Qualitative Research. Thousand Oaks: SAGE
Publications, 2012.
Maxwell, Joseph Alex. Qualitative Research Design: An Interactive Approach. 3rd ed. Applied
Social Research Methods, v. 41. Thousand Oaks, Calif: SAGE Publications, 2013.
McLeod, Julie, and Lyn Yates. Making Modern Lives: Subjectivity, Schooling, and Social
Change. SUNY Series Power, Social Identity, and Education. Albany, NY: State University of
New York Press, 2006.
Miles, Matthew B., A. Michael Huberman, and Johnny Saldaña. Qualitative Data Analysis: A
Methods Sourcebook. Third edition. Thousand Oaks, California: SAGE Publications, Inc, 2014.
Morojele, Pholoho and Nithi Muthukrishna. “The journey to school: Space, geography and
experiences of rural children.” Perspectives in Education 30(1) (2012): 90-100.
Morrow, Virginia. “Practical Ethics in Social Research with Children and Families in Young
Lives: A Longitudinal Study of Childhood Poverty in Ethiopia, Andhra Pradesh (India), Peru and
Vietnam.” Methodological Innovations Online 8, no. 2 (August 2013): 21–35.
https://doi.org/10.4256/mio.2013.011.
Morrow, Virginia. The Ethics of Social Research with Children and Families in Young Lives:
Practical Experiences. Working Paper, no. 53. Oxford: Young Lives, Dept. of International
Development, University of Oxford, 2009.
Morrow, Virginia, Janet Boddy, and Rowena Lamb. “The Ethics of Secondary Data Analysis:
Learning from the Experience of Sharing Qualitative Data from Young People and Their
Families in an International Study of Childhood Poverty.” Reports and working papers, April 11,
2014. http://sro.sussex.ac.uk/id/eprint/49123/.
Morrow, Virginia, and Gina Crivello. “What Is the Value of Qualitative Longitudinal Research
with Children and Young People for International Development?” International Journal of
Social Research Methodology 18, no. 3 (May 4, 2015): 267–80.
https://doi.org/10.1080/13645579.2015.1017903.
Mosse, David. Cultivating Development: An Ethnography of Aid Policy and Practice.
Anthropology, Culture, and Society. London ; Ann Arbor, MI: Pluto Press, 2005.

69
Narayan-Parker, Deepa, ed. Can Anyone Hear Us? Voices of the Poor. New York: Published by
Oxford University Press for the World Bank, 2000.
Naveed, Arif, Nozomi Sakata, Anthoula Kefallinou, Sara Young, and Kusha Anand.
“Understanding, Embracing and Reflecting upon the Messiness of Doctoral Fieldwork.”
Compare: A Journal of Comparative and International Education 47, no. 5 (September 3, 2017):
773–92. https://doi.org/10.1080/03057925.2017.1344031.
Norton, Andy, and Bella Bird, eds. A Rough Guide to PPAs: Participatory Poverty Assessment ;
an Introduction to Theory and Practice. London: Overseas Development Inst, 2001.
“What Does Teaching Look Like?: A New Video Study.” Teaching in Focus, January 17, 2018.
https://doi.org/10.1787/948427dc-en.
Palinkas, Lawrence A., Sarah M. Horwitz, Carla A. Green, Jennifer P. Wisdom, Naihua Duan,
and Kimberly Hoagwood. “Purposeful Sampling for Qualitative Data Collection and Analysis in
Mixed Method Implementation Research.” Administration and Policy in Mental Health and
Mental Health Services Research 42, no. 5 (September 2015): 533–44.
https://doi.org/10.1007/s10488-013-0528-y.
Patton, Michael Quinn. Qualitative Research and Evaluation Methods. 3 ed. Thousand Oaks,
Calif: Sage Publications, 2002.
Patton, Michael Quinn. Qualitative research. Chichester: John Wiley & Sons, Ltd, 2002.
Pawson, Ray. Evidence-Based Policy a Realist Perspective. London; Thousand Oaks, Calif.:
SAGE, 2006.
Pawson, Ray. The Science of Evaluation: A Realist Manifesto. London ; Thousand Oaks, Calif:
SAGE, 2013.
Pillay, Jace. “Ethical Considerations in Educational Research Involving Children: Implications
for Educational Researchers in South Africa.” South African Journal of Childhood Education 4,
no. 2 (December 24, 2014): 18. https://doi.org/10.4102/sajce.v4i2.211.
Ponterotto, Joseph. “Brief Note on the Origins, Evolution, and Meaning of the Qualitative
Research Concept Thick Description.” The Qualitative Report 11, no. 3 (September 1, 2006):
538–49.
Popova, Anna, David Evans, Mary E. Breeding, and Violeta Arancibia. “Teacher Professional
Development around the World : The Gap between Evidence and Practice.” The World Bank,
August 30, 2018. http://documents.worldbank.org/curated/en/349051535637296801/Teacher-
Professional-Development-around-the-World-The-Gap-between-Evidence-and-Practice.
Ragin, Charles C. The Comparative Method: Moving beyond Qualitative and Quantitative
Strategies. Berkeley: University of California Press, 1987.
Ragin, Charles C. The Comparative Method: Moving beyond Qualitative and Quantitative
Strategies. Oakland: University of California Press, 2014.
Ravitch, Sharon M., Matthew J. Tarditi, Nayibe Montenegro, Duilio Baltodano, and Eveling
Estrada. “Learning Together: Dialogue, Collaboration, and Reciprocal Transformation in a
Nicaraguan Educational Program.” In The Palgrave International Handbook of Action

70
Research (pp. 791–805), edited by Lonnie L. Rowell, Catherine D. Bruce, Joseph M. Shosh, and
Margaret M. Riel. New York: Palgrave MacMillan, 2017.
Reason, Peter, and Hilary Bradbury, eds. The Sage Handbook of Action Research: Participative
Inquiry and Practice. 2nd ed. London ; Thousand Oaks, Calif: SAGE Publications, 2008.
Robinson-Pant, Anna, and Nidhi Singal. Researching Ethically across Cultures: Issues of
Knowledge, Power and Voice, 2018.
https://nls.ldls.org.uk/welcome.html?ark:/81055/vdc_100056965047.0x000001.
Rodriguez Gomez, Diana. “When war enters the classroom: An Ethnographic Study of Social
Relationships Among School Community Members on the Colombian–Ecuadorian Border.”
In (Re)Constructing Memory: Education, Identity, and Conflict, edited by Michelle J. Bellino
and James, H. Williams, 267–289. Rotterdam: Sense Publishers, 2017.
Rubin, Herbert J., and Irene Rubin. Qualitative Interviewing: The Art of Hearing Data. 3rd ed.
Thousand Oaks, Calif: SAGE, 2012.
Rubinstein-Ávila, Eliane. “Accounting for – and Owning up to – the Messiness in Cross-
Cultural/Linguistic Qualitative Research: Toward Methodological Reflexivity in South
America’s Internet Cafés.” International Journal of Qualitative Studies in Education 26, no. 8
(September 2013): 1041–61. https://doi.org/10.1080/09518398.2012.736642.
Ryle, Gilbert. The Concept of Mind. Senior Series. London, New York: Hutchinson’s University
Library, 1949.
Saldaña, Johnny. Longitudinal Qualitative Research: Analyzing Change through Time. Walnut
Creek, Calif.: AltaMira Press, 2003.
http://public.ebookcentral.proquest.com/choice/publicfullrecord.aspx?p=1351175.
Saunders, Benjamin, Julius Sim, Tom Kingstone, Shula Baker, Jackie Waterfield, Bernadette
Bartlam, Heather Burroughs, and Clare Jinks. “Saturation in Qualitative Research: Exploring Its
Conceptualization and Operationalization.” Quality & Quantity 52, no. 4 (July 2018): 1893–
1907. https://doi.org/10.1007/s11135-017-0574-8.
Seidman, Irving. Interviewing as Qualitative Research: A Guide for Researchers in Education
and the Social Sciences. 4th ed. New York: Teachers College Press, 2013.
Seth, Sanjay. Subject Lessons: The Western Education of Colonial India. Politics, History, and
Culture. Durham: Duke University Press, 2007.
Setlhare, Rubina, Lesley Wood, and Lukas Meyer. “Exploring Group Life Design with Teachers
in the Context of Poverty Related Psychosocial Challenges.” South African Journal of Education
37, no. 4 (November 30, 2017): 1–8. https://doi.org/10.15700/saje.v37n4a1493.
Shamim, Fauzia, and Rashida Qureshi. “Informed Consent in Educational Research in the South:
Tensions and Accommodations.” Compare: A Journal of Comparative and International
Education 43, no. 4 (June 27, 2013): 464–82. https://doi.org/10.1080/03057925.2013.797729.
Sharma‐Brymer, Vinathe, and Christine Fox. “Being an Educated Woman in India: A
Phenomenological Approach to Narratives.” Compare: A Journal of Comparative and
71
International Education 38, no. 3 (June 2008): 321–33.
https://doi.org/10.1080/03057920802066626.
Sikes, Pat. “Working Together for Critical Research Ethics.” Compare: A Journal of
Comparative and International Education 43, no. 4 (June 27, 2013): 516–36.
https://doi.org/10.1080/03057925.2013.797765.
Singh, Renu, and Protap Mukherjee. “Effect of Preschool Education on Cognitive Achievement
and Subjective Wellbeing at Age 12: Evidence from India.” Compare: A Journal of Comparative
and International Education 49, no. 5 (September 3, 2019): 723–41.
https://doi.org/10.1080/03057925.2018.1453349.
Singhal, Arvind, and Elizabeth Rattine-Flaherty. “Pencils and Photos as Tools of Communicative
Research and Praxis: Analyzing Minga Perú’s Quest for Social Justice in the Amazon.”
International Communication Gazette 68, no. 4 (August 2006): 313–30.
https://doi.org/10.1177/1748048506065764.
Sinha, Pallawi. “Listening Ethically to Indigenous Children: Experiences from India.”
International Journal of Inclusive Education 21, no. 3 (March 4, 2017): 272–85.
https://doi.org/10.1080/13603116.2016.1260843.
Smith, Linda Tuhiwai. Decolonizing Methodologies: Research and Indigenous Peoples. London:
Zed Books Ltd, 2013.
Souto-Manning, Mariana. “Critical Narrative Analysis: The Interplay of Critical Discourse and
Narrative Analyses.” International Journal of Qualitative Studies in Education 27, no. 2
(February 7, 2014): 159–80. https://doi.org/10.1080/09518398.2012.737046.
Souto-Manning, Mariana. “Education for democracy: The text and context of Freirean culture
circles in Brazil.” In Reimagining civic education: How diverse societies form democratic
citizens, 121-146, edited by Bradley A. Levinson and Doyle Stevick. Lanham: Rowman &
Littlefield, 2007.
Sutton, Margaret, and Bradley A. Levinson. Policy as Practice: Toward a Comparative
Sociocultural Analysis of Educational Policy. Greenwood Publishing Group, 2001.
Teamey, Kelly and Rachel Hinton. “Reflections on Participation and its links with
Transformative Processes.” In Children and Young People’s Participation and Its
Transformative Potential: Learning from across Countries. Studies in Childhood and Youth,
edited by E.K.M. Tisdall, Andressa M. Gadda, and Udi Mandel Butler. Palgrave Macmillan UK,
2014.
Teddlie, Charles, and Abbas Tashakkori. Foundations of Mixed Methods Research: Integrating
Quantitative and Qualitative Approaches in the Social and Behavioral Sciences. Los Angeles:
SAGE, 2009.
Teddlie, Charles, and Fen Yu. “Mixed Methods Sampling: A Typology With Examples.”
Journal of Mixed Methods Research 1, no. 1 (January 2007): 77–100.
https://doi.org/10.1177/1558689806292430.
Temple, Bogusia, and Alys Young. “Qualitative Research and Translation Dilemmas.”
Qualitative Research 4, no. 2 (August 2004): 161–78.

72
https://doi.org/10.1177/1468794104044430.
Thaler, Richard H., and Cass R. Sunstein. Nudge: Improving Decisions about Health, Wealth
and Happiness. London: Penguin Books, 2009.
Tikly, Leon, and Tim Bond. “Towards a Postcolonial Research Ethics in Comparative and
International Education.” Compare: A Journal of Comparative and International Education 43,
no. 4 (June 27, 2013): 422–42. https://doi.org/10.1080/03057925.2013.797721.
Turk, Carrie, Patti Petesch, and Ana Maria Muñoz Boudet. “Defining Gender in the XXI
Century : Methodology Guide - Rapid Qualitative Assessment to Inform the World Development
Report on Gender (2010).” The World Bank, September 30, 2010.
http://documents.worldbank.org/curated/en/974381468339258350/Defining-gender-in-the-XXI-
century-methodology-guide-rapid-qualitative-assessment-to-inform-the-World-Development-
Report-on-Gender-2010.
Turner, Julianne C., Carol Midgley, Debra K. Meyer, Margaret Gheen, Eric M. Anderman,
Yongjin Kang, and Helen Patrick. “The Classroom Environment and Students’ Reports of
Avoidance Strategies in Mathematics: A Multimethod Study.” Journal of Educational
Psychology 94, no. 1 (March 2002): 88–106. https://doi.org/10.1037/0022-0663.94.1.88.
UNICEF. “Procedure for Ethical Standards in Research, Evaluation, Data Collection and
Analysis.” UNICEF, 2015.
Vagle, Mark D. “Locating and Exploring Teacher Perception in the Reflective Thinking
Process.” Teachers and Teaching 15, no. 5 (October 2009): 579–99.
https://doi.org/10.1080/13540600903139597.
van Manen, Max. Researching Lived Experience: Human Science for an Action Sensitive
Pedagogy. SUNY Series in the Philosophy of Education. Albany, N.Y.: State University of New
York Press, 1990.
Vance, Angela, Donna Pendergast, and Susanne Garvis. “Teaching Resilience: A Narrative
Inquiry into the Importance of Teacher Resilience.” Pastoral Care in Education 33, no. 4
(October 2, 2015): 195–204. https://doi.org/10.1080/02643944.2015.1074265.
Vavrus, Frances, and Maud Seghers. “Critical Discourse Analysis in Comparative Education: A
Discursive Study of ‘Partnership’ in Tanzania’s Poverty Reduction Policies.” Comparative
Education Review 54, no. 1 (February 2010): 77–103. https://doi.org/10.1086/647972.
Wang, Caroline, and Mary Ann Burris. “Photovoice: Concept, Methodology, and Use for
Participatory Needs Assessment.” Health Education & Behavior 24, no. 3 (June 1997): 369–87.
https://doi.org/10.1177/109019819702400309.
Wang, Dan. “The Dilemma of Time: Student-Centered Teaching in the Rural Classroom in
China.” Teaching and Teacher Education 27, no. 1 (January 2011): 157–64.
https://doi.org/10.1016/j.tate.2010.07.012.
Wickenden, Mary, and Gayatri Kembhavi-Tam. “Ask Us Too! Doing Participatory Research
with Disabled Children in the Global South.” Childhood 21, no. 3 (August 2014): 400–417.

73
https://doi.org/10.1177/0907568214525426.
Yin, Robert K. Case Study Research and Applications: Design and Methods. Thousand Oaks:
SAGE, 2017.
Young Lives. “Guide to Research, Section 11: Planning and Managing Fieldwork” (2017b).
http://www.younglives.org.uk/sites/www.younglives.org.uk/files/GuidetoYLResearch-S11-
Fieldwork.pdf
Young Lives. “Guide to Research, Section 12: Cohort Maintenance” (2017c).
http://www.younglives.org.uk/sites/www.younglives.org.uk/files/GuidetoYLResearch-S12-
Young Lives. “Guide to Research, Section 14: Methods of analysis” (2017a).
http://www.younglives.org.uk/sites/www.younglives.org.uk/files/GuidetoYLResearch-S14-
MethodsforAnalysis.pdf

74
 Appendix 1: Checklist to assess quality in qualitative research
Conceptual framing
____ The study has a clear overall approach that situates it within a research paradigm
(interpretive/constructionist; critical; deconstructionist).
____ The goals for the qualitative study are clearly articulated, and these goals align with the
research paradigm (e.g., if the goal is to understand participants’ perceptions of a program, it is
aligned with an interpretive/constructionist perspective and related methodologies).
____ The researchers have identified a question (or a set of questions) that can best be answered
through qualitative research, and the ethics of asking (or not asking) the questions are considered.
Robustness of methodology
____ The researchers have selected a methodology that aligns with the overall paradigm and
purpose and have provided justification as to why this methodology is well-suited to answer the
research questions.
____ If the qualitative study is being used as part of a mixed-methods study, its goals and approach
are considered and aligned with the other components of the mixed-methods study.
____ The researchers have considered how the methodology (and data) can be used to inform
policy and practice.
Culturally appropriate tools and analysis
____ The selection of methods to collect data are aligned with the overall purpose and
methodology, and cost, cultural appropriateness, and relevancy are discussed.
____ Protocols for interviewing, observations, or participatory and arts-based methods are
developed with consideration of the different respondent groups and the ethical guidelines of the
countries involved in the research project.
____ The project team should have contingency plans for alternative methods and participants if
they face challenges in the field in carrying out their planned data collection.
____ Time and budget are adequately considered for collecting data using the appropriate methods.
In addition, the ethics of paying (or not paying) participants should be considered.
Credibility
____ The process for sampling has been identified and aligns with the overall purpose and
methodology. Considerations of consent, confidentiality, and participation have been given to
including children/youth participants, illiterate participants, or others who may be considered
vulnerable populations.
____ There is a clear plan for piloting the protocols for data collection, including considerations
of translation, if necessary, and revising protocols if used over multiple waves of data collection.
____ The researchers/team have considered who will collect data using different methods,

75
including a plan to train local researchers and translators in the processes for doing qualitative
research.
Reliability
____ The project has a plan for data translation (if required), analysis, and storage, including a
clear statement about who will have access to data and who will analyze it. The project should
clearly state data ownership and uses and plans to share the data with participants.
Openness and transparency; cogency
____ A data analysis plan has been created that aligns with the overall methodology and purpose
of the research and the research team is adequately trained to analyze the data. Considerations for
ensuring the quality of the analysis, and time and cost are clear in the data analysis plan.
____ There is a clear plan for sharing the research findings with participants and other stakeholders
(national, international), and considerations are given to how the research can affect practice and
policy in a timely manner.

76
This page intentionally left blank.

77