2016 IEEE/ACM 38th IEEE International Conference on Software Engineering

Grounded Theory in Software Engineering Research:

A Critical Review and Guidelines
Klaas-Jan Stol Paul Ralph Brian Fitzgerald
Lero—the Irish Software Research Department of Computer Science Lero—the Irish Software Research
Centre, University of Limerick University of Auckland Centre, University of Limerick
Ireland New Zealand Ireland
[email protected] [email protected] [email protected]

ABSTRACT in computer science has been growing for the last decade (Fig. 1).
Grounded Theory (GT) has proved an extremely useful research Early examples of the use of GT in software engineering are by
approach in several fields including medical sociology, nursing, Carver [13] and Coleman and O’Connor [18].
education and management theory. However, GT is a complex Grounded theory is a method originally described by Glaser and
method based on an inductive paradigm that is fundamentally Strauss in their seminal book The Discovery of Grounded Theory
different from the traditional hypothetico-deductive research [38]. The goal of GT is to generate theory rather than test or
model. As there are at least three variants of GT, some ostensibly validate existing theory. GT is suitable for investigating questions
GT research suffers from method slurring, where researchers such as what’s going on here? [2].
adopt an arbitrary subset of GT practices that are not recognizable
As a relatively young discipline, SE has yet to establish and
as GT. In this paper, we describe the variants of GT and identify
validate abundant formal theories. Given the unique and novel
the core set of GT practices. We then analyze the use of grounded
aspects of the underlying technology in SE, theories from other
theory in software engineering. We carefully and systematically
disciplines may not be easy to borrow and adapt for SE. Inductive
selected 98 articles that mention GT, of which 52 explicitly claim
approaches such as GT are therefore useful to construct a relevant
to use GT, with the other 46 using GT techniques only. Only 16 conceptual and theoretical foundation for the field.
articles provide detailed accounts of their research procedures. We
offer guidelines to improve the quality of both conducting and Since its inception, GT has provided an extremely useful method-
reporting GT studies. The latter is an important extension since ological approach in numerous disciplines—notably medical
current GT guidelines in software engineering do not cover the re- sociology [15], nursing [4], education [58] and management [52].
porting process, despite good reporting being necessary for However, researchers have been criticized for using GT too
evaluating a study and informing subsequent research. casually, without clarifying that they have appreciated the
intricacies of grounded theory, which is not only quite complex
CCS Concepts but also based on an inductive paradigm that is entirely different
• General and reference → Empirical studies from the traditional hypothetico-deductive model [72]:

Keywords “‘Grounded theory’ is often used as rhetorical sleight of

Grounded theory, software engineering, review, guidelines hand by authors who are unfamiliar with qualitative
research and who wish to avoid close description or
1. INTRODUCTION illumination of their methods. More disturbing, perhaps,
“And make-it-up-as-you-go-along may be OK, but then you is that it becomes apparent, when one pushes them to
have to say, ‘I’m making-it-up-as-I-go-along, guys.’ ” describe their methods, that many authors hold some
serious misconceptions about grounded theory.”
In: “Dialogue: More on Muddling Methods” [57]
There is growing awareness that Software Engineering (SE) It is therefore crucial that researchers appropriately design and
research must consider social, cultural and human aspects of accurately report studies using inductive methods including
software construction [6, 26, 62]. Scholars have consequently grounded theory. However, some software engineering articles
adopted diverse research methods from the social sciences. that claim to use GT manifest considerable discrepancies between
Qualitative research methods are increasingly employed in SE their description of what they have actually done and seminal GT
research as evidenced by journal special issues on their use in
2007 [23] and 2011 [26]. One method that is attracting particular 200
attention is grounded theory (GT) [2, 47]. A quick search in the
Scopus database indicates the number of grounded theory studies 150

100
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed 50
for profit or commercial advantage and that copies bear this notice and the full 0
citation on the first page. Copyrights for components of this work owned by others
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015

than the author(s) must be honored. Abstracting with credit is permitted. To copy
otherwise, or republish, to post on servers or to redistribute to lists, requires prior
specific permission and/or a fee. Request permissions from [email protected].
ICSE '16, May 14 - 22, 2016, Austin, TX, USA Figure 1. Rise of grounded theory studies in computer science
Copyright is held by the owner/author(s). Publication rights licensed to ACM. Source: Scopus (Aug 2015); search string: TITLE-ABS-KEY
ACM 978-1-4503-3900-1/16/05...$15.00 (“grounded theory”), limited to “computer science”
DOI: http://dx.doi.org/10.1145/2884781.2884833

120
guidance. To assess the scale of this problem in the software engi- and analysis are simultaneous [16], and subsequent data
neering literature, we posed the following research question: collection is driven by theoretical sampling, discussed next.
Research Question: What is the state of practice of grounded • Theoretical sampling. The researcher identifies further data
theory research in software engineering? sources based on gaps in the emerging theory or to further
explore unsaturated concepts. Theoretical sampling is
Several SE researchers have recently reported their experiences indeterministic, as opposed to conventional sampling
using GT and these provide useful guidance for prospective GT techniques [16] (see theoretical saturation below).
researchers [1, 17, 44]. However, this guidance does not extend to • Theoretical sensitivity, which refers to the researcher’s
reporting GT studies. Reporting is important because this pro- ability to conceptualize, and to establish relationships
duces the persistent record that supports extension and contributes between concepts, lies at the heart of developing grounded
to the field’s cumulative body of knowledge. Furthermore, this theory. Both Glaser [35] (Ch. 5) and Strauss and Corbin [68]
paper presents a review of almost 100 articles through which we highlight the role of creativity in this process.
identify a number of key issues with GT studies in SE. • Coding. The researcher uses inductive and abductive logic to
We emphasize that our purpose is not to pedantically analyze and construct analytical codes and infer theoretical categories
criticize the papers included in our study, nor to criticize the au- from the data by labeling ‘incidents’ and their properties. The
thors of those studies in any way. Instead, we draw attention to researcher does not classify data into a preconceived coding
prevalent misunderstandings of grounded theory as an approach, scheme, or infer categories from logically deduced
and contend that only research that embodies GT’s core principles hypotheses [16]. Glaser and Strauss [38] did not use the term
(Sec. 2.1) should claim to be a grounded theory study. Based on abduction but emphasized induction to distance themselves
the results below, we explore numerous considerations for con- from the deductive theorizing that was prevalent at the time
ducting and reporting grounded theory and uncover challenges of their publication. Both Glaser and Strauss later admitted a
peculiar to software contexts. Our contribution is consequently role for deduction in GT [36, 70].
fourfold—we provide (1) an in-depth comparison of the three • Memoing. The researcher writes memos (e.g. notes, dia-
main variants of GT; (2) a critical analysis of the state of practice grams, sketches) to elaborate categories as they emerge,
of the use of grounded theory in the software engineering litera- describe preliminary properties and relationships between
ture; (3) a set of considerations for conducting and reporting GT categories, and identify gaps [16]. These memos play such an
studies in SE; (4) three significant challenges for applying GT to important role in theory generation that Glaser baldly stated
software engineering phenomena. that, “if the researcher skips this stage, he is not doing
This paper proceeds as follows. Section 2 presents a brief history grounded theory” [34] (Ch. 5, emphasis original).
of grounded theory (including its terminology and philosophical • Constant comparison. From the start of the study, the re-
foundations) and a comparison of the different GT versions. searcher constantly compares data, memos, codes and cate-
Section 3 presents the research design that we employed. Section gories [8]. Both categories and data interpretations evolve
4 presents the analysis and results of our study. Section 5 dis- and saturate until they ‘fit’ the data [34].
cusses the results and offers a checklist for future GT studies. • Memo sorting, also called theoretical sorting is the
Section 6 concludes the paper. continuous process of oscillating between the memos and the
emerging theory outline to find a suitable fit for all categories
2. GROUNDED THEORY that resulted from the coding [34, 70]. Like memoing, Glaser
argues that sorting cannot be skipped [35].
2.1 Key Components of Grounded Theory • Cohesive theory. The researcher attempts to move beyond
Grounded Theory refers to a method of inductively generating
superficial categories and develop a cohesive theory of the
theory from data [38]. GT studies often focus on unstructured text
studied phenomenon.
(e.g. interview transcripts, documents, field notes); however, they
• Theoretical saturation. The researcher stops collecting and
may also include structured text, diagrams and images, and even
analyzing data when theoretical saturation is reached. Theo-
quantitative data [35].
retical saturation refers to the point at which a theory’s com-
For the presentation of our analysis in Sec. 4, it is imperative to ponents are well supported and new data is no longer trig-
establish the key components of grounded theory. While GT has gering revisions or reinterpretations of the theory [34].
several variants (discussed in Sec. 2.3) they share many core fea-
While this list of core features is by no means a complete descrip-
tures, including the following:
tion of grounded theory (both Glaser and Strauss have written
• Limit exposure to literature. Rather than beginning with a numerous books to explain GT [34-36, 71]), it does highlight
comprehensive literature review, grounded theory proponents some distinctions from more traditional, deductive research meth-
(e.g. [19, 35]) recommend limiting exposure to existing ods. The above, however, largely ignores the differences between
literature and theories to promote open-mindedness and pre- the various versions of GT, discussed in Section 2.3.
empt confirmation bias (see Sec. 2.3 for different positions
regarding the literature). A major reason to limit study of the 2.2 Philosophical Foundations
literature is to prevent the researcher from testing existing Research approaches are commonly (and simplistically) classified
theories, or thinking in terms of established concepts. into two broad groups based on the epistemological positions of
• Treat everything as data. When Glaser says, “all is data,” positivism and interpretivism [40]. GT can be confusing because
he means all—qualitative data, quantitative data, semi-struc- it does not fit cleanly into either group. We briefly discuss
tured data, pictures, diagrams, videos and even existing theo- ontology and epistemology, and then focus on how GT resists the
ries and literature [36, 69]. classification of positivism and interpretivism.
• Immediate and continuous data analysis. The researcher The positivist approach has long been applied in the physical
begins analyzing data immediately and does not finish sciences, and has led to tremendous growth of knowledge in the
collecting data before beginning analysis—data collection area. It comprises five pillars [43]:

121
• Unity of the scientific method: the same approach to 2.3 Versions of Grounded Theory
knowledge acquisition applies to all forms of enquiry. Although Glaser and Strauss never explicated their epistemologi-
• Search for causal relationships: science aims to find cal position in Discovery (and Glaser later argued that GT is
regularity and causal relationships among studied elements. paradigm-agnostic), their terminology reflects an objectivist
• Belief in empiricism: sense-experience is the only source of stance. The title of their seminal book is ‘The Discovery of
knowledge but subjective perception is not acceptable. Grounded Theory’ [38] rather than for example Sensemaking with
• Science (and its process) is value-free: science has no grounded theory—the term discovery suggests that an objective
intrinsic values or perspectives; science is independent of reality exists out there waiting to be discovered. Glaser speaks of
politics, ideology, morality, society and culture. an indicator-concept model, analysis of a core variable, and aims
• Science is founded upon logic and mathematics: causal for parsimony in the developed theory, reflecting a position that
relationships are demonstrated quantitatively, using the aligns with objectivism. Both Glaser and Strauss and Corbin also
universal language of math and the formal basis of logic. used objectivist terminology in their definitions of theory as a set
Positivism assumes that: (1) the universe behaves according to of concepts and relationships among them that together offer
inalterable, discoverable laws; (2) systems are merely the sum of explanations and predictions (i.e. causality) [34, 68]. While Glaser
their components (reductionism); (3) science should be maintains that GT is independent from any philosophical stance,
reproducible, reliable, rigorous and objective. Different scientists Corbin has gradually shifted towards interpretivism [19].
observing the same phenomenon should therefore reach Meanwhile, Charmaz (a student of Glaser), developed
equivalent conclusions. ‘constructivist’ grounded theory by reinterpreting GT from a con-
structivist’s stance [16] that is closely connected to interpretivism.
Interpretivism makes the opposite assumptions (cf. [41]): (1) no
universal truth or reality exists, rather, “the important reality is Due to extensive discussions on what constitutes grounded theory,
what people imagine it to be” [9]; (2) systems exhibit emergent it has been labeled a ‘contested concept’ [11]. Since Glaser and
behaviors not reducible to their component parts [33, 48]; (3) Strauss’s seminal book, GT has seen considerable evolution
social science, which aims to understand and to interpret human resulting in the emergence of different versions. Denzin lists no
behavior, is fundamentally different from natural science, and less than seven different versions [22], although he does not
natural science methods including quantitative measurement, specify the differences between all of them. It is now widely
statistical significance and hypothesis testing are insufficient for acknowledged that there are at least three main streams of GT [1]:
understanding social phenomena [76]. Therefore, formulating Glaser’s GT (classic or Glaserian GT); Strauss and Corbin’s GT
hypotheses is not relevant to an interpretivist study. (Straussian GT); and Charmaz’s constructivist GT.
Understanding and explaining the social world requires emotion Glaser’s perspective is well reflected in the fact that he refers to
and empathy, which preclude pure objectivity [76]. Interpretivists his version of grounded theory as “classic” GT. He strongly disa-
have attacked positivism for promoting the myth of objectivity grees with Strauss and Corbin’s version of GT [35] and has ar-
[56] and Berger and Kellner point out that “direct access to facts gued that Strauss and Corbin’s method is not grounded theory, but
and laws ... is never possible, no matter what one’s standpoint ... refers to it as “full conceptual description” [35]. Furthermore,
there is no magic trick by which one can bypass the act of inter- Glaser has called ‘constructivist’ grounded theory a “misnomer”
pretation” [5]. Interpretivists prefer qualitative methods, [37]. In this paper, we accept any version of grounded theory as
including interviews, case studies, ethnography and action ‘grounded theory’—although we will argue below that
research, arguing that these keep the researcher grounded in “the consistency with a particular version is important. Table 1
first-order, primary, lived concepts of everyday life” [21]. summarizes some of the key differences between the three main
While positivism and interpretivism can be cast as polar opposites strands of GT. An additional difficulty in comparing GT versions
[31], many studies do not sit neatly in either paradigm. We have is that Straussian GT is still evolving, as briefly mentioned above.
experiments where the dependent variable is ‘measured’ by com- Of the three main versions of GT, the difference between classic
bining the subjective ratings of expert judges [55], case studies and Straussian GT has been discussed most extensively [10, 42,
with upfront hypotheses [63], interview studies where text is ana- 51]. Classic GT can be characterized as having a strong focus on
lyzed quantitatively [61] and mixed-method inquiries that com- emergence (of research questions, of codes, of theory), whereas
bine questionnaires with case studies [60]. “All qualitative data Straussian GT meticulously suggests a set of ‘mini-steps.’ This
can be coded quantitatively” by counting words and categorizing difference in focus on emergence is captured succinctly by Stern:
statements; meanwhile “all quantitative data is based on “Strauss, as he examines the data, stops at each word to ask,
qualitative judgment” because we have to make assumptions to ‘What if?’ Glaser keeps his attention focused on the data and
interpret the numbers [73]. More fundamentally, these groups asks, ‘what do we have here?’” [64] (our emphasis). Glaser
involve several interconnected philosophical positions that cannot requires any concept to be grounded in the data, whereas Strauss
be reduced to a single spectrum, let alone a Boolean variable. and Corbin go beyond the data by asking various questions on
It is easy to mistake GT as a qualitative or interpretivist method what might be to develop the emerging theory [35] (Ch. 8).
because many GT studies focus on collecting and analyzing Strauss’s approach has been described as “more free-wheeling
unstructured text. However, GT was developed in the 1960s, flights of imagination,” which contrasts strongly with Glaser’s
during the ontological and epistemological shift from positivism faithfulness to the data.
and objectivism to social constructionism and postmodernism. GT There is little agreement on what constitutes theory. In classic GT,
stems from a dissatisfaction with the way research was done, theory consists of concepts that are related to one another,
whereby new researchers were trained as “theoretical serfs” who offering explanation and prediction. Constructivist GT
tested the theories of “theoretical capitalists” [34] (p. 9), which emphasizes understanding and acknowledges that data,
could lack relevance to the real world. GT was developed due to a interpretations, and resulting theory depend on the researcher’s
desire to build theories more rigorously and dispassionately by view. In practice, however, such ontological and epistemological
grounding them in objective reality. differences are rarely apparent in generated theories.

122
 Table 1. Some of the key differences between the three main strands of grounded theory
Element Classic / Glaserian grounded theory Straussian grounded theory Constructivist grounded theory
Research Should not be defined a priori, but Research question may be defined Research begins with “initial
question emerge from the research—this makes upfront, derived from the literature or research questions,” which evolve
the RQ relevant to the field. The re- suggested by a colleague; RQ is often throughout the study [16].
searcher starts with an ‘area of interest.’ broad and open-ended.
Literature in other areas may be
consulted to increase the researcher’s
“theoretical sensitivity.” Defining a RQ
a priori is considered ‘forcing’ [35].
Role of the An extensive literature review should be The literature may be consulted Acknowledges not only Glaser’s
literature delayed until after the theory is throughout the process, as concepts reasons for delaying the literature
emerging to prevent the influence of from the literature may be used if review but also the impracticality of
existing concepts on the emerging applicable; to enhance theoretical this strategy. Charmaz highlights
theory. Until the researcher has defined sensitivity, as a secondary data the need to tailor a literature review
the RQ, it is not clear which literature source; to formulate questions for data to fit the purpose of the GT study
should be consulted. Existing concepts collection or stimulate questions [16] (p. 306).
such as gender and age should not be during analysis; to suggest areas for
included a priori, but must ‘earn’ their theoretical sampling [70] (p. 49).
way into the emerging theory.
Coding Open coding: ‘fracturing’ of the data; Open coding: generation of Initial coding: examining data
procedures line by line coding is recommended to ‘categories’ and how they vary word-by-word, line-by-line or
achieve full theoretical coverage, but dimensionally. Coding can be done incident-by-incident to make sense
does not reject coding sentences or line by line or by sentence or of the text without injecting the
paragraphs, or whole documents [35]. paragraph, or even the whole researcher’s assumptions, biases,
Selective coding: delimiting coding to document [70]. motivations. Similar to Glaser’s
only those variables that relate to one Axial coding: putting back data in open coding. Charmaz recommends
(or in some cases, several) core new ways after open coding by “coding with gerunds.”
variables to establish a parsimonious identifying relationships between Focused coding: selecting
theory. The core variable guides further categories; this is effectively Glaser’s categories from the most frequent
data collection. theoretical coding. Use of the or important codes, and using them
Theoretical coding: establishing ‘paradigm model’ or ‘conditional to categorize the data; does not
conceptual relations between matrix’ (an analytical tool in require a single core category or
substantive codes, resulting in the Straussian GT [70], Ch. 12) to variable.
development of hypotheses. Glaser identify context, conditions, action / Theoretical coding: specifying the
proposes several ‘coding families,’ interaction strategies and relationship between categories to
which are theoretical codes that can be consequences. integrate them into a cohesive
used by researchers, though these must Selective coding: deciding on the theory.
‘earn’ their way into the emerging central category that all major
theory (e.g. the Six C family in Fig. 4). categories can link to [70].
Questions • What is this data a study of? [34] Asking questions about whom, when, • What is this data a study of? [16]
asked • What category or what property of where, how, with what consequences, • What do the data suggest? Pro-
during what category does this incident in- and under what conditions phenomena nounce? Leave unsaid?
analysis dicate? occur, helps to ‘discover’ important • From whose point of view?
• What is actually happening in the ideas for the theory [69]. ‘Free- • What theoretical category does
data? wheeling flights of imagination’ [16] this specific datum indicate? [16]
Philo- Objectivism: There exists a single, Pragmatism and symbolic Social constructionism: social
sophical correct description of reality; the interactionism: actors engage in a reality is constructed by our
influences researcher therefore discovers grounded world that requires reflexive individual and collective action. GT
theory from data [11]. interaction; reality is constructed emerges from “shared experiences
through interaction and relies on and relationships with participants”;
language and communication [14]. Observers are not neutral [16].
Evaluation The generated categories must fit the Seven criteria for the research process Credibility (e.g. is there sufficient
criteria data, the theory should work (it must be e.g. information on sample selection, data to merit claims?), originality
able to explain or predict what will major categories, derived hypotheses (do your categories offer new
happen); the theory must have and discrepancies. Eight criteria insights?), resonance (does the GT
relevance to the action of the area, and regarding the empirical grounding, make sense to participants),
the theory must be modifiable as new e.g. “are concepts generated?” “is usefulness (does the GT offer use-
data appear [34] (p. 4-5). variation built into the theory?” [70]. ful interpretations?) [16] (p. 337).

123
While the 1998 edition of Strauss and Corbin’s book specifies Table 3. Selected journals and number of papers included
open, axial and selective coding, the 2008 edition (authored by
Journal Articles
Corbin alone after Strauss’ death in 1996, making the term
Straussian GT a misnomer and Corbinian more appropriate) no Information and Software Technology 42
longer defines open and axial coding as separate activities [74]. Journal of Systems and Software 16
This paper focuses on the 1998 version since it is very prevalent IEEE Transactions on Software Engineering 11
(in particular axial coding). As Table 1 shows, the three variants Empirical Software Engineering 10
differ in their position with respect to key elements such as the Software Process: Improvement and Practice a 8
role of the literature, but also in terminology and order of Journal of Software: Evolution and Process b 4
practices (e.g. coding procedures). For example, Strauss and Software Quality Journal 3
Corbin interpret selective coding differently from Glaser. ACM Trans. Software Engineering and Methodology 3
Furthermore, Strauss increasingly saw GT as a verificational
Journal of Software Maintenance and Evolution: 1
method [16], a position that Glaser strongly rejects [35].
Research and Practice c
3. RESEARCH DESIGN a
b
Merged with J Software: Evolution and Process in 2012
Successor of J Softw Maint Evol Research & Practice since vol. 24, 2012
3.1 Study Identification and Selection c
Vol. 1-12 published as J Software Maintenance: Research and Practice
To investigate the state of practice of GT research in SE, we
reviewed a selection of articles reporting GT studies. We adopted We removed editorials, secondary studies (systematic reviews),
an automated search strategy; that is, we collected our sample by and articles that present methodological reflections on the use of
searching specific online databases using specific search strings GT, rather than a specific GT study (e.g. [1, 12, 18, 46, 59]),
(see below). We chose this over manually browsing selected resulting in a final set of 98 papers (available in an appendix
publication outlets because it is more efficient and replicable. We [67]). Fig. 2 shows the articles’ distribution of publication year.
pilot tested several search strings. For example, we conducted a
search on “grounded theory,” but this resulted in thousands of 3.2 Data Extraction
papers from other disciplines. We also tried limiting the search to We read all 98 papers to investigate the following questions.
the title, abstract and keywords, but some GT studies appear not • What is claimed concerning the use of grounded theory? (e.g.
to use the term ‘grounded theory’ in any of these fields. Based on “we used grounded theory,” “we took a grounded theory
this pilot test, we adopted the following query. approach,” “the data were coding using GT techniques”);
Search String: “grounded theory” AND “software engineering” • To what extent are different versions of grounded theory
We searched Scopus, IEEE Xplore, the ACM Digital Library and discussed and used? To what extent do papers state their
ScienceDirect. We excluded Wiley Online and SpringerLink, as epistemological stance?
these are subsumed by Scopus. We adapted the search string to • Is grounded theory mentioned in the title, keywords, abstract,
the specific characteristics of each database. Further constraints or research question (or objective / topic / purpose)?
were introduced case-by-case to eliminate obviously irrelevant • What specific GT techniques and practices are used? (e.g.
papers. Combining the search results and removing duplicates open coding, constant comparison, memoing);
produced an initial dataset of 1,763 papers (Table 2). As this • How is data collected and analyzed?
dataset is too large for manual analysis, we focused on articles • What do GT studies produce and how do they present it?
published in well-known, peer-reviewed SE journals (Table 3). (e.g. as a diagram);
• Was the literature review (if any) conducted before, during or
We did not consider conference contributions because journal
after the study; was the resulting theory (if any) integrated
papers tend to have endured greater review, be more polished and
back into the literature?
have more liberal page limits. We also did not consider articles
from peer-reviewed magazines including Communications of the All information was recorded in a spreadsheet. We also took
ACM and IEEE Software because they tend to have briefer extensive notes concerning interesting findings that did not fit in
methodological descriptions, given their practitioner-oriented our predefined questions. In several studies, for example, we
focus. In the interests of representativeness, we further excluded noted clear deviations from GT principles, such as the use of
specialist journals such as Requirements Engineering and the (preconceived) ‘seed categories’ to guide initial analysis which is
International Journal of Open Source Software and Processes. viewed as inappropriate in GT. The data extraction and coding
was done by the primary author, which was reviewed by the
The selected journals coincide with those used in previous
remaining authors.
reviews (e.g. [39], except magazines as stated). We further added
the Software Quality Journal and the journals that descended from 25"
the Journal on Software Maintenance: Research and Practice.
20"
Table 2. Searched databases and search constraints
15"
Database Search constraints No.
10"
Scopus N/A (full text) 1,668
5"
ScienceDirect Computer Science only (full text) 249
IEEE Xplore Search on metadata only 73 0"
ACM DL Title, Abstract, Keywords only 13
95
98
00
01
02
03

06
07
08
09
10
11
12
13

15
04

14
20
19
19
20
20
20
20

20
20
20
20
20

20
20

20
20

20

Subtotal 2,003
Duplicates 240 Figure 2. Distribution of publication year of selected articles
Total 1,763 Note: Search conducted in Spring 2015, hence the drop in 2015.

124
4. ANALYSIS AND RESULTS However, deciding whether or not a study uses grounded theory is
In this section we address the use of GT, the level of detail far from trivial. While some articles clearly claim to use grounded
presented, variants of GT and the type of output of studies. theory, the phrasing of these claims varies substantially and some
are ambiguous. For example, some studies use a “grounded theory
4.1 Grounded Theory “Use” is Ambiguous approach.” In the absence of further clarification, we assume this
We analyzed all 98 articles to investigate their claim of using means GT was used, however, it could be interpreted as an
grounded theory, and found that many claims are quite ambigu- approach based on GT. This made it more difficult to decide
ous. Fig. 3 (Box 1) shows that almost half (n=46) of the surveyed whether or not the authors were actually claiming to use grounded
articles (n=98) merely borrow from grounded theory; for example: theory. This is simultaneously a potential threat to the validity of
our findings and a surprising finding itself. While our exact count
• “Using concepts of grounded theory […]”
(52 studies making claims to use GT) should be interpreted with
• “data analysis was carried out using a modified version
caution, the fact that this is ambiguous, and any large proportion
of Grounded Theory”
of studies borrowing from a method rather than using it, is
Fifteen articles (Box 1.1 in Fig. 3) state that they use an approach unusual and potentially problematic for a sound evaluation of
that resembles, adapts, or is inspired by grounded theory, but do such a study.
not in actual fact present a grounded theory study. An example of Of the 52 studies making a claim to have used GT, four studies
such a claim is: “In a method similar to the first step in grounded
(Box 2.1) deviated so sharply from GT that they have not used
theory (Glaser and Strauss 1967) […] we identified a set of cate-
grounded theory at all. In three cases, the authors developed a set
gories.” Such studies are clearly not grounded theory studies.
of preliminary categories, which were then combined with a
Eighteen articles (Box 1.2 in Fig. 3) do not use the term ‘grounded “grounded theory approach”—starting with a classification from
theory’ in the main text at all (but only in its bibliography). the literature is highly suspect, even when considering Strauss and
Rather, they mention specific techniques such as ‘coding’ or Corbin’s quite liberal use of the literature (see Table 1).
‘theoretical saturation’ and cite seminal works on grounded
Of the 98 articles included in our review, six used the term
theory, such as Glaser and Strauss’s Discovery book [38].
‘grounded theory’ in the title and 14 specified ‘grounded theory’
Thirteen other articles (Box 1.3) state that they use grounded
as a keyword. This suggests that grounded theory was essential to
theory ‘techniques’ or ‘procedures,’ and in most cases refer to these studies rather than an afterthought. While clearly no
coding and constant comparison. One example of such a statement conclusion should be drawn based on the presence of GT as a
is: “The ‘Open Coding’ and ‘Theoretical Coding’ techniques of
keyword, given the limitations of some journals on the number of
Glaser (1978) have been applied iteratively to identify different
keywords (as low as three), it might suggest that these authors
categories and their properties.” Such statements do not claim more consciously wished to signal the role of GT in their study.
that GT was used, merely GT techniques. In several cases, authors
explicitly acknowledge that their study is not a GT study. 4.2 Many Studies Present Little to No Detail
This borrowing rhetoric is unusual in research methodology. We Of the 52 articles claiming to have done a grounded theory study,
do not recall ever reading about studies that “use randomized 18 (Box 2.1 in Fig. 3) present no details at all beyond claims such
controlled trial techniques,” were “inspired by survey methodol- as: “[we] used a grounded theory approach for data gathering
ogy,” or “adopted a modified questionnaire approach.” Claiming and data analysis.” In some cases, a brief and usually incomplete
to “use grounded theory techniques” rather than GT wholly sug- summary of grounded theory is provided, for example, by stating
gests that authors are aware that GT is a comprehensive research that grounded theory consists of three coding phases. Besides
method from which they are borrowing certain elements. being incomplete, it also suggests coding happens in three distinct
phases, which is not what Glaser or Strauss had in mind. Many of
The remaining 52 papers (Box 2 in Fig. 3) explicitly claim to use these articles state that the conceptualization presented in those
GT. Typical examples of such claims include (e.g. [30]): articles were developed using grounded theory, without shedding
• “Using a grounded theory approach […]” any light on the process through which this was done.
• “We used grounded theory to […]” We further inspected the 30 articles (Box 2.3) that present signifi-
• “We generated a grounded theory” cant methodological details, to investigate the extent to which
different GT practices are mentioned and used (Table 4). While
[1.1] Adapted, inspired, resembles GT GT is not reducible to a set of independent practices, one still
N=15 expects GT studies to report details on key practices associated
[1.2] “GT” not mentioned, only specific with GT (cf. Sec. 2.1).
[1] Using GT techniques
techniques N=18
However, many authors use GT techniques à la carte. Fewer than
N=46 half of the 30 articles describe or confirm the use of key practices,
[1.3] Claiming GT techniques such as simultaneous data collection and analysis (n=13), mem-
N=13
Review oing (n=12), memo sorting (n=4), constant comparison (n=13), or
N=98 theoretical sampling (n=12). Fifteen articles confirm that data
[2.1] No details at all
N=18 [2.3.1] Comprehensive collection continued until theoretical saturation was reached. All
[2] Explicitly and detailed but one article discuss data sources, elucidate data collection and
claiming GT [2.2] Deviating from GT N=5
N=52 N=4 describe coding practices. Details varied from a brief paragraph to
[2.3] Detailed [2.3.2] Comprehensive an extensive presentation. We also found misinterpretations of
N=11
N=30 key practices. One article claimed theoretical sampling, but
[2.3.3] Coding details instead of collecting additional data to further investigate as of yet
only
N=14
unsaturated concepts in the emerging theory, a number of case
companies were selected seemingly a priori based on their
Figure 3. Breakdown of the articles included in our review experience in the area that the researchers were investigating.

125
 Table 4. Grounded theory practices used GT (n=52) Three articles do not refer to any of the seminal texts on GT but
refer to other sources. These may be innocent mistakes or
Practice Papers reporting
benevolent simplifications. Alternatively, and more worryingly,
GT Practice details reported 30 they may indicate researchers who are presenting their research
Simultaneous data collection and analysis 13 under the guise of techniques they have heard of, but not
Data sources and collection 29 investigated.
Theoretical sampling 12 Thirteen papers, however, do acknowledge the distinction
Coding 29 between classic and Straussian GT—some in more detail than
Memoing 12 others. For example, one article stated that it incorporated “a
Memo sorting 4 Strauss and Corbin grounded theory approach to data gathering
Constant comparison 13 and analysis,” whereas other articles laid out the differences
Theoretical saturation 15 between the variants in detail. Of these, five claim to use classic
GT, the other eight Straussian GT. None of the articles in our
Sixteen articles (Boxes 2.3.1, 2.3.2) provide a comprehensive sample explicitly claim to use Charmaz’s constructivist GT.
presentation of their research method, of which five articles Finally, only five articles state an epistemological position; in all
present extensive documentation about the GT research process cases the authors claim their study to be an interpretivist one. In
[2, 18, 45, 47, 49]. Fourteen other articles provided details on the four of those cases, reference was made to seminal works by
coding process only (Box 2.3.3). Glaser, and Strauss and Corbin, which align more closely with
positivism, as outlined in Sec. 2.
4.3 Many Studies Ignore GT Variants
As discussed in Sec. 2.3, GT has several variants with significant 4.4 Few “GT” Studies Generate Theory
differences with regards to the use of the literature, specific Since grounded theory is a method of generating theory, we
coding practices, and reflections on the role of the researcher in investigated the extent to which the 52 studies claiming to have
the research process. Of the 52 articles that claim to use grounded used GT developed theories. While it depends on one’s definition
theory, 39 did not acknowledge the existence of different variants. of theory, few of the studies appear (or claim) to develop a theory,
even though “a lack of existing theories” in a particular area is
To investigate which sources authors might have consulted in often given as a motivation to conduct a GT study.
their study design, we looked at the citations to seminal GT
works. Of the 39 articles that do not claim a specific GT variant, Eight articles presented contributions that were clearly cohesive
10 cited works on classic GT (Glaser, Glaser & Strauss), 13 cited theories consisting of constructs and relationships, while a ninth
works on Straussian GT (Strauss, Strauss & Corbin), and none article presented a set of hypotheses that could be considered a
cited constructivist GT. Thirteen articles cite conflicting seminal theory. Some of the topics theorized by these studies include:
works on GT (e.g. [16, 19, 35]) without acknowledging any • How is the software development process managed?
differences or indicating whose guidance they are following. Two • How do software processes form and evolve?
articles cite works on all three variants of GT. One interpretation • How do self-organizing agile teams self-organize?
of this is that authors are now aware of the differences, and, in
seeking to confer legitimacy on their research, provide copious Some articles present ‘theory’ in alternative forms instead of a set
references to several seminal works. However, we would argue of concepts and relationships. For example, Hoda et al. [47]
that, had the authors actually read all three works, the existence of presented six roles that members of agile teams assume. Together
different variants would have been likely acknowledged. these roles provide an explanation for the “social” process of self-
organization in agile teams, and as such they go beyond a mere
In several cases we found inconsistent usage of the claimed taxonomy of roles. Therefore, we argue such a coherent set of
variant of grounded theory. Two articles claim or cite classic GT findings can be considered a theory.
but use axial coding, a Straussian practice (Sec. 2.3). Another
article claims to use Straussian GT, but uses one of the coding In most cases, articles presented a graphical representation of the
families offered by Glaser for increasing theoretical sensitivity theory, usually simple boxes-and-arrows diagrams, to illustrate
[34] (p. 74). theoretical concepts and relationships. Three articles use Glaser’s
‘Six C’ coding family [34] (p. 74) for visualization (e.g. Fig. 4).
Table 5. Grounded theory variants acknowledged (n=52) Other articles synthesized their results into various other types of
contributions, including:
Grounded theory variant claimed Articles
• Conceptual frameworks, such as a framework of factors that
Acknowledgment of different GT variants 13 influence Software Process Improvement initiatives [29];
Explicit claim classic GT 5 • Conceptual models, such as a model of the process for
Explicit claim Straussian GT 8 managing collaborations in open source [7];
Explicit claim constructivist GT 0 • A set of factors, such as success factors for globally-
Variants not acknowledged 39 distributed XP projects [53];
Citing classic (Glaser / Glaser & Strauss) 10 • A set of themes or categories, such as a set of categories
Citing Straussian (Strauss / Strauss & Corbin) 13 representing the characteristics of product managers [54].
Citing constructivist (Charmaz) 0 Such contributions are useful as they offer new foundations for
Citing a combination of the above 13 empirical studies, but often they do not form a ‘theory’ that, in
Citing others 3 Glaser’s words, “account for a pattern of behavior.” We observe
Epistemology acknowledged 5 that studies that produce a ‘set of themes’ rather than a theory tend
Interpretivist or constructivist 5 only to borrow discrete practices from GT—what we call
Other 0 grounded theory à la carte.

126
Due to space reasons we cannot describe all the underlying key
points, codes, and concepts from our interviews and observation Most participants did not receive the level of customer involve-
that further ground the discussion. ment that Agile methods demand (P1–P12, P14–P19, P21–P23,
P25, P26, P28–P30).
4.1. Context ‘‘Sometimes [customers] only want to come back and see in 6
months what happened [in development].’’ — P16, Developer,
We interviewed 30 Agile practitioners from 16 different soft- India
ware development organizations over 3 years, half of whom where
from New Zealand and half from India. Fig. 3 shows the partici- ‘‘To get client involved in the process I think is the most difficult
pants and project details. In order to respect their confidentiality, part of Agile.’’ — P4, Developer, NZ

too familiar with the relevant literature, to excuse skipping

necessary background work [72].
3. For simplicity. It is easier to state “we used grounded
theory” than to thoroughly explain how a researcher
converted a large amount of unstructured text into a cohesive
theory. However, given that grounded theory is not widely
understood (misunderstood, even) or known amongst SE
researchers, we argue that such claim does not suffice.
4. Because they simply do not understand GT or its
relationship to other research methods. Suddaby notes that
“researchers claim to have performed grounded theory
research, support their claims with a cursory citation to Gla-
ser and Strauss (1967),” while offering little description of
the applied method. When authors are invited to elaborate,
Suddaby continues, “to reveal how the data were collected
and analyzed, it becomes clear that the term ‘grounded
theory’ was interpreted to mean ‘anything goes’ ” [72].
5. Per referee’s suggestion. We know of cases where referees
have suggested to authors that the method they used “looks
Fig. 2. The theory of Lack of Customer Involvement depicted using the Six C’s model (Context, Condition, Causes, Consequences, Contingencies, and Covariance).
Figure 4. Example of the Six C coding family (from: [45])
like grounded theory.” Such authors may post-hoc present
their research as grounded theory where such a claim is not
Finally, ten articles present mere description. In many cases, the
valid, simply to satisfy reviewers.
study’s results are structured based on a set of research questions,
which are answered in detail using quotes from participants. This Method slurring undermines grounded theory. Authors in the
type of output is quite common for those studies that only used management literature have observed an “overly generic use of
coding techniques, but do not make a theoretical contribution. the term ‘grounded theory’ ” [72]. Researchers in Information
Systems (which has considerable overlap with software
5. DISCUSSION engineering) have lamented that “the term ‘grounded theory’
A significant number of articles in our sample did not provide itself has almost become a blanket term for a way of coding data”
sufficient details for reviewers or readers to evaluate their [75]. Others have referred to the “erosion of GT as a research
methodological rigor. Several factors may contribute to a lack of method” [25, 64]. Using the term ‘grounded theory’ to denote any
methodological detail, including space constraints and simply not kind of theory building or qualitative data analysis undermines the
knowing what details to report. Since we only analyzed journal legitimacy of GT, which prescribes a highly structured analytical
articles (rather than conference papers), space constraints are less approach. This engenders undue suspicion of GT studies, possibly
valid as an excuse. Most seminal works on GT focus on how to hindering publication.
collect and analyze data rather than what details to give in the
Similarly, it undermines other qualitative methods. Grounded
methodology section of a paper. We therefore provide some
Theory is not the only valid method of either analyzing
general advice for reporting grounded theory studies followed by
predominately qualitative data or generating theories. Numerous
a list of specific details to include.
other qualitative methods exist [20]. Recasting interpretive
5.1 Method Slurring interview studies, positivist case studies and ethnographies as
Several SE articles claim to use grounded theory, yet do not grounded theory implicitly disparages and devalues these
appear to embrace its core characteristics (Sec. 2.2). If a study legitimate research approaches. There is nothing wrong with
does not involve simultaneous data collection and analysis, conducting an ethnography, for example, and researchers should
constant comparison, coding, memoing and theory development, not be hesitant to label their research as such. Theories can also be
it is not a grounded theory study. If researchers collect most or all developed based on intuition and experience, or by extending and
of their data before beginning analysis, collect or categorize data synthesizing existing research.
according to existing theory, base analysis on seed categories or Furthermore, method slurring misrepresents the current research.
preconceived analytical frameworks, they are not using grounded A key principle of science communication is accurately describing
theory. how data was collected and analyzed [32]. This allows reviewers
Claiming to use a research method without actually following its and readers to evaluate the quality of a study. If a study claims to
guidelines is referred to as “method slurring” [3]. Based on other have used GT while actually doing something different, it violates
authors’ and our own observations, we suggest researchers might this principle.
commit method slurring for at least five reasons: Because so many GT articles lack methodological detail, it is
1. To confer legitimacy. Grounded theory is more structured difficult for readers to assess whether studies actually use
(and is therefore often perceived as more scientific) than grounded theory or simply reference grounded theory “as a
other methods of building theories from primarily unstruc- methodological rationale” [16].
tured data. Charmaz lamented that “Numerous researchers
have invoked grounded theory as a methodological rationale
5.2 Considerations for Conducting and
to justify conducting qualitative research rather than adopt- Reporting Grounded Theory
ing its guidelines to inform their studies” [16]. Individual researchers will have their own styles and preferences
2. To avoid detailed and exhaustive literature review and for conducting and reporting their studies. However, to avoid
initial conceptualization: Many researchers may readily method slurring (among other problems) we offer four broad
embrace the grounded theory maxim of avoiding becoming recommendations.

127
Firstly, it is important to study grounded theory before starting. As literature, researchers should simply state that those techniques
several authors have noted, grounded theory may suffer from its have been used without discussing GT. Practices including
‘apparent simplicity’ [31]. Superficially, GT appears to involve coding, memoing and constant comparison are all part of the
simply reading and categorizing some text. However, a key contemporary qualitative data analyst’s toolbox. They can exist on
challenge in GT is that of theoretical sensitivity: a researcher’s their own, independent of their proponents or any particular
capability to develop useful and interesting concepts that research method. Bringing in GT clouds the issue.
contribute to a theory. Furthermore, GT is a complicated research Finally, and related to the previous point, researchers should not
method with multiple variants and conflicting guidance. Many claim to have used grounded theory when they have not.
overviews and guiding literature for SE researchers do not even Researchers should describe how they analyzed data or generated
include grounded theory (cf. [27, 77]). Anyone considering a GT theory. If using another method it should be named. If a
study should read several books before even deciding whether GT researcher has developed his/her own method, it should be
is the right method, let alone beginning data collection. Good explained. If a researcher has proceeded ad hoc, such a
introductions are available for classic GT [34, 38], Straussian GT “pragmatic, agile approach” should be explained rather than
[19], and constructivist GT [16]. Our review contains numerous dressing it up as grounded theory. To be clear, we accept any
exemplars (e.g. [2, 18, 45, 47, 49]), which may be consulted. GT variant of GT as grounded theory, in contrast to Glaser who only
should be considered from the conception of a study as it differs recognizes ‘classic’ GT (as described in the ‘Discovery’ book)
in quite significant ways from traditional studies as outlined in and considers Straussian GT not to be GT [35] (p. 123).
Sec. 2. Research cannot be reconstructed as GT at write-up.
We further provide an extensive list of considerations for
Secondly, researchers should describe their implementation of grounded theory in software engineering (Fig. 5), which include a
GT, not GT in principle. Some studies in our sample provided variety of potentially relevant issues for consideration when
quite reasonable summaries of GT, but did not explain their conducting or evaluating a GT study. The items in Fig. 5 may be
practices, deviations or precisely what they did. Because GT is especially useful for novices writing their first GT study, experts
relatively new to SE, and to avoid method slurring, it is crucial to who need to jog their memories for methodological dimensions to
explain exactly what was done in the study at hand. In particular, address, or anyone who struggles to explain how they collected
we recommend explicitly describing how key practices (e.g. and analyzed predominately qualitative data. The items in Fig. 5
simultaneous data collection and analysis, constant comparison, are synthesized from existing methodological guidance for GT
memoing) were used. We also recommend explicitly describing and predominately qualitative studies (cf. [16, 24, 28, 78]), as well
deviations from GT guidelines. as our own experience in conducting qualitative studies. No single
Thirdly, researchers should avoid ‘borrowing’ rhetoric. If article can or should include all of these items. Instead, we offer
techniques have been borrowed from the grounded theory them as a reminder of “questions to ask oneself” before and

General Grounded Theory Issues Grounded Theory Data Collection and Analysis
• What variant of grounded theory have you adopted? What • What data was collected (e.g. field notes, documents,
published guidance did you follow? emails, video of meetings), how and when?
• How and why have you adapted, or deviated from, this variant • Who collected and analyzed the data? Was it an individual
and guidance? researcher or research team? If a team, who did what? How
• State the research area or research question—either your initial was this coordinated?
question, the question that emerged during your study, or • Describe the pacing of analyzing data, and how it continued
preferably both. throughout the project.
• State your epistemological and ontological positions (e.g. • Describe your coding, memoing and sorting with examples.
interpretivism, critical realism). • Describe the emergence of your core category, and how this
• State the duration of the study. affected your analysis.
Site Selection and Description • If using classic GT, did you use any of Glaser’s coding
• What organization, team, dataset, etc. did you study? families? If so, which, and did the theoretical codes earn
• Why did you study this data? their way into the theory?
• Describe the context of the study (e.g. the kind of organization, • If using Straussian GT: state how you used the conditional
who is involved, what kind of software is being developed). matrix.
Role of the Literature in the Grounded Theory Study • How and where was your data stored? How did you manage
• Did you begin data collection with a clean theoretical slate? the volume and heterogeneity of data?
• What topic areas did you review before and during the study? • Describe your theoretical sampling with examples.
• How does the literature inform, support or refute your analysis • Confirm that you employed constant comparison.
and results? • When did you stop collecting data? Describe how
Presenting and Evaluating Grounded Theory theoretical saturation became apparent.
• Is the theoretical contribution clearly stated? • Describe how the selected GT variant affected data
• Is the generated theory integrated back into the literature? collection and analysis.
• Is the theory evaluated? If so, using which criteria? • Did you conduct a reliability check; i.e., have your analysis
• How might your own biases, preconceptions, background and reviewed by someone else. If so, who, how, what did they
beliefs affect your analysis? find and what changes resulted? Describe their expertise.
Figure 5. Specific considerations for conducting and reporting grounded theory

128
during a study and write-up. Simply confirming that a study cognizant of its historical development or appreciate the
follows the various core GT guidelines (e.g. simultaneous data differences across its three main variants. This paper aims to draw
collection and analysis, constant comparison, theoretical attention to this issue and to report on the use of Grounded Theory
saturation) should be unnecessary. However, because GT is still in SE. The contributions of this paper are fourfold:
relatively new to software engineering, and our study 1. We provide a detailed comparison of the three main variants
demonstrates some confusion about how GT works, clearly of grounded theory, which may help aspiring grounded
describing what was done and enumerating adherence to core theory researchers in software engineering to select the
guidelines will benefit readers, reviewers and editors. variant that suits them best (Sec. 2);
5.3 Challenges in Doing Grounded Theory 2. Based on an analysis of almost 100 articles in nine prominent
SE journals, we found that many SE articles do not generate
Research in Software Engineering a theory, do not clearly indicate which variant of grounded
Software development contexts present several unusual challenges theory is used and do not provide sufficient methodological
for grounded theory research. Most of the GT research we have detail for rigorous evaluation (Sec. 4);
read relies primarily on interviews and documents. However, 3. We offer integrated guidance for conducting and reporting
software contexts provide diverse data sources including: source grounded theory research in software engineering, including
code, test suites, code commit logs, task and effort data from a set of suggestions for explaining the study’s data collection
project management software, design diagrams (e.g. wireframes, and analysis procedures (Fig. 5);
class diagrams), design documents, domain models (e.g. 4. We enumerate substantial challenges peculiar to conducting
scenarios, personas, user stories, use cases), project management GT research in software engineering, including the
documents (e.g. backlogs, burn-down charts), performance data, proliferation of heterogeneous unstructured, semi-structured
issue tracker data, photos of temporary diagrams (e.g. on white and structured data (Sec. 5).
boards), online discussions (e.g. on IRC or Slack), contracts and
financial statements. Combining these with the usual data (i.e. These contributions should be interpreted in light of several
audio/video recordings of interviews and meetings, documents, limitations. We limited our study to those articles published in
email, field notes) exacerbates at least three challenges: nine well-known software engineering journals. While we believe
these journals are a reasonable surrogate for the broader SE
1. Managing large amounts of heterogeneous data. Version literature, the field has many more, including journals which are
control, project management, team communication systems focused on specific research areas (e.g. Requirements
and other technical affordances make it easy to get access to Engineering). We also excluded conference papers, reasoning that
an enormous, unreadable dataset. Capturing, storing, page limits would force authors to include less methodological
indexing and managing all this data is practically detail. Further sampling bias could come from articles missed due
challenging. Systems appropriate for storing some data types to our specific search string and search strategy, or due to
(e.g. NVivo for audio, video, transcripts and documents) may publication bias. Furthermore, we can only analyze the way each
be unsuitable for storing other data types (e.g. code). study is reported rather than how it was done. A few missing
Determining what to read when you have more text than you methodological details clearly does not mean that the research is
can read in a lifetime is even more challenging. The poor or that the authors are unskilled. Our review simply reveals
implications of data magnitude for theoretical sampling that more methodological detail is needed and suggests potential
remain unclear. However, one strategy is to choose an details to include in future articles.
explicit primary data source (e.g. interviews) and
theoretically sample from the remaining data based on leads We believe that grounded theory offers a highly suitable
arising from the primary data source. methodology to address social, cultural and human aspects in
2. Coding unconventional texts. While they may apply more software engineering—several GT studies in SE have contributed
broadly, the coding approaches associated with GT (e.g. novel and rich insights. As described above, software engineering
open and theoretical coding) were developed primarily for presents non-trivial challenges for grounded theory research.
analyzing unstructured text. It is not clear how to apply open However, grounded theory remains one of the most rigorous
coding to design diagrams, structured text (e.g. use cases) or methods to generate new theories. This is a significant issue as the
source code. One approach is to open-code unstructured text establishment of a strong theory base has been identified as an
and move directly to memoing for more structured data. important challenge for the software engineering discipline [50,
Another is to adopt completely different analytical 65, 66]. We believe well conducted GT studies can make
techniques; for instance, static code analysis. significant contributions to our field and help to develop rich
3. Cross-referencing participant statements with records. theories to inform future empirical studies in SE.
Participants’ post-hoc reconstructions of how and why they 7. ACKNOWLEDGMENTS
performed certain actions are less reliable than, for example,
We thank Lutz Prechelt and the anonymous reviewers for
their accounts of their current frustrations or enduring values.
constructive feedback. This work was supported, in part, by
Source code, commit logs, project management data and di-
Science Foundation Ireland grant 13/RC/2094 to Lero—the Irish
rect observation allow the researcher to triangulate many in- Software Research Centre (www.lero.ie); the Irish Research
terviewee claims. This presents myriad challenges regarding
Council New Foundations Scheme 2014; Enterprise Ireland grant
not only how to triangulate but also how to resolve conflict-
IR/2013/0021 to ITEA2-SCALARE (www.scalare.org); and the
ing evidence. Royal Irish Academy under the Charlemont Award Programme.
6. CONCLUSION
Grounded theory is an increasingly popular research method in
8. REFERENCES
software engineering (see Fig. 2). However, grounded theory is [1] Adolph, S., Hall, W. and Kruchten, P. 2011. Using grounded
complex and demanding, with several variants and conflicting theory to study the experience of software development.
guidance, and software engineering researchers may not be Empirical Software Engineering, 16, 4, 487-513.

129
[2] Adolph, S., Kruchten, P. and Hall., W. 2012. Reconciling [23] Dittrich, Y., John, M., Singer, J. and Tessem, B. 2007.
perspectives: A grounded theory of how people manage the Editorial: For the Special issue on Qualitative Software
process of software development. J Sys Softw, 85, 1269- Engineering Research. Inform Soft Technol, 49, 6, 531-539.
1286. [24] Dube, L. and Pare, G. 2003. Rigor in information systems
[3] Baker, C., Wuest, J. and Stern, P.N. 1992. Method slurring: positivist case research: Current practices, trends and
the grounded theory/phenomenology example. Journal of recommendations. MIS Quart., 27, 4, 597-635.
Advanced Nursing, 17, 1355-1360. [25] Duchscher, J.E.B. and Morgan, D. 2004. Grounded theory:
[4] Benoliel, J.Q. 1996. Grounded theory and nursing reflections on the emergence vs. forcing debate. Journal of
knowledge. Qualitative Health Research, 6, 3, 406-428. Advanced Nursing, 48, 6.
[5] Berger, P. and Kellner, H. 1981. Sociology Reinterpreted: An [26] Dybå, T., Prikladnicki, R., Rönkkö, K., Seaman, C. and
Essay on Method and Vocation. Penguin, Harmondsworth. Sillito, J. 2011. Special issue on qualitative research methods
[6] Bertelsen, O. 1997. Toward a unified field of se research and in software engineering. Empir Software Eng, 16, 2.
practice. IEEE Softw., 14, 6, 87-88. [27] Easterbrook, S., Singer, J., Storey, M.-A. and Damian, D.
[7] Bettenburg, N., Hassan, A.E., Adams, B. and German, D.M. 2008. Selecting empirical methods for software engineering
2013. Management of community contributions: A case research. In: F. Shull, J. Singer and D. I. K. Sjøberg (Eds.)
study on the Android and Linux software ecosystems. Guide to Advanced Software Engineering. Springer.
Empirical Software Engineering, 20, 1, 252-289. [28] Eisenhardt, K.M. 1989. Building theories from case study
[8] Birks, M. and Mills, J. 2011. Grounded Theory: A Practical research. Academy of Management Review, 14, 4, 532-550.
Guide. Sage. [29] Espinosa-Curiel, I.E., Rodríguez-Jacobo, J. and Fernández-
[9] Bogdan, R. and Taylor, S. 1975. Introduction to Qualitative Zepeda, J.A. 2013. A framework for evaluation and control
Reseach Methods. Wiley & Sons, New York. of the factors that influence the software process
improvement in small organizations. Journal of software:
[10] Boychuk Duchscher, J.E. 2004. Grounded Theory: Evolution and Process, 25, 4, 393-406.
Reflections on the emergence vs. forcing debate. Journal of
Advanced Nursing, 48, 6, 605-612. [30] Fagerholm, F., Ikonen, M., Kettunen, P., Münch, J., Roto, V.
and Abrahamsson, P. 2015. Performance Alignment Work:
[11] Bryant, A. and Charmaz, K. 2007. The SAGE Handbook of How software developers experience the continuous
Grounded Theory. Sage. adaptation of team performance in Lean and Agile
[12] Carvalho, L., Scott, L. and Jeffery, R. 2005. An exploratory environments. Inform Soft Technol.
study into the use of qualitative research methods in [31] Fitzgerald, B. and Howcroft, D. 1998. Towards dissolution of
descriptive process modelling. Information and Software the IS research debate: from polarization to polarity. Journal
Technology, 47, 2, 113-127. of Information Technology, 13, 4, 313-326.
[13] Carver, J. 2004. The Impact of Background and Experience [32] Garvey, W.D. and Griffith, B.C. 1971. Scientific
on Software Inspections. Empir Software Eng, 9, 259-262. communication: Its role in the conduct of research and
[14] Chamberlain-Salaun, J., Mills, J. and Usher, K. 2013. creation of knowledge. American Psychologist, 26, 4.
Linking symbolic interactionism and grounded theory [33] Gell-Mann, M. 1999. Complex adaptive systems. In:
methods in a research design: From Corbin and Strauss' Complexity: Metaphors, models and reality Westview Press.
Assumptions to Action. SAGE Open, 3, 3, 1-10.
[34] Glaser, B.G. 1978. Theoretical Sensitivity. Sociology Press.
[15] Charmaz, K. 1990. "Discovering" illness: using grounded
theory. Social Science and Medicine, 30, 11, 1161-1172. [35] Glaser, B.G. 1992. Basics of Grounded Theory Analysis:
Emergence vs Forcing. Sociology Press.
[16] Charmaz, K. 2014. Constructing Grounded Theory. Sage,
2nd Ed. [36] Glaser, B.G. 1998. Doing Grounded Theory: Issues and
Discussions. Sociology Press.
[17] Coleman, G. and O'Connor, R. 2008. Investigating software
process in practice: A grounded theory perspective. Journal [37] Glaser, B.G. 2002. Constructivist Grounded Theory? Forum:
of Systems and Software, 81, 772-784. Qualitative Social Research, 3, 3, Art. 12.

[18] Coleman, G. and O’Connor, R. 2007. Using grounded theory [38] Glaser, B.G. and Strauss, A.L. 1967. The Discovery of
to understand software process improvement: A study of Grounded Theory: Strategies for Qualitative Research.
Irish software product companies. Information and Software Aldine de Gruyter, New York.
Technology, 49, 6, 654-667. [39] Glass, R.L., Vessey, I. and Ramesh, V. 2002. Research in
[19] Corbin, J. and Strauss, A. 2015. Basics of Qualitative software engineering: an analysis of the literature. Inf Softw
Research: Techniques and Procedures for Developing Technol, 44, 8, 491-506.
Grounded Theory. Sage, 4th Ed. [40] Goulding, C. 2002. Grounded Theory: A Practical Guide for
[20] Creswell, J.W. 2013. Qualitative Inquiry & Research Management, Business and Market Researchers. Sage.
Design: Choosing Among Five Approaches. Sage, 3rd Ed. [41] Guba, E. and Lincoln, Y. 1994. Competing paradigms in
[21] Denzin, N. 1983. Interpretive interactionism. In: G. Morgan qualitative research. In: N. Denzin and Y. Lincoln (Eds.) The
(Ed.) Beyond Method. Sage, California. Handbook of Qualitative Research. Sage.

[22] Denzin, N. 2007. Grounded Theory and the Politics of [42] Heath, H. and Cowley, S. 2004. Developing a grounded
Interpretation. Sage. theory approach: a comparison of Glaser and Strauss.
International Journal of Nursing Studies, 41, 141-150.

130
[43] Hirschheim, R. 1985. Information systems epistemology: an Implementation Theory and Function-Behavior-Structure
historical perspective. In: E. Mumford, R. Hirschheim, G. Theory. Inform Soft Technol, 70, 232-250.
Fitzgerald and A. Wood-Harper (Eds.) Research Methods in [61] Ralph, P. and Kelly, P. 2014. The Dimensions of Software
Information Systems. Elsevier. Engineering Success. In: International Conference on
[44] Hoda, R., Noble, J. and Marshall, S. 2011. Grounded theory Software Engineering. ACM, Hyderabad, India.
for geeks. In Proc. 18th Conference on Pattern Languages of [62] Seaman, C.B. 1999. Qualitative methods in empirical studies
Programs. of software engineering. IEEE Trans Softw Eng, 25, 4.
[45] Hoda, R., Noble, J. and Marshall, S. 2011. The impact of [63] Seo, H., Sadowski, C., Elbaum, S., Aftandilian, E. and
inadequate customer collaboration on self-organizing Agile Bowdidge, R. 2014. Programmers' build errors: a case study
teams. Information and Software Technology, 53, 5, 521-534. (at google). In Proc. International Conference on Software
[46] Hoda, R., Noble, J. and Marshall, S. 2012. Developing a Engineering. ACM.
grounded theory to explain the practices of self-organizing [64] Stern, P.N. 1994. Eroding grounded theory. In: J. M. Morse
Agile teams. Empir Software Eng, 17, 6, 609-639. (Ed.) Critical Issues in Qualitative Research Methods. Sage.
[47] Hoda, R., Noble, J. and Marshall, S. 2013. Self-organizing [65] Stol, K.J. and Fitzgerald, B. 2015. Theory-Oriented Software
roles on agile software development teams. IEEE Trans Engineering. Science of Computer Programming, 101, 79-98.
Softw Eng, 39, 3, 422-444.
[66] Stol, K.J., Goedicke, M. and Jacobson, I. 2016. Introduction
[48] Holland, J.H. 1992. Complex Adaptive Systems. Daedalus, to the special section—General Theories of Software
121, 1, 17-30. Engineering: New advances and implications for research.
[49] Jantunen, S. and Gause, D.C. 2014. Using a grounded theory Inf Softw Technol, 70, 176-180.
approach for exploring software product management [67] Stol, K.J., Ralph, P. and Fitzgerald, B. 2016. Appendix to
challenges. Journal of Systems and Software, 95, 32-51. "Grounded Theory Research in Software Engineering".
[50] Johnson, P., Ekstedt, M. and Jacobson, I. 2012. Where's the University of Limerick.
Theory for Software Engineering? IEEE Softw., 29, 5, 94-96. [68] Strauss, A. and Corbin, J. 1991. Basics of Qualitative
[51] Kelle, U. 2005. "Emergence" vs. "Forcing" of Empirical Research: Techniques and Procedures for Developing
Data? A crucial problem of "Grounded Theory" Grounded Theory. Sage.
Reconsidered. Forum: Qualitative Social Research, 6, 2. [69] Strauss, A. and Corbin, J. 1994. Grounded Theory
[52] Kenealy, G. 2008. Management Research and Grounded Methodology: An Overview. In: N. Denzin and Y. Lincoln
Theory: A review of grounded theory building approach in (Eds.) Handbook of Qualitative Research. Sage.
organisational and management research. The Grounded [70] Strauss, A. and Corbin, J. 1998. Basics of Qualitative
Theory Review, 7, 2, 95-117. Research: Techniques and Procedures for Developing
[53] Layman, L., Williams, L., Damian, D. and Bures, H. 2006. Grounded Theory. Sage, 2nd Ed.
Essential communication practices for Extreme Programming [71] Strauss, A.L. 1987. Qualitative analysis for social scientists.
in a global software development team. Information and Cambridge University Press.
Software Technology, 48, 9, 781-794.
[72] Suddaby, R. 2006. From the editors: What grounded theory
[54] Maglyas, A., Nikula, U. and Smolander, K. 2013. What are is not. Academy of Management Journal, 49, 4, 633-642.
the roles of software product managers? An empirical
investigation. Journal of Systems and Software, 86, 12, 3071- [73] Trochim, W. 2001. Research Methods Knowledge Base.
3090. Atomic Dog Publishing, Cincinnati, OH, USA.
[55] Mohanani, R., Ralph, P. and Shreeve, B. 2014. Requirements [74] Urquhart, C. 2013. Grounded Theory for Qualitative
Fixation. In Proc. International Conference on Software Research: A Practical Guide. Sage.
Engineering. ACM. [75] Urquhart, C., Lehmann, H. and Myers, M.D. 2010. Putting
[56] Morgan, G. (Ed.). 1983. Beyond Method. Sage, CA, USA. the 'theory' back into grounded theory: guidelines for
grounded theory studies in information systems. Information
[57] Morse, J.M. (Ed.). 1994. Critical Issues in Qualitative Systems Journal, 20, 357-381.
Research Methods. Sage.
[76] Walker, R. 1988. Applied Qualitative Research. Gower,
[58] Opie, C. 2004. Research Approaches. In: C. Opie (Ed.) Hampshire.
Doing educational research. Sage, London.
[77] Wohlin, C., Höst, M. and Henningsson, K. 2003. Empirical
[59] Prechelt, L. and Oezbek, C. 2011. The search for a research research methods in software engineering. In: ESERNET,
method for studying OSS process innovation. Empirical volume LNCS 2765. Springer.
Software Engineering, 16, 4, 514-537.
[78] Yin, R.K. 2008. Case study research: Design and methods.
[60] Ralph, P. 2015. Software engineering process theory: A Sage, CA, USA, 4th Ed.
multi-method comparison of Sensemaking-Coevolution-

131