Critical Case
Critical Case
Systems Research
Knut H. Rollanda & Jo Herstadb
[email protected] and [email protected]
Department of Informatics, University of Oslo
Abstract
Information systems research has taken many different directions and a host of
different approaches have been used. Different researchers within the
multidisciplinary field of IS seem to prefer some approaches to others based on
both epistemological and more practical grounds. In this paper we look at the case
study approach in IS research and focus on the importance of selecting ‘critical
cases’. A case study is an in-depth study of the particular where the researcher
seeks to increase his understanding of the phenomena studied. Often case studies
and particularly interpretive case studies have been criticized because of the
assumed difficulty with generalizations. We argue that this critique is misdirected,
and based on Flyvbjerg’s concept of a ‘critical case’, ways of generalizing from
case studies are indicated.
Keywords: Case study, Information systems research, Critical case, IS and organization
Introduction
There exist a rich body of literature discussing epistemological and methodological issues
concerning the selection and use of different approaches in information systems research
(e.g. Braa & Vidgen, 1999; Cornford & Smithson, 1996; Galliers, 1992a; Hirschheim,
1992; Irani et al., 1999; Lee, 1999; 1997; Walsham, 1995). In fact, IS related literature
seems to be teeming with discussions on epistemological issues, reviews of different
approaches, concrete methods and techniques, and guidelines for embarking on
information system research. This paper focuses on one particular approach frequently
discussed and used in IS research – namely the case study.
In both IS and management related literature the problems of generalizing upon
one single case has been emphasized as a serious drawback. Case studies have often been
seen as a fruitful way of studying a phenomenon before using a more formal and rigorous
approach for verifying or falsifying existing theory. Some researchers even define the
case study in itself as almost of no value, and hence an inappropriate approach for doing
scientific research. In this paper we strongly reject this view and want to focus on how
case studies can make profound contributions within information systems research by
selecting critical cases.
The concept of a critical case has been suggested by Flyvbjerg (1991), and a
critical case is a case of particular interest and with strategic content in relation to the
research questions investigated. In general, extreme and atypical cases tend to give more
information, and thus the phenomenon studied becomes more visible (Stake, 1994). The
different aspects of a context or mechanisms from which a particular problem situation
originates become increasingly visible and more accessible for the researcher.
Our motivation for looking at critical cases in information systems research is
also founded in the nature of information systems and information technologies. Critical
cases where contradictory and paradoxical issues can be illuminated could contribute to
existing theory or illustrate the inadequacy of theories and methods used. For example,
the way in which information systems and information technologies effect individuals,
organizations and societies are often contradictory. That users use work arounds and
improvisations in order to integrate new technology in their context specific and situated
work seems to be the rule rather than the exception. Studies of implementation of
information technologies in organizations have revealed contradictory findings both
within and across studies (see Robey & Boudreau, 1999). Furthermore, on a societal
level, nobody could have predicted the impact of the Internet as a global information
infrastructure – which as a technology, for a very long time was seen as inferior to other
approaches to data communication as for instance the OSI standard (Hanseth et al.,
1996).
Defining information systems research by giving an extensive review of different
research methods is not our aim here. Neither is it our objective to focus on
epistemological issues – for example the question whether interpretative methods are
more appropriate than positivistic research methods. We agree that these issues are
important for the information systems field as an academic discipline because of the
profound implications of the underlying epistemological assumptions (Checkland &
Holwell, 1998). In this paper, however, we want to concentrate on the case study
approach – and particularly on how and why critical cases are useful.
The paper is structured in the following way. In the following section, we discuss
the case study approach in information systems research and in general – and particularly
how a critical case can be used for generalizations. Next, in the discussion we exemplify
this by drawing from critical cases found in the IS literature and point at the importance
of understanding context in relation to IS research. Finally, we conclude that the critical
case can be very valuable for generalizing upon findings in case studies within
information systems research.
In line with Flyvbjerg (1991), it can be argued this criticism of the case study approach is
misdirected, and that these five misconceptions are often – at least implicitly, to be found
within information systems research as well. In this paper though, the three first points
related to context, generalization, and hypothesis testing will be further discussed.
In disqualifying these propositions about case studies, Flyvbjerg goes on
discussing their philosophical foundations and that learning basically is based on specific
and context sensitive cases or examples. Subsequently, since case studies deal with real-
world situations and concepts, it is clear that case studies are important for researchers’
own learning processes. Flyvbjerg builds on Dreyfus’ five levels of learning and claims
that contextless and generic concepts will only get you on the first level of learning. In
order to reach the higher levels, you need to build on your own experiences and concrete
cases. Experts act on a basis of thousands of different concrete cases – not purely on
statistical generalizations. Furthermore, Flyvbjerg underscores that it is likely that
predictive theory of human action and societies will never exist. Thus, the first
proposition does not hold because context dependent knowledge could be just as valuable
as context independent.
Regarding the question of generalization, Flyvbjerg (1991) emphasizes that the
case study is well suited for the identification of “black swans” – or what Karl Popper has
denoted falsification. Case studies are useful for falsifying existing theories, because case
studies pays particularly attention to context and situations that might explain why the
outcome of an action is inconsistent or consistent with theory. A critical case can often be
identified as a case of particular interest and with strategic content in relation to the
research questions investigated. Similarly, Stake (1994) argue that extreme and atypical
cases tend to give more information, and thus the phenomenon studied becomes more
visible:
Whereas single or a few cases are poor representation of a population of cases and poor
grounds for advancing grand generalization, a single case as negative example can
establish limits to grand generalization. (Stake, 1994: p. 245)
Hence, in contrast to a statistical analysis of data based on surveys, which tend to exclude
extreme examples in the analysis of the data, those extreme examples should be
deliberately searched for when following a case study approach. Such extreme examples
could be critical cases where it is possible to gain information on a particular
phenomenon that opens up for falsifying or verifying existing theories. Insights based on
a critical case could enable the researcher to draw logical conclusions as “if this is true
for this case, then this is true for all cases” or “if this is not true for this case, then it is
not true for any other case either”. The former can be referred to as “most-likely cases”
and the other can be called “least-likely cases”. To be able to use a specific case for
falsifying or verifying existing theories depends on how strategic the case is in relation to
the given research a question. Selecting a critical case increases the possibility for making
generalizations and falsifying or verifying existing theory.
Furthermore, Flyvbjerg adds that critical cases often activate a larger number of
actors and fundamental mechanisms in the situation studied. The different aspects of a
context or mechanisms from which a particular problem situation originates become
increasingly visible and more accessible for the researcher. Thus, the second and the third
propositions do not hold and Flyvbjerg even argues that in many cases it is the other way
around: case studies are good basis for making generalizations and they are especially
well suited.
Discussion
Conclusion
In this paper we have presented some arguments for using case studies in information
systems research, and especially argued for the importance of selecting ‘critical cases’. A
brief review of the IS literature shows various views concerning the validity of
generalizations from single cases, and some – at least implicitly assumes that generic
knowledge and universal rules are more useful than concrete examples and cases. On the
contrary, empirical case studies in the IS literature show that such statistical
generalizations and “universal rules” are not always useful, because they do not – among
other things, explain the contradictory effects of IT in organizations. This is not because
the case study approach is an inappropriate approach for scientific research, but due to the
fact that information systems are embedded in social and cultural structures or a context
through its use. However, this does not exclude quantitative techniques like surveys and
statistical analysis from information systems research - but not all phenomena can be
understood and explained using such techniques.
However, the critical case is no “silver bullet” or a “final solution” for how to do
case study based research. Identifying a critical case a priori – before the researcher has
started to investigate the actual case is often difficult – or even impossible. Research
projects, and especially case studies, often develop over time becoming something else
than first intended. As noted by Stake (1994: p. 237) “a case study is both the process of
learning about the case and the product of our learning”. It will always be easier to tell
whether the case was critical or not after the study has ended. There is no generic method
to apply to find out whether or not a particular case is a critical case in relation to a
specific research question. However, one solution to this problem can be to look at
similar cases within information systems research or related fields. For example, if you
are interested in building a theory for development and diffusion of communication
standards, you could build on insights from critical cases like the “standards war”
between JVC and Sony and try to find similar cases or cases that seem very different.
Grindley (1995) draws from this insight in understanding the profound importance of the
standards involved in technology products, and particularly in IT-based products. A
standard that builds up a larger installed base with complementary products, becomes
more attractive, and eventually reinforces itself. JVC succeeded in the introduction of the
VHS standard in competition with Sony’s Betamax – not because of superior technology
– but rather because JVC managed to build up a larger installed base. A complementary
product, in terms of pre-recorded tapes appeared first in VHS, which resulted in rapid
decrease in sales of Betamax. In addition, JVS provided other vendors with the
specifications of the standard, so other complementary products and more VCRs using
the VHS standard were deployed.
We will argue that even though there are no generic rules or grand theories that
can explain information systems in organizations and the way IT effects and is effected
by social aspects etc., we do think it is important that IS researchers try to generalize from
their findings in case studies – but not in a deterministic way. This can be done by
selection of a critical case in order to be able to falsify or verify existing theory.
Moreover, like Walsham (1995) has indicated, critical cases in interpretive case studies
can give implications for specific domains and rich insights. By selecting a critical case
specific contradictions, paradoxes and side-effects becomes increasingly visible in the
phenomena studied. In this way a case study can serve as an important approach for
generalizing based on rather descriptive research findings. Thus, we do not believe that
the case study approach has the generic weaknesses as often anticipated – it depends on
how critical the particular case is in relation to the phenomena studied and the ways in
which the researcher is able to generalize from his findings.
References
Beyer, H. & Holtzblatt, K. 1998. Contextual Design, San Francisco: Morgan Kaufmann
Publishers.
Bowers, J, Button, G, & Sharrock, W. 1995. Workflow from Within and Without: Technology
and Cooperative Work on the Print Industry Shopfloor. In: Marmolin et al. (eds) Proceedings
of the Fourth European Conference on Computer-Supported Cooperative Work, ECSCW’95,
pp. 51-66.
Braa, K. & Vidgen, R. 2000. Chapter 12. Research: From observation to intervention. In: Braa,
K., Sørensen, C. & Dahlbom, B. (eds.). Planet Internet. Lund: Studenlitteratur.
Checkland, P. & Holwell, S. 1998. Information, Systems, and Information Systems – Making
sense of the field. Chichester: Wiley.
Cornford, T. & Smithson, S. 1996. Project Research in Information Systems. London:
Macmillian.
Durnati, A. & Goodwin, C. 1992. Rethinking context: an introduction. In: Durranti & Goodwin
(eds.) Rethinking Context: Language as an interactive phenomenon. Cambridge, UK:
Cambridge University Press.
Easterby-Smith, M., Thorpe, R. & Lowe, A. 1991. Management Research – an introduction.
London: Sage.
Flyvbjerg, B. 1991. Rationalitet og Makt (in Danish: Rationality and Power). Akademisk forlag.
Galliers, R.D. (ed.). 1992a. Information Systems Research – Issues, Methods and Practical
Guidelines. Oxford: Blackwell Scientific Publications.
Galliers, R.D. 1992b. Choosing Information Systems Research Approaches. In: Galliers, R.D.
(ed.). 1992. Information Systems Research – Issues, Methods and Practical Guidelines.
Oxford: Blackwell Scientific Publications.
Giddens, A. 1984. The Constitution of Society. Polity Press.
Grindley, P. 1995. Standards, Strategy, and Policy – Cases and Stories. Oxford: Oxford
University Press.
Hanseth, O., Monteiro, E, and Hatling, M. 1996. Developing Information Infrastructure: The
tension between standardisation and flexibility. Science, Technology, & Human Values, Vol.
21, No 4, pp. 407-426.
Hirschheim, R.A. 1992. Information Systems Ephistemology: An Historical Perspective. In:
Galliers, R. (ed.). 1992. Information Systems Research – Issues, Methods and Practical
Guidelines. Oxford: Blackwell Scientific Publications.
Irani, Z, Ezingeard, J-N, Grieve, R.J. & Race, P. 1999. A case study strategy as part of an
information systems research methodology: a critique. International Journal of Computer
Applications in Technology, Vol. 12, Nos.2/3/4/5, pp. 190-198.
King, J.L. 1996. Where are the payoffs from computerization? Technology, learning, and
organizational change. In: Kling (ed.). Computerization and Controvercy – Value Conflict
and Social Choices. San Diego: Academic Press.
Klein, H.K. & Myers, M.D. 1999. A set of principles for conducting and evaluating interpretive
field studies in information systems. MIS-Quarterly. Vol.23, No.1; March 1999; pp.67-93.
Kling, R. 1987. Defining the boundaries of computing across complex organizations. In: Boland
& Hirschheim (eds.). Critical issues in Information Systems research. Chichester: Wiley.
Kling, R. 1996. Computers as Tools and Social Systems: the Car-Computer Analogy. In: Kling
(ed.). Computerization and Controvercy – Value Conflict and Social Choices. San Diego:
Academic Press.
Kling, R. and Iacono, S. 1989. The Institutional Character of Computerized Information Systems.
Office: Technology and People, Vol. 5, No. 1, pp. 7-28.
Lee, A.S. 1999. Researching MIS. In: Currie, W. & Galliers, B. (eds.) 1999. Rethinking
Management Information Systems – An Interdiciplinary Perspective. Oxford: Oxford
University Press.
Lee,A.S., Liebenau, J. & DeGross, J.I. 1997. Information Systems and Qualitative Research.
London: Chapman & Hall.
Macredie, RD & Sandom, C. 1999. IT-enabled change: evaluating an improvisational
perspective. European Journal of Information Systems, Vol. 8, No. 4, pp. 247-259.
Markus, M.L. 1983. Power, Politics, and MIS Implementation. Communications of the ACM,
Vol. 26, pp. 430-444.
Monteiro, E. 1998. Scaling Information Infrastructure: The Case of Next-Generation IP in the
Internet. The Information Society, Vol. 14, pp. 229-245.
Orlikowski, W.J. 1992a. Learning from Notes: Organizational Issues in Groupware
Implementation. Proceedings of the Conference on CSCW, ACM, pp. 362-369.
Orlikowski, W.J. 1992b. The Duality of Technology: Rethinking the Concept of Technology in
Organizations. Organization Science, Vol. 3, No. 3, 1992, pp. 398-427.
Orlikowski, W.J. & Robey, D. 1991. Information Technology and the structuring of
organizations. Information Systems Research, Vol.2, No.2, pp. 143-169.
Robey, D. & Boudreau, M. 1999. Accounting for the Contradictory Organizational
Consequences of Information Technology: Theoretical Directions and Methodological
Implications. Information Systems Research, Vol. 10, No. 2, pp. 167-185.
Stake, R. 1994. Case Studies. In: Denzin, N & Lincoln, Y. Handbook of Qualitative Research,
pp. 236-247. Thousand Oaks, CA: Sage Publications.
Suchman, L. 1987. Plans and Situated Actions. Cambridge: Cambridge University Press.
Suchman, L.A. 1995. Making Work Visible. Communications of the ACM, Vol.38, No.9, pp.56-
63.
Walsham, G. 1993. Interpreting Information Systems in organisations. J. Wiley & Sons
Walsham, G. 1995. Interpretive Case Studies in IS Research: nature and method. European
Journal of Information Systems, Vol. 4, No.2, pp.74-81.
Walsham, G. 1998. IT and Changing Professional Identity: Micro-Studies and Macro-Theory.
Journal of the American Society for Information Science, Vol. 49, No. 12, pp. 1081-1089.
Yin, R.K. 1989. Case Study Research – Design and Methods. Newbury Park, California: Sage
Publications.
Zuboff, S. (1988). In the age of the smart machine. New York: Basic Books.