The Marketing Review 2003, 3, 479-498 www.themarketingreview.

com

Tracy G. Harwood 1 and Tony Garry

Leicester Business School, De Montfort University
Delivered by Publishing Technology to: Chinese University of Hong Kong

An Overview of Content Analysis

Content analysis is a method for analysing the content of a variety of data,
such as visual and verbal data. It enables the reduction of phenomena or
events into defined categories so as to better analyse and interpret them.
This paper provides an overview of content analysis from a marketing
perspective. The basic concepts and techniques are presented for
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

operationalising content analyses. As a methodology, it can be both

qualitative, usually in developmental stages of research, and quantitative
where it is applied to determine frequency of phenomena. Thus, it lends
itself to the use of computers to analyse data and, therefore, some of the
main packages currently available to researchers are mentioned. The
benefits and limitations of adopting content analysis are discussed along with
an introduction to sequential analysis, a complementary approach which may
be used to enhance understanding and strengthen research design.

Keywords Content analysis, qualitative, quantitative, sequential analysis,

reliability, validity.

Introduction

Content analysis is a technique that may be used with either qualitative or

quantitative data (Collis and Hussey 2003). Typically, it is a technique that
enables analysis of ‘open-ended’ data to be structured for purposes of
diagnosis. It has been applied to diverse fields of research, including
psychology, anthropology, education, linguistics and history (Krippendorff
1980) since its origin as a method of analysing trends in mass
communications. The technique was first used more than 200 years ago to
analyse textual material from newspaper and magazine articles,
advertisements, political speeches, hymns, folktales and riddles. In recent
times, technological developments facilitate the means for further and deeper
analysis of the vast amounts of data that may be generated through research
techniques such as interviews and discussion group recordings.
The aim of this paper is to provide an overview of content analysis from a
marketing perspective. Basic concepts are firstly presented, including the

1
Correspondence: Tracy G. Harwood, Leicester Business School, De Montfort
University, The Gateway, Leicester, LE1 9BH, Tel: 0116 257 7214, Fax: 0116 257
7204, Contact: [email protected], http://www.dmu.ac.uk/business

ISSN 1472-1384/2003/4/00479 + 19 £4.00/0 ©Westburn Publishers Ltd.

 480 Tracy G. Harwood and Tony Garry

main techniques employed by researchers and a flowchart of a research

design. Reliability and validity issues are, as with all research, particularly
important considerations and these are discussed at length, including
mention of the most commonly used statistical tests. Approaches to
operationalising the methodology are then reviewed. Content analysis lends
Delivered by Publishing Technology to: Chinese University of Hong Kong

itself to a pluralistic approach and, therefore, the complementary

methodology of sequential analysis is introduced and briefly reviewed.
Finally, the benefits and limitations of content analysis are discussed.

Basic Concepts
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

Content analysis is most appropriate where the research interest is in the

characteristics of language and communication (Tesch 1990; Mariampolski
2001). It comprises of a coding scheme based on specific research
questions determined from a review of the relevant literature (Carson et al.
2001). This enables the identification of data units which are categorised,
recorded, compared and contrasted to reach some conclusion about the
content of the communication (Harris 1996; Collis and Hussey 2003). Verbal
data units may be analysed by sentence, single word or image (Kolbe and
Burnett 1991). It is a method for the analysis of the communication message
itself, rather than the sender’s or receiver’s interpretation of that message
(Kassarjian 1977).
To a certain extent, design of the methodology is determined by the
researcher (Clarke 1997; Collis and Hussey 2003) in relating to the
substantive problem and theoretical premise for the study. Application of
content analysis has not been limited to testing previously developed theories
but includes analysing unexplained variations in any uncoded data (Lijphart
1971). It is a valuable qualitative tool in further development of hypotheses.
Examples of data may include documents such as meeting notes and
minutes, letters, memoranda, diaries, speeches, newspaper articles,
timetables, notices, films, television programmes, photographs,
advertisements, open-ended responses to questions as well as more obvious
sources such as interviews and direct observation (Kolbe and Burnett 1991;
Robson 1993). It may be used as a technique in both qualitative and
quantitative studies (Robson 1993; Neuendorf 2002; Collis and Hussey 2003)
and is usually described as being qualitative in the development stages of
research and quantitative where it is applied to determine frequency of
phenomena of interest. Consequently, it has numerous potential uses that
may include (Berelson 1952; Carson et al. 2001)-

· auditing content against objectives;

· constructing and applying communication standards;
· identifying features of style;
· identifying the characteristics of communicators;
· determining psychological states of individuals or groups;
· identifying international differences in communications;
 An Overview of Content Analysis 481

· determining cultural patterns (attitudes, interests, values);

· revealing the focus of attention; and
· describing communication responses (attitude and behaviour).

Content categories must contain homogenous data (Janis 1965) according to

Delivered by Publishing Technology to: Chinese University of Hong Kong

the scheme to be used. That is, all the data classified into the same category
refers to the same object or event and everything which has been excluded
must not refer to that object or event. Various levels of classifying the
content of communications have evolved (Janis 1965) where the meanings of
words are inferred from the context in which they have occurred relative to
accompanying behaviours –
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

· pragmatical content analysis – analysing likely cause and effect, ie.,

frequency of an utterance producing a desired effect on an audience
eg., the number of times something is said which is likely to have the
effect of producing favourable attitudes towards a business proposal in
a negotiation process;
· semantical content analysis – analysing meanings, ie., frequency of
utterances used to describe a particular phenomenon, irrespective of
words used eg., counting the number of times a business proposal is
referred to –
· designation analysis – frequency of reference to objects (persons,
things, groups or concepts);
· attribution analysis – frequency of characterisations (eg., references to
trust);
· assertions analysis – frequency of characterisations to certain objects
ie., thematic analysis (eg., references to trust in the business proposal);
· sign-vehicle analysis – the frequency of an actual utterance.

Techniques

Content analysis provides a methodology to objectively quantify the content

of communication between a sender and a receiver (Holsti 1969) within the
context of the communication (Berelson 1952, in Krippendorff 1980). When
relating communication to context using content analysis, a number of logical
mechanisms may be employed to describe the nature of this analytical
technique. These include systems, standards, indices, linguistic
representations, communications and institutional processes (Krippendorff
1980).

Systems
Krippendorff (1980) defines a system as a conceptual device describing a
portion of reality. Broader descriptions are “an entity that maintains its
existence and functions as a whole through the interaction of its parts”
(O’Connor 1996, p 1); “a boundary-maintaining set of interdependent
 482 Tracy G. Harwood and Tony Garry

particles or sub-units... viewed as occupying one level of a multi-level

hierarchy of systems” (North et al. 1963, p 5). The essential constituents may
be summarised as follows –

· components – of variable states, eg., functional departments within an

Delivered by Publishing Technology to: Chinese University of Hong Kong

organisation such as sales, finance, human resources management,

etc;
· relations – constraints on the co-occurrences of states across
components eg., the Memorandum of Articles of Association specifying
the legal operational environment of a private limited company;
· transformations – the interaction of components (states) within the
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

system constraints (relations), either spatially or temporally.

More familiar examples of systems include the human body, an organisation,

Government and even the Solar System.
Of interest in the systems approach to content analysis are the
transformations which do not vary over time or space and, therefore, enable
extrapolation. Thus, the Hoover Institution’s study of Revolution and the
Development of International Relations (RADIR) resulted in Pool (1951)
correlating hostilities towards other nations with insecurity. More recently,
policy makers have expressed interest in Gerbner et al’s (1979) study of
television violence.
Analysis identifies past trends, patterns and differences which enhances
prediction. Thus, in combining patterns and trends, interesting content
analyses have been developed. For example, when analysing the New York
dailies, Speed (1893) observed a decline in literary matters corresponding to
an increased level of gossip. Bales’ (1950) Interaction Process Analysis
revealed patterns in the inter-relationships of small groups. Holsti et al.
(1965) predicted with a degree of success the emotional response of each
group of US and Soviet Union decision-makers during the 1962 Cuban
Missile Crisis. Subsequently, Bales’ system has been the foundation of many
mechanisms developed to analyse small group situations.

Standards
Standards allow comparison of objects through relative measurement by
identifying, evaluating and auditing the content of communication. Evaluation
is somewhat subjective since many studies have lacked defensible criteria for
measurement against some standard (Krippendorff 1980). Audits are also
subject to judgment although the standard may be intertwined with legal or
professional directives, eg., the Advertising Standards Association, against
whose guidelines of ‘legal, decent, honest and truthful’ advertisements for
non-broadcast media are deemed fit for public consumption
(www.asa.org.uk/). Identification of a given characteristic is, however, easier
since it either does or does not exhibit that characteristic. Janis (1965)
argues most content analyses involve the categorisation of ‘sign vehicles’,
based on meanings interpreted by the receiver. This is at least partially due
 An Overview of Content Analysis 483

to the problems of understanding complex human behaviours, alluded to by

Donohue et al. (1984) in their development of a cue-response coding
mechanism.

Indices
Delivered by Publishing Technology to: Chinese University of Hong Kong

An index is of interest when it may be causally related to other

phenomena. Krippendorff (1980) drew the metaphor of a physician’s
diagnosis of an illness from the patient’s symptoms. Here content analysis
has long been used to measure importance, attention or emphasis;
determine the direction of bias; and, infer the level of belief, conviction or
motivation. For example, McClelland (1958) recorded frequencies of a class
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

of words as an index to the motive of achievement through analysis of

sentence lengths and types of words used; the authenticity of written works
has been repudiated by Morton (1963); and, it has also become a popular
tool of the Computer Age with ‘readability statistics’ facilities now found in
most word-processing software.

Linguistic representations
Discourse (the sequence of a communication) is of considerable interest
since reference to an utterance alone obviates the connotation and
understanding of language. That is, behaviours exhibited may contain a
series of lexical items with essentially similar meanings in isolation, however,
in their context may have entirely different meanings. An example from Hays
(1969) compares the content of three sentences: “the door opened”; “the
janitor opened the door”, “the key opened the door” (p 62). Here the subject
of each sentence is different (door, janitor, key) but the action of opening the
door is the same. It is the relationship between the subject and the event
which changes.
Linguistic representation is, therefore, a method of content analysis
whereby language is classified according to qualitative references to, rather
than a quantitative index of, an object. This has been used by researchers to
develop models of behaviour to analyse, for example, options remaining
during arms limitation negotiations (Allen 1963). Developments in computer-
aided content analysis also enhance the opportunity to analyse inferences,
objectives, disagreements, or alternative courses (Hays 1969; Krippendorff
1980; Wilson and Rayson 1996).

Communications
Communication takes place in the context of an existing relationship
among subjects and, in so doing, may alter that relationship. Although rarely
used in content analysis, this technique has been utilised to analyse
response and cue elements of interactions by Donohue et al. (1984) within
the context of small groups. A more common methodology for analysis of
communication, however, is interaction analysis (Bales 1950; Krippendorff
1980).
 484 Tracy G. Harwood and Tony Garry

Institutional processes
Verbal communications within organisations have many different purposes
and meanings depending on structure and processes. Face value may,
however, be open to a number of interpretations, particularly when viewed by
an external party. Indeed, organisational communications are governed by
Delivered by Publishing Technology to: Chinese University of Hong Kong

legal and professional directives overlaid with macro- and micro-

environmental influences. This has been demonstrated by Clarke (1997) in
her analysis of published company Annual Report and Accounts.
In summary, there are numerous means of adapting the technique to
analyse different forms of data in a variety of research settings. Of key
relevance, however, is the reliability with which such analysis is conducted,
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

and the validity of the categories or coding mechanism that is used in a given
context. This paper now reviews these areas in some detail. Figure 1
provides an overview of a typical content analytic research design, based on
Neuendorf (2002).

Figure 1 Flowchart for Content Analytic Research Design

source: adapted from Neuendorf (2002)

Reliability and Validity

Criticism of content analysis stems from its subjectivity. Researcher bias

affects all stages of the technique from decisions on data collection methods
 An Overview of Content Analysis 485

to analysis and ultimately interpretation of results. Identification of the

mechanism or coding scheme and categories are of prime importance since
reliability is enhanced through validity of data (Kolbe and Burnett 1991).

Reliability
Delivered by Publishing Technology to: Chinese University of Hong Kong

Reliability takes three forms in content analysis: stability, reproducibility

and accuracy (Krippendorff 1980) –

· stability – is the extent to which the analysis is immutable, referring to

the re-coding of data by the same coder or ‘judge’ at different points in
time. Such intra-judge consistency is the weakest form of reliability in
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

content analysis since accurate re-coding of data may be subject to

cognitive dissonance, interpretation of coding rules, data or the context
of the communication (Krippendorff 1980). Furthermore, a common
phenomenon is that of ‘instrument decay’ (Johnson and Bolstad 1973),
in this context used to describe post-training coding of further data.
This is particularly problematic when data is collected and coded ‘live’
as in naturalistic situations but may be at least partially overcome by
continuous training (Johnson and Bolstad 1973).
· reproducibility – is the duplication of coding under different
circumstances eg., different judges or situations. Lack of consensus in
coding may, therefore, result from inconsistencies between judges ie.,
inter-judge.
· accuracy – refers to the process of conforming to a known standard.
This involves comparison of one judge’s coding performance to the
correct measure. Differences represent inconsistencies in intra- or
inter-judge reliability, ie., within or between judges, and deviations from
the standard. It is the strongest measure of reliability but it does
assume the underlying standard is valid. As a training aid for judges,
accuracy is particularly useful since it enables performance to be
monitored.

Training in coding mechanisms may overcome some aspects of judge bias.

It may, however, also present a particular problem when the judge
anticipates certain results. Johnson and Bolstad (1973) outline several
useful techniques to overcome this issue including masking the true purpose
of the assignment, using naive or untrained judges, using new judges at
different phases in the research, overlapping judges to check accuracy and
subtly feeding judges different theories. These techniques are recommended
predominantly for naturalistic (real-life) observations. Ironically this is where
they may be most impractical to implement, not least for reasons of
credibility!
In particular, reliability has been found to vary, depending on how the data
is unitised. This, in turn, is related to the method of data collection
(Guetzkow 1950; Weiner 1971, in Morley and Stephenson 1977), eg.,
responses to questions compared to continuous observation of verbal
 486 Tracy G. Harwood and Tony Garry

behaviour. This is not surprising given the apparent relative ease of unitising
data, which may be by one of the following means –

· temporal – by some discretional time interval;

· transactional – unbroken speech burst;
Delivered by Publishing Technology to: Chinese University of Hong Kong

· psychological – expression of ‘simple thought’;

· categorical – by the system of categories; and
· hybrid – any combination of the above.

Using a number of independent judges to analyse data enhances objectivity

IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

(Kolbe and Burnett 1991) while the process of training of judges may also
offer opportunities to pre-test identified categories through discussion and
consensus. It is also the case that unit sizes may not be appropriate for the
categories chosen. For example, units may be too small, resulting in an
increase in unclassifiable data; or too large, resulting in ambiguities between
judges.
Measures of reliability enhance the research findings, supporting
decisions, drawing inferences from or making recommendations based on
the original data. Such measures also enable the methodology to be used
quantitatively (Guetzkow 1950). Researchers have, however, been criticised
for their under-use of rigorous statistical calculation of category and judge
reliability (Kolbe and Burnett 1991), raising obvious questions on credibility.

Statistical Tests of Reliability

A commonly used measure (Kolbe and Burnett 1991) is the coefficient of
agreement between judges, ie., the number of agreements divided by the
number of coding decisions which should be calculated on a category by
category basis. This is because an ‘overall’ measure may hide poor
individual reliabilities.
Spearman Rank Order Correlation (or D2) is also a commonly used
measure for where a standard is known (as it may be when assessing
accuracy during the training of judges). Here coded behaviours are ranked.
Differences in ranking to the standard calculated are used to assess an
overall measure of agreement (Johnson and Bolstad 1973).
A similar statistic is that of Kendall’s Tau (Garson 2002). It is a measure
of association between ordinal variables computed as the excess of
concordant over discordant pairs. In other words, it tests for the difference
between the probability that order in the data is similar between judges and
that it is dissimilar. It has been used as a measure of agreement between
multiple judges and is operationalised for use within SPSS. It is, however,
difficult to judge the result where the direction of causation is unknown.
Cronbach’s (1951) coefficient alpha is also a measure for internal
consistency of the instrument. It considers, in addition to correlation between
judges, the number of categories and coded items.
Inherent problems arise with these measures as the probability of correctly
categorising data improves where there are small number of categories or
 An Overview of Content Analysis 487

some are rarely used. A measure developed to remove the impact of chance
agreement is Cohen’s (1960) kappa although it is difficult to calculate where
chance is unknown or cannot be approximated. Kappa may be tested for
significance (Bakeman and Gottman 1987) and is also a commonly used
measure in sequential analyses, where the sequence of categories is of
Delivered by Publishing Technology to: Chinese University of Hong Kong

interest. It remains a popular calculation among researchers for its ability to

test agreement and a number of analytical computer software programmes
have been developed as a result (Silver and Hittner 1998), eg., Chan (1987);
Cicchetti and Heavens (1990); Valiquette et al. (1994).
Similarly, Scott’s (1955) Pi corrects for the number of categories and the
frequency of use for each, while an index proposed by Perreault and Leigh
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

(1989) estimates the complete process of judgment-based research,

assessing variations due to the number of categories.
Guetzkow (1950) proposes reliability measures for both the coding
procedures and the separation of data into units. The probability is estimated
for the number of units on which the judges agree (based on two judges),
both correctly and incorrectly classified according to the standard. The
reliability of unitising is expressed by the difference between the judges as a
percentage of the sum of the number of units obtained by each judge.
Results enable the analyst to estimate the amount of data needed to be
check-coded to achieve the desired level of judge accuracy. These
estimates may be calculated for up to 4 judges (see Guetzkow 1950, for a full
review of the procedures).
Analysis of variance in reliability although important (for a full discussion,
see Krippendorff 1980) is, however, of little practical benefit where categories
are poorly defined. The use of sound operational definitions using rules and
descriptives for coding data improves this (Krippendorff 1980). As suggested
by Kolbe and Burnett (1991), “research will be improved more by focusing on
operational procedures that increase interjudge reliability” (p 249).
Operational definitions are fundamental to all research designs (refer eg.,
Deming 1993). Citing an example of “how many people live in Slough?” Gall
(1989) suggests definitions will be required for ‘people’, ‘live’ and ‘in Slough’
to reduce ambiguity. For example, does the word ‘people’ include children;
does ‘live’ mean only those who own homes; does ‘in Slough’ include the
outskirts? There are obviously any number of interpretations to these
questions. It is clear from this that operational definitions are essential.
These must be relevant to the research problem, indeed, a high level of
disagreement between judges may indicate validity problems in the research
design. The next section discusses validity of content analysis techniques.

Validity
Validity is the extent to which the research may be generalised to the
population. Two types of validity exist in relation to content analysis (Weber
1990) –

· internal validity – referring to the classification scheme or categories as

 488 Tracy G. Harwood and Tony Garry

being representative of the research hypotheses. Such ‘face’ validity is

the weakest form;
· external validity – there are various types of external validity which refer
to the extent the research results correspond with previous and future
findings. The most commonly used are –
Delivered by Publishing Technology to: Chinese University of Hong Kong

construct validity – relating to the underlying theoretical rationale of

the data measured. Construct validity increases where there is a
high predictive correlation in the construct of interest and other
related constructs from previous research (Kinnear and Taylor
1991);
hypothesis validity – pertaining to the consistency of the data with
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

expected relationships and theoretical argument;

predictive validity – is the verification of the inferences made from
content analysis by observing actual events, whether these are past,
present or future phenomena (Krippendorff 1980);
semantic validity – relating to judge reliability but specifically refers
to agreement on meanings or connotations of units being analysed
by persons familiar with contextual language. This is particularly
relevant, for example, in analysis of cross-cultural marketing where
important differences may be overlooked (Holsti et al. 1973).

Other elements of validity are universal to any research design, including

sampling and data related measures.

Operationalising Content Analyses

Content analysis is most commonly operationalised using manual recording

techniques (Suen and Ary 1989; Carson et al. 2001). Based on the research
design, having captured the data, the researcher categorises or codes it. As
already stated, this may be by temporal units, eg., all behaviours or events
occurring within a particular time frame or at specific points in time;
transactional eg., individual speaking turns in a focus group discussion;
psychological eg., expression of thought; by the system of categories devised
eg., everything directly relating to an object or event; or a hybrid of these
techniques. Having completed this, the researcher undertakes data analysis,
conducting systematic tests for reliability and validity as outlined above.
The process of data analysis lends itself to automation, reducing the
human cost and simultaneously improving reliability. Although, exclusive use
of computers for data analysis may result in lost opportunities to pre-test or
discuss the mechanism with judges. Automated analysis is particularly useful
where the amount of data is large although sample sizes have been found to
vary depending on availability of such data (Kolbe and Burnett 1991).
Packages range from those specific to a given context to free form, user
specified contexts. Examples are ACAMRIT for ‘automated content analysis
of marketing research interview transcripts’, designed by Lancaster
University in collaboration with a major market research organisation to
 An Overview of Content Analysis 489

analyse interview transcripts (Wilson and Rayson 1996); QSR NUDIST for
‘non-numerical unstructured data indexing searching and theorizing’,
supporting processes such as managing, exploring and searching the text of
documents; manage and explore ideas about the data; link ideas and
construct theories about the data; test theories about the data; and, generate
Delivered by Publishing Technology to: Chinese University of Hong Kong

reports including statistical summaries; BEST for ‘behavioural evaluation

strategy and taxonomy software’ which, according to its publishers (Sage)
facilitates real-time collection and analysis of observational category system
data.
More sophisticated packages are also appearing with the ability for data
warehousing, data mining, data visualisation and applications development.
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

An example is SAS Text Minor for ‘statistical analysis system’, launched mid-
2002 (www.sas.com). Text Miner can analyse the content of large text-
based files such as emails, call reports and other documents and assist in
uncovering patterns in data. It is generally used in the context of customer
relationship management (CRM) programs wherein managers receive and
process large amounts of text-based information; human resources where it
is used for reviewing and matching applicant curriculum vitaes; and, analysis
of open-ended survey responses to develop hypotheses and identify trends.
It enables full text ‘pre-processing’ by integrating unstructured textual data
with existing structured data such as demographic and psychographic
purchasing data.
As intimated in the preceding text, content analysis can be applied at
differing levels of complexity and so, therefore, this article would not be
complete without consideration of the levels of complexity required to capture
phenomena of interest. Clearly, as a method for observation, it is possible to
design a mechanism which captures both verbal and non-verbal behaviour.
Whilst verbal behaviour is relatively easy to capture with recording equipment
eg., tape, video, pencil and paper, non-verbal behaviour provides
researchers with a more complex set of considerations, particularly if the
phenomena of interest are only to be observed in real-life. Consider, for
example, the resource required to capture in sufficient detail the physical
movements and expressions of every participant in a sales negotiation as
unobtrusively as possible. Of course, modern technology can provide
solutions to this dilemma but there then remains the problems of deciphering
the meanings of non-verbal behaviour, suffice to say there is much
disagreement by social scientists in this area.

The Pluralistic Approach

There are instances where a combination of methodologies is not only

acceptable but indeed desirous, largely depending on whether the research
is exploratory or confirmatory in nature (eg., Olson et al. 1994). In order to
enhance the study of many interacting systems or a single interacting system
over time (Wampold 1986), sequential analysis may be used to produce
models of social interaction. Sackett (1987) suggests two interrelated
 490 Tracy G. Harwood and Tony Garry

purposes for doing this: to statistically characterise the links between

behaviours and to interpret the model in terms of statistically non-random
behaviours. In essence, important events recur and are combined at
different points in time to produce similar patterns (Bakeman and Dabbs
1976). Sequential analyses, however, depend on the approach taken to
Delivered by Publishing Technology to: Chinese University of Hong Kong

content analysis. There follows a brief review of sequential analysis including

some of the main considerations for researchers when taking this additional
step.

Sequential Analysis
Sequential data analysis has been used by a number of authors (eg.,
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

Olson et al. 1994; Putnam 1991; Hawks 1987, et al) and quoted by others
(Morley and Stephenson 1977) as a complementary methodology for content
analysis. Its various techniques have been described as "methods for
dealing with behaviour as it unfolds over time..." (Sanderson 1991, in Cuomo
1994, p 171), providing meaningful insights into the behaviour (Faroane and
Dorfman 1987; Lichtenberg and Heck 1986) of both small groups (Becker-
Beck and Fische 1984) and dyads (Moran et al. 1992; Weingart et al. 1990
and 1993).
The main premise of sequential analysis is the contingency that some
serial dependency exists between events or behaviours (Moran et al. 1992;
Suen and Ary 1989). Consequently, analytical techniques include –

· time series – determines the coherence of behaviours using regression

techniques, providing the basis for inferring temporal causality (Sackett
1987; Corter 1994);
· log linear modelling (see Fienberg 1989) – constructing and evaluating
contingency table data, used for assessing the degree of sequential
structure, rather than random, and changes over time or different
conditions (see also Gottman and Roy 1990);
· lag sequential analysis – exploratory and confirmatory technique for
finding which events follow others at specified times and frequencies
greater, or lower, than chance (Olson et al. 1994; Sackett 1979; Corter
1994; Gottman and Roy 1990);
· grammar – a syntactic approach used commonly in human-computer
interaction analyses (eg., Olson et al. 1994; Wilson and Rayson 1996);
· Markov chain – explores the relatedness in [categorical] behaviour
sequences by assessing significance of increasing lengths of events
until no further association is found (Sanderson and Fisher 1994;
Corter 1994; Gottman and Roy 1990; Lichtenberg and Heck 1986).

The process of sequential analysis is inherently complex and involves a

number of important steps (Wampold 1986; Bakeman and Dabbs 1976).
This includes –

· development or selection of an adequate coding system, suitably tested

 An Overview of Content Analysis 491

(refer content analysis above);

· interaction data coded as a sequence of behaviours;
· tests of reliability conducted on the coding;
· sequence data converted to a transition matrix;
· if applicable, analysis of between group/over-time questions.
Delivered by Publishing Technology to: Chinese University of Hong Kong

As intimated in the operationalising of content analysis, an appropriate format

for analysing the sequence of data is essential. Morley and Stephenson
(1977) stated, in choosing a coding mechanism, "if it is felt desirable to
analyse sequences of acts... it is much more convenient to have categories
which are mutually exclusive" (p 192). Moran et al. (1992), in proposing
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

coding for conflictual interactions between parents and children, suggest an

event based approach with global categorisation is “more straightforward
than even the most convenient time based coding procedures” (p 86).
Defining a mutually exclusive or ‘global’ coding mechanism does, therefore,
have distinct benefits – it prevents researchers from getting lost in the
considerable detail of behaviours; identification of individual behaviours may
help to clarify others; and, it aids statistical analysis.
Statistical procedures may be a simple calculation of straight probability or
conditional probability ie., probability of one event calculated in relation to
another event. A more useful statistic, however, is transitional probability
where conditional probability occurs with some displacement of time between
related events (Bakeman and Gottman 1986 and 1987). A widely used
measure is the z score which calculates the difference between simple
probability and transitional probability thus showing any deviations from
expected values.
The added complication with sequential analysis is that judges must not
only be in agreement on the frequencies of behaviours occurring but also at
the time point each behaviour occurred (if interval-based) and/or on the
actual stream of events (for both event- and time-based data) (Johnson and
Bolstad 1973). In practice, this suggests a very rigid methodology and one
which is unlikely to produce satisfactory results. This limitation is recognised
and overcome with the suggestion being that agreement between judges
should yield ‘similar’ sequential structures (Gottman 1980). The extent of
similarity will, of course, be related to the sample size used; the number of
events coded by judges; the practicalities of ‘check’ coding ie., one judge
codes a portion of data coded by another judge; the homogeneity of the
judges; and, the sequential structure across subjects.
Other criticisms arise from statistical tests used to assess the logical
independence of behaviours during an interaction. It has been argued (see
Wampold and Margolin 1982, for a full discussion) that these tests are
insufficient for a variety of reasons eg., over sensitivity to sample sizes, or
’run’ length ie., length of a particular sequence. Furthermore, the statistical
procedures proposed for measurement of a commonly used analytical
technique ie., lag sequential analysis, (see Sackett 1979, and Gottman 1979)
have themselves been questioned (Allison and Liker 1982; Faroane and
 492 Tracy G. Harwood and Tony Garry

Dorfman 1987). Several authors (see Sanderson and Fisher 1994, for a
summary) have raised doubts as to the appropriateness of measures,
particularly in exploratory studies, suggesting the independence of
behaviours lacks a ‘common-sense perspective’, as assumed by a null
hypothesis. Indeed, communication is a social interaction of two or more
Delivered by Publishing Technology to: Chinese University of Hong Kong

individuals and if behaviours are wholly independent perhaps ’interaction’ is

the wrong descriptor of communication!
Despite these criticisms, however, sequential analysis may be used to
enhance understanding where simple frequencies of behaviours derived from
content analysis hide significant patterns in the data. For example, Margolin
and Wampold (1981, in Wampold and Margolin 1982) found differences in
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

the affect of behaviours on subsequent behaviours in distressed and non-

distressed married couples but no difference in frequencies of negative
behaviours.

Discussion

The principal limitation of content analysis is that it assumes frequency is

allied to intensity (Clarke 1997). This is true only if the underlying premise is
frequency related which may not always be the case. For example, does a
very occasional reference to trust during an exchange in a collaborative
business partnership meeting mean that it is irrelevant? Any research design
must, therefore, consider whether there is a relationship with frequency of
occurrence. For example, Olekalns et al. (1996) argue that in negotiations,
outcomes are directly linked to frequency because of the strategies that have
been used to reach them. Criticism has also been levied for the under-
utilisation and reporting of reliability and validity measures by researchers
(Kolbe and Burnett 1991). Furthermore, selective analysis of data, rather
than exhaustive categorisation of all data, may result in undue bias.
Nonetheless, data has often been selected to support an area of particular
interest to the researcher (Clarke 1997).
Despite the criticisms levelled at it, content analysis offers researchers
several major benefits – it is unobtrusive, unstructured, context sensitive and
able to cope with large quantities of data (Krippendorff 1980). These are
particularly attractive to the researcher undertaking observation of
phenomena where the intent is not to influence the process. Indeed, more
obvious techniques, such as interviewing, questionnaire completion and
projective tests, may result in biased responses (Kinnear and Taylor 1991).
Although, of course, the technique can also be used to analyse response to
any discourse, including that generated by, say, interviews or open-ended
questions on questionnaires. It is particularly useful as part of multi-method
analyses where findings may be verified through triangulation. The relative
benefits and limitations of the approach are summarised in Table 1.
 An Overview of Content Analysis 493

Table 1 Summary of Benefits and Limitations of Content Analysis

Benefits Limitations
· flexibility of research design ie., · analyses the communication
Delivered by Publishing Technology to: Chinese University of Hong Kong

types of inferences (message) only

· supplements multi-method · findings may be questionable
analyses alone, therefore, verification
· wide variety of analytical using another method may be
application required
· may be qualitative and/or · underlying premise must be
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

quantitative frequency related

· may be automated – improves, · reliability – stability,
reliability, reduces cost/time reproducibility, accuracy of
· range of computer software judges
developed · validity – construct, hypothesis,
· copes with large quantities of predictive and semantic
data · less opportunity to pre-test,
· unobtrusive, unstructured, discuss mechanism with
context sensitive independent judges
· development of standards · undue bias if only part data is
applicable to specific research, analysed, possibly abstracting
eg., negotiations from context of communication
· lack of reliability and validity
measures reported, raising
questions of credibility

Conclusion

Review of the content analysis literature has highlighted the malleability of

the technique to analyse a wide range of research studies and on a variety of
data types. Most notably, the review has revealed the importance of
reliability and validity issues, which unless satisfied, may raise doubts as to
generalisability of findings from analyses. Indeed, it is most popular as a
complimentary technique rather than a stand-alone approach. A number of
authors have used, for example, sequential analysis, which overall improves
the quality of analysis.
Furthermore, Harris (1996) attempts to promote the use of standards
developed, which indicates robustness, although recognises the need to be
innovative to develop understanding in areas of specific research interest.
She states “a well developed standarized system does not necessarily
restrict the user; the system that is sufficiently complex may be adapted to
the needs of the particular researcher through cumulation or expansion of
existing categories. Repeated use of a limited number of coding systems
affords analysts the opportunity to make comparison among results of
 494 Tracy G. Harwood and Tony Garry

disparate research programs” (p 464). This is a partial reason why content

analytic mechanisms appear to have evolved at a slow pace. Further
explanation is highlighted by Johnson and Johnson (1997) who state reasons
as being the underdevelopment, until recently, of appropriate technology to
support social research which generates such a vast amount of data.
Delivered by Publishing Technology to: Chinese University of Hong Kong

Similarly, there are considerable complexities (physical and resource

constraint) of accessing naturalistic settings to obtain data in the initial
instance.
Content analytic mechanisms have a number of weaknesses, particularly
relating to objectivity, rigorous analysis of the category and inter-judge
reliabilities, and the technique considers only frequencies. The literature
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

review identifies that design of any content analytic coding mechanism needs
to consider whether data will be sequentially analysed and, therefore, the
limitations associated with the unitising of data. Careful consideration needs
to be given to the appropriateness of a design which affords a continuous
approach to data collection - this clearly gives the greatest richness to
analyses and improves potential accuracy in subsequent analyses to one
which has merely sampled data on an intermittent basis. Quite simply,
phenomena are not missed out with a continuous design. Of the two main
data unitising techniques which enable subsequent sequential analysis,
event-based offers more opportunity to analyse phenomena of an unknown
length whereas if time intervals are used, important phenomena may be
missed.

References

Advertising Standards Association, www.asa.org.uk

Allen, L. E. (1963), “Automation: substitute and supplement in legal
practice”, American Behavioral Scientist, 7, pp. 39-44
Allison, P. D. and Liker, J. K. (1982), “Analyzing sequential categorical data
on dyadic interaction: a comment on Gottman”, Psychological Bulletin, 91,
2, pp. 293-403
Bakeman, R. and Dabbs, J. M. Jnr (1976), “Social interaction observed:
some approaches to the analysis of behavior streams”, Personality and
Social Psychology Bulletin, 2, pp. 335-345
Bakeman, R. and Gottman, J. M. (1986), Observing Interaction: An
Introduction to Sequential Analysis, Cambridge: Cambridge University
Press, pp. 118-199
Bakeman, R. and Gottman, J. M. (1987), “Applying observational methods: a
systematic view, in Osofsky J D Ed, Handbook of Infant Development, 2nd
ed, New York: Wiley, pp. 818-854
Bales, R. F. (1950), Interaction process analysis: a method for the study of
small groups, London: University of Chicago Press Ltd
Becker-Beck, U. and Fisch, R. (1987), “Issues in the sequential analysis of
group interaction processes”, International Journal of Small Group
Research, 3, 2, pp. 198-212
 An Overview of Content Analysis 495

Berelson, B. (1952), Content Analysis in Communication Research, Glencoe,

The Free Press Publishers
Carson, D., Gilmore, A., Perry, C. and Gronhaug, K. (2001), Qualitative
marketing research, London: Sage
Cicchetti, D. V. and Heavens, R. Jnr (1990), “A computer program for
Delivered by Publishing Technology to: Chinese University of Hong Kong

determining the significance of the difference between pairs of

independently derived values of kappa or weighted kappa”, Educational
and Psychological Measurement, 41, pp. 189-193
Chan, T. S. C. (1987), “A DBASE III program that performs significance
testing for the kappa coefficient”, Behavior Research Methods,
Instruments and Computers 19, pp. 53-54
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

Clarke, G. (1997), Working paper: Content analysis – applications and

limitations, Leicester: De Montfort University
Cohen, J. (1960), “A coefficient of agreement for nominal scales”,
Educational and Psychological Measurement 20, 1, pp. 37-46
Collis, J. and Hussey, R. (2003), Business research, 2nd ed, Palgrave
MacMillan
Corter, J. E. (1994), “The statistical analysis of sequencies”, New Ideas in
Psychology, 12, 1, pp. 95-97
Cronbach, L. J. (1951), “Coefficient alpha and the internal structure of tests”,
Psychometrika, 16, pp. 297-334
Cuomo, D. L. (1994), “Understanding the applicability of sequential data
analysis techniques for analysing usability data”, Behaviour and
Information Technology, 13, 1 and 2, 171-182
Donohue, W. A., Diez, M. E. and Hamilton, M. (1984), “Coding naturalistic
negotiation interaction”, Human Communication Research, 10 (3), Spring,
pp. 403-425
Faroane, S. V. and Dorfman, D. D. (1987), “Lag Sequential Analysis: Robust
Statistical Methods”, Psychological Bulletin, 101, 2, pp. 312-323
Fienberg, S. E. (1989), The Analysis of Cross-Classified Categorical Data
(2nd edit), London: MIT Press
Garson (2002), Cronbach’s Alpha, website http://www2.chass.ncsu.edu/
garson/pa765/standard.htm, accessed 11 August
Garson (2002), Kendall’s tau-b, website http://www2.chass.ncsu.edu/
garson/pa765/assocordinal.htm, accessed 11 August
Geutzkow, H. (1950), “Unitizing and categorising problems in coding
qualitative data”, Journal of Clinical Psychology, 6, pp. 47-58
Gerbner, G., Gross, L., Signorielli, N., Morgan, M. and Jackson-Beeck, M.
(1979), Violence profile no 10: trends in network television drama and
view conceptions of social reality 1967-78, Annenberg School of
Communications, University of Pennsylvania
Gottman, J. M. (1980), “Analyzing for sequential connection and assessing
interobserver reliability for the sequential analysis of observational data”,
Behavioral Assessment, 2, pp. 361-368
Gottman, J. M. (1979), “Detecting cyclicity in social interaction”,
Psychological Bulletin, 86, 338-348
 496 Tracy G. Harwood and Tony Garry

Gottman, J. M. and Roy A. K. (1990), Sequential analysis: a guide for

behavioral researchers, Cambridge: Cambridge University Press
Harris, K. L. (1996), “Content analysis in negotiation research: a review and
guide”, Behavior Research Methods, Instruments and Computer, 28, 3,
pp. 458-467
Delivered by Publishing Technology to: Chinese University of Hong Kong

Hawks, I. K. (1987), “Facilitativeness in small groups: a process-oriented

study using lag sequential analysis”, Psychological reports, 61, pp. 955-
962
Hays, D. C. (1969), “Linguistic foundations for theory of content analysis”, in
Gerbner et al., eds (1979), The analysis of communications content, John
Wiley, pp. 57-67
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

Holsti, O. R. (1969) in Krippendorff (1980), Content analysis: an introduction

to its methodology, London: Sage
Holsti, O. R., Brody, R. A. and North, R. C. (1965), “Measuring affect and
action in interntional reaction models: empirical materials from the 1962
Cuban crisis”, Peace Research Society Papers, 2, pp. 170-190
Janis, I. L. (1965) in Krippendorff K (1980), Content analysis: an introduction
to its methodology, London: Sage
Johnson, S. M. and Bolstad, O. D. (1973), “Methodological issues in
naturalistic observation: some problems and solutions for field research”,
in Hamerlynck L. A., Handy, L. C. and Mash, E. J (1973), Behavior
change: methodology concepts and practice, US: Research Press
Company, pp. 7-67
Johnson, D. W. and Johnson, F. P. (1997), Joining together: group theory
and group skills, 6th ed, London: Allyn and Bacon
Kassarjian, H. H. (1977), “Content Analysis in Consumer Research”, Journal
of Consumer Research, 4, pp. 8-18
Kinnear, T. C. and Taylor, J. R. (1991), Marketing Research: An Applied
Approach, 4th ed.. London: McGraw-Hill Inc
Kolbe, R. H. and Burnett, M. S. (1991), “Content analysis research: an
examination of applications with directives for improvement research
reliability and objectivity”, Journal of Consumer Research, 18 (Sept), pp.
243-250
Krippendorff, K. (1980), Content Analysis: An Introduction to its Methodology,
London: Sage
Lichtenberg, J. W. and Heck, E. J. (1986), “Analysis of sequence and pattern
in process research”, Journal of Counseling Psychology, 33, 170-181
Mariampolski, H. (2001), Qualitative market research, London: Sage
McClelland, D. C. (1958), “The use of measures of human motivation in the
study of society”, in Atkinson, J. W. (ed), Motives in fantasy, action and
society, Princeton, NJ: D Van Nostrand, pp. 518-552
Moran, G., Dumas, J. E. and Symons, D. K. (1992), “Approaches to
sequential analysis and the description of contingency in behavioral
interaction”, Behavioral Assessment, 14, pp. 65-92
Morley, I. and Stephenson, G. (1977), The Social Psychology of Bargaining,
London: George Allen and Unwin Limited, p 26
 An Overview of Content Analysis 497

Morton, A. Q. (1963), “A computer challenges the church”, The Observer, 3

Nov, p 21
Neuendorf, K. A. (2002), The content analysis guidebook, London: Sage
North, R. C., Holsti, O. R., Zaninovich, M. G. and Zinnes, D. A. (1963),
Content analysis: a handbook with applications for the study of
Delivered by Publishing Technology to: Chinese University of Hong Kong

international crises, Evanston: Northwestern University Press

O’Connor, J. (1996), “Thinking past the obvious: what is a system?” Internet:
http://www.radix.net/~crbnblu/assoc/oconnor/chapt1.htm, accessed 14
Jan 1997
Olekalns, M., Smith, P. L. and Walsh, T. (1996), “The process of negotiating:
strategy and timing as predictors of outcomes”, Organizational Behavior
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

and Human Decision Processes, 68, 1, pp. 68-77

Olson, G. M., Herbsleb, J. D. and Rueter, H. H. (1994), “Characterizing the
Sequential Structure of Interactive Behaviors Through Statistical and
Grammatical Techniques”, Human-Computer Interaction, 9, 427-472
Perrault, W. and Leigh, K. (1989), “Reliability of nominal data based on
qualitative judgements”, Journal of Marketing Research, 26 (May), pp.
135-148
Pool, I de S. (1951), Symbols of internationalism, Stanford, CA: Stanford
University Press, Series 3, C, Aug, pp. 1-63
Putnam, L. L. (1981), “Procedural Messages and Small Group Work
Climates: A Lag Sequential Analysis”, in Burgoon M (ed), Communication
Yearbook 5, Transaction Books, pp. 331-350
Robson, C. (1993), Real world research, Oxford: Blackwell
Sackett, G. P. (1987), “Analysis of Sequential Social Interaction Data: Some
Issues, Recent Developments, and a Causal Inference Model”, in
Osofsky, J. D. (ed), Handbook of Infant Development, New York: Wiley,
pp. 855-878
Sanderson, P. M. and Fisher, C. (1994), “Exploratory Sequential Data
Analysis: Foundations”, Human-Computer Interaction, 9, pp. 251-317
SAS Text Miner, http://www.sas.com, accessed 3 August 2002
Scott, W. A. (1955), “Reliability of content analysis: the case of nominal scale
coding”, Public Opinion Quarterly 19, 3, pp. 321-325
Silver, N. C. and Hittner, J. B. (1998), A guidebook of statistical software for
the social and behavioral sciences, London: Allyn and Bacon
Speed, G. (1893), “Do newspapers now give the news? ” Forum, 15, pp.
709-711
Suen, H. K. and Ary, D. (1989), Analyzing quantitative behavioral
observation data, London: Lawrence Erlbaum Associates
Symons, D. K. (1992), “Estimating the Expected Probability of an Event in
Continuously Recorded Social Behavior”, Psychological Bulletin, 111, 1,
pp. 185-187
Tesch, R. (1990), Qualitative research: analysis types and software tools,
New York: The Falmer Press
Valiquette, C. A. M., Lesage, A. D., Cyr, M. and Toupin, J. (1994),
“Computing Cohen’s kappa coefficients using SPSS MATRIX”, Behavior
 498 Tracy G. Harwood and Tony Garry

Research Methods, Instruments and Computers, 26, pp. 60-61

Wampold, B. E. (1986), “State of the Art in Sequential Analysis: Comment on
Lichtenberg and Heck”, Journal of Counseling Psychology, 33, 2, pp. 182-
185
Wampold, B. E. and Margolin, G. (1982), “Nonparametric Strategies to Test
Delivered by Publishing Technology to: Chinese University of Hong Kong

the Independence of Behavioral States in Sequential Data”, Psychological

Bulletin, 92, 3, pp. 755-765
Weiner, R. S. P. (1971) in Morley, I E and Stephenson, G M (1977), The
social psychology of bargaining, London: George Allen and Unwin Ltd
Weingart, L. R., Bennett, R. J. and Brett, J. M. (1993), “The impact of
consideration of issues and motivational orientation of group negotiation
IP: 119.92.61.209 On: Sat, 10 Oct 2015 22:04:28

process and outcome”, Journal of Applied Psychology, 78, 3, pp. 504-517

Weingart, L. R., Thompson, L. L., Bazerman, M. H. and Carroll, J. S. (1990),
“Tactical behavior and negotiation outcomes”, The International Journal of
Conflict Management, 1, 1 (Jan), pp. 7-31
Wilson, A. and Rayson, P. (1996), “The automatic content analysis of spoken
discourse: a report on work-in-progress”, internet
http://www.comp.lancs.ac.uk/.../papers/war93.txt, accessed 13 Dec

About the Authors

Tracy Harwood is a senior lecturer and teacher fellow within the Department
of Marketing at Leicester Business School, De Montfort University. Her
research specialism is in negotiation processes of strategic business
partners, on which she has worked closely with a number of major
organisations over the past 5 years and is an assistant examiner for the
Chartered Institute of Purchase and Supply. Her publications include
conference proceedings, academic journal articles, textbooks and trade
articles.

Tony Garry is a senior lecturer within the Department of Marketing at

Leicester Business School, De Montfort University. His primary research
interests are the role of marketing and service encounters within a
professional services context. This has become the focus for his current PhD
research and is the subject of a number of conference publications