Conceptualizing Analyses of Ecological Momentary Assessment Data
Conceptualizing Analyses of Ecological Momentary Assessment Data
Review
Abstract
Ecological momentary assessment (EMA) methods, which involve collection of real-time data in subjects’ real-world environ-
ments, are particularly well suited to studying tobacco use. Analyzing EMA datasets can be challenging, as the datasets include
a large and varied number of observations per subject and are relatively unstructured. This paper suggests that time is typically
a key organizing principle in EMA data and that conceptualizing the data as a timeline of events, behaviors, and experiences
can help define analytic approaches. EMA datasets lend themselves to answering a diverse array of research questions, and
the research question must drive how data are arranged for analysis and the kinds of statistical models that are applied. This is
illustrated with brief examples of diverse analyses applied to answer different questions from an EMA study of tobacco use and
relapse.
Introduction text. Indeed, this paper argues that no such guide is possible
because the approach to analysis needs to be dictated by the
This paper discusses the use of ecological momentary research question, not by fixed rules. Rather, this paper pre-
assessment (EMA; Shiffman, Stone, & Hufford, 2008; Stone sents examples of EMA analyses of tobacco use, as a way of
& Shiffman, 1994)—collection of real-time data in subjects’ demonstrating the flexibility of EMA data, and the diversity of
real-world environments—to study tobacco use. It begins by approaches to analyzing it to address a diverse set of research
briefly introducing basic concepts of EMA and explaining questions. This paper argues that the framing of a clear research
why these methods are particularly well suited for the study question, and the organization of the EMA data to address that
of tobacco use. The paper is not intended to be a comprehen- question, and not just the selection of a statistical model, is the
sive exposition on EMA methods, nor a review of EMA studies EMA investigator’s primary challenge. Finally, although this
of tobacco. This paper’s purpose is to fill a gap in the EMA article focuses exclusively on analyses of tobacco use, the con-
literature by illustrating by example some of the ways EMA cepts articulated here are broadly applicable to a broad range of
data can be structured and analyzed to answer theoretically behaviors and research questions that EMA data can address.
and clinically relevant questions about tobacco use. Reviews of
EMA methods, including practical and technological consid-
erations, are available (Hufford, 2007; Shiffman et al., 2008; What is EMA?
Stone, Shiffman, Atienza, & Nebeling, 2007), and their appli-
cation to tobacco and substance abuse has also been reviewed EMA methods are defined by repeated collection of real-time
(Shiffman, 2009; Shiffman, in press), as have the relevant sta- data in subjects’ real-world environments. An example of
tistical methods (Li, Root, & Shiffman, 2006; Raudenbush & EMA data collection on tobacco use would be the use of diaries
Bryk, 2002; Schwartz & Stone, 2007; Singer & Willett, 2003; to capture data on the circumstances in which smokers light up.
Snijders & Bosker, 2011; Zeger, Liang, & Albert, 1988). Yet, EMA’s focus on subjects’ natural environment derives from an
analysis of EMA data continues to be a daunting task for many interest in ecological validity, in studying how tobacco use (or
investigators, seemingly deterring adoption of EMA methods, any other endpoint of interest) varies under the range of real-
and questions about how to conceptualize and structure analy- istic circumstances that subjects encounter in their daily lives.
ses of EMA data continue to arise for investigators and review- EMA’s focus on momentary assessment—collecting data about
ers, who sometimes wonder how such voluminous, complex, what is going on right at the moment or over the very recent
and unstructured data can be analyzed meaningfully. past—derives from two sources. First is the concern that retro-
While it focuses on data analysis, the paper is not intended spective recall is subject to serious biases (Bradburn, Rips, &
as a step-by-step guide to EMA analysis, nor as a statistical Shevell, 1987; Hufford & Shiffman, 2002; Shiffman, Hufford,
doi:10.1093/ntr/ntt195
Advance Access publication 9 December 2013
© The Author 2013. Published by Oxford University Press on behalf of the Society for Research on Nicotine and Tobacco.
All rights reserved. For permissions, please e-mail: [email protected].
S76
Nicotine & Tobacco Research, Volume 16, Supplement 2 (May 2014)
et al., 1997) that can impair valid scientific inference; asking spacing. To prompt the subject for assessment at scheduled
subjects about what is happening here-and-now has advan- times, EMA studies typically use a device that can “beep” at
tages over asking what happened there-and-then. Second is the the appropriate time (e.g., mobile phones, palmtop computers,
interest in understanding how events, behaviors, and experi- pagers), and these sorts of assessments are sometimes referred
ences play out over time and context, which requires data with to as “signal-contingent” assessments (Shiffman, 2007;
degree of temporal resolution appropriate to the dynamics of Wheeler & Reis, 1991).
the behavior of interest. The actual degree of temporal resolu- A final way to schedule assessments is to tie them to par-
tion needed should be determined by theory or data about the ticular milestone times, rather than administer them throughout
timeframe for relevant processes (Collins, 2006); some pro- the day. Examples include having subjects complete a diary
cesses (e.g., impulsive responses to emotional change) might “every evening,” while leaving the exact timing to the subject’s
be very rapid, while others (e.g., exhaustion of motivation to discretion, or having subjects complete diary “when first wak-
quit) might progress more slowly. An important consideration ing up.” Such assessments should not typically be treated as
is that phenomena that might initially be thought to be rela- representing the subject’s overall experience but are meant to
tively stable often turn out to be quite volatile, when measured collect somewhat retrospective data on experience over larger
with greater temporal resolution (e.g., changes in self-efficacy intervals (e.g., how stressful the day was) or to capture particu-
[SE] documented in Gwaltney, Shiffman, & Sayette, 2005). lar experiences that are time bound (e.g., sleep quality).
Temporal resolution is limited by the density of data collec- It can be useful to combine these different modes of
tion, so these issues need to be considered when first designing assessment scheduling. A very common EMA design com-
an EMA study. bines event-contingent assessments of smoking episodes with
EMA typically involves numerous assessments over time, signal-contingent assessments of nonsmoking moments. For
yielding what has sometimes been called intensive longitu- investigators interested in characterizing smoking events, the
dinal data (Collins, 2006; Walls & Schafer, 2006). The need nonsmoking data add value in at least two ways. First, without
for many assessments follows from several considerations. a contrast to nonsmoking moments, the data collected about
Assessments focus on particular moments, but the individual smoking events are hard to interpret and can be misleading
moments themselves are seldom of interest to the investigator. (Paty, Kassel, & Shiffman, 1992). For example, someone who
Instead, the investigator may conceptualize them collectively reports being anxious when smoking may not be prompted to
as samples of the subject’s experience and require a number of smoke by anxiety—the individual may just be anxious all the
assessments to characterize the overall experience. The array- time. The nonsmoking assessments function like the controls
ing of assessments over time also allows investigators to trace in a case–control design, enabling valid inference by contrast-
the trajectory of experience over time and context, demanding ing cases and controls. Second, randomly sampled data can
many assessments. Indeed, consideration of the role of time also be used to characterize the subject’s general experience
can help structure analyses of EMA data, as will be illustrated and the trajectory of experience over time. A previous paper
later in this paper. discussed these assessment types and design considerations in
more detail (Shiffman, 2007).
EMA Designs
A useful way to conceptualize collection of EMA data is to EMA in Tobacco Research
consider how assessments are scheduled. Assessments may be
linked to events, such as occasions when the subject is about EMA methods are particularly well suited to studying tobacco
to smoke a cigarette. Such “event-contingent” assessments use. Tobacco use itself is a discrete event that lends itself to
(Shiffman, 2007; Wheeler & Reis, 1991) are typically triggered event-based assessment. Moreover, many theoretical accounts
by the subject reporting that they are about to smoke or have of tobacco use reference proximal influences that drive
smoked; thus, these assessments are sometimes referred to as tobacco use. For example, theory suggests that smoking may
subject-initiated. But they need not actually be initiated by the be triggered by symptoms attending a drop in nicotine levels
subject. Researchers are developing systems that can automati- (Benowitz, 2008; Stolerman & Jarvis, 1995), by stimuli pre-
cally detect smoking (Ali et al., 2012), and an event might be viously associated with smoking (Marlatt & Gordon, 1985;
defined by some other objective measure, such as entering a Niaura et al., 1988), or by a need to mitigate acute negative
particular location, detected by Global Positioning System affect (Kassel, Stroud, & Paronis, 2003). Similarly, during ces-
(Kirchner, Cantrell, et al., 2013). Smoking event data inform sation, theory suggests that lapses (limited episodes of use)
the investigator about how many smoking events occurred and may be triggered by craving (West & Schneider, 1987), by
when they occurred, allowing temporal patterns to be analyzed. negative affect (Sinha, 2001), or by conditioned cues (Marlatt
If the assessments also collect data about the person’s situation & Gordon, 1985; Niaura et al., 1988). EMA is ideally suited
at the time of smoking, they can document such circumstances, to assessing such microprocesses, examining proximal influ-
for example, characterizing smokers’ locations or moods when ences on such episodes. Accordingly, it is not surprising that
they smoke. EMA methods are being widely applied to studies of tobacco
Another approach to scheduling EMA data is to schedule use, particularly smoking. EMA methods have been used to
assessments for certain times, most often at random; randomly study antecedents and consequences of smoking in adolescents
time sampling subjects’ state provides a representative and (Mermelstein, Hedeker, & Wesintein, 2010) and adults (Carter
unbiased estimate of subjects’ typical state. Other variations et al., 2010; Warthen & Tiffany, 2009), in smokers with post-
include scheduling assessments at random, but within blocks traumatic stress disorder and other disorders (Beckham et al.,
of time of day, assuring that all blocks of time are sampled, 2008; Epstein, Marrone, Heishman, Schmittner, & Preston,
or scheduling assessments at regular intervals, ensuring equal 2010; Piper, Cook, Schlam, Jorenby, & Baker, 2011), during
S77
Conceptualizing analyses of EMA data
ad libitum smoking (Shapiro, Jamner, Davydov, & James, smoking episodes by making event entries for 2 weeks before
2002) and smoking cessation (Bolt, Piper, Theobald, & Baker, a target quit date. Each cigarette “event” is recorded. A ran-
2012; Cooney et al., 2007; Minami, McCarthy, Jorenby, & dom subsample of these smoking events is selected for a more
Baker, 2011), and in relation to a range of variables ranging extensive assessment of the surrounding circumstances. This
from alcohol use (Holt, Litt, & Cooney, 2012; Piasecki, Wood, more complete assessment was thought to be too burdensome
Shiffman, Sher, & Heath, 2012; Witkiewitz et al., 2012) to to implement for every cigarette smoked. During this time,
worried thoughts about smoking (Magnan, Köblitz, McCaul, subjects were also prompted at random (nonsmoking) times
& Dillard, 2013), and exposure to media messages (Shadel, to complete parallel signal-contingent assessments. After the
Martino, Setodji, & Scharf, 2012) or proximity to tobacco target quit date, subjects monitored lapses (limited episodes of
sales outlets (Kirchner, Cantrell, et al., 2013). EMA methods smoking) and episodes of temptation to smoke for 4–6 weeks,
have also been fruitfully applied to study the effects of treat- while continuing to receive randomly scheduled signal-contin-
ment, both to define treatment outcome (Shiffman et al., 2000; gent assessment prompts. Subjects were considered to have quit
Shiffman et al., 2006) and to examine the processes that medi- when the EMA data indicated they had gone without smoking
ate treatment effects (Ferguson, Shiffman, & Gwaltney, 2006; for 24 hr and were considered relapsed when they smoked ≥5
McCarthy et al., 2008; Piper et al., 2008). cigarettes on 3 consecutive days. One study (Shiffman, Paty,
et al., 1996) enrolled 304 smokers who were all provided with
cognitive behavioral treatment; a second enrolled 412 smokers
and subsequently randomized 324 to high-dose patch or pla-
Structuring Analyses of cebo (Shiffman et al., 2006).
EMA Data The first study illustrates the challenges faced by EMA
investigators in determining how to analyze their data. The
Conceptualizing EMA Analysis final database contained 191,841 records. Of these, 74,270
One of the aspects of EMA that investigators new to the meth- were momentary assessments (45,959 signal-contingent
ods find most intimidating is data analysis. Not only do EMA assessments, and 28,311 event assessments: 22,016 smoking
data include multiple observations per person, but the number occasions, 1,729 lapses, and 4,566 temptations); each assess-
of observations, and their timing, varies between subjects. Such ment comprised 45–73 individual variables. An additional
data are not amenable to analysis by simple methods requiring 22,825 records were associated with two types of daily assess-
independent observations. However, a variety of methods for ments (morning and evening). The remaining 94,746 records
handling such data are now available and accessible in several represent recorded events without any self-report assessment
popular statistical packages, the most common of these being (71,895 smoking events not selected for assessments and
hierarchical or multilevel regression models (Rabe-Hesketh 22,851 records of bedtime and waking), for which only the
& Skrondal, 2008; Raudenbush, Bryk, Cheong, & Congdon, timing is known. Figure 1 shows graphically the records for
2010; Singer, 1998; Snijders & Bosker, 2011; see Schwartz & five subjects and illustrates the relatively unstructured nature
Stone, 2007). of the data. As there were 304 subjects in the study, the vol-
Nevertheless, many investigators continue to find analysis ume and variety of data seen in Figure 1 must be multiplied
of EMA data daunting. Given the availability of accessible sta- 60-fold. Such EMA datasets are rich, but also large, complex,
tistical methods and software, this is not so much a statistical and potentially overwhelming.
challenge as a conceptual one of formulating appropriate anal- The richness of EMA data is an asset: A single dataset can
yses for EMA data. EMA data are often relatively unstructured, lend itself to answering many different questions. But the rich-
consisting of a detailed temporal record of subject experiences, ness comes at a price. The data structure does not inherently
from which investigators aim to extract meaningful relation- dictate the nature of the analysis, and many different analy-
ships. In these datasets, subjects’ behaviors and experiences ses are possible—the particular research question dictates the
often determine which observations or portions of the data are analysis, what part of the data are of interest, and how the data
of interest, for example, when stress peaks or when a smoking needs to be structured for analysis. Before one can ask which
lapse occurs. This makes selection and framing of the data for statistical procedure might be appropriate, the investigator has
analysis more challenging, but crucial, as illustrated below. to frame the research question and appropriately structure the
data. This involves selecting the appropriate part of the data
(as will be seen, many questions are best addressed by a subset
Two Illustrative EMA Studies of Smoking and Relapse
of observations), and structuring it in a way that can answer
Illustrating the relatively unstructured nature of EMA data are the question, and that fits an appropriate statistical model. The
studies conducted by our research group to understand ante- task of analyzing EMA data requires that investigators and ana-
cedents of ad libitum smoking and the processes that lead to lysts think very explicitly and very hard about what question
relapse in smoking cessation. The two studies used illustra- they are asking of the data and how the data might be selected,
tively below were studies of smoking and relapse process in structured, and analyzed to answer that question.
which smokers were observed while smoking and then fol- The examples that follow, drawn from the studies described
lowed through a quit attempt (Chandra, Shiffman, Scharf, above, are intended to illustrate a range of analytic approaches
Dang, & Shadel, 2007; Ferguson et al., 2006; Shiffman et al., to EMA data on smoking, organized according to how they
2002; Shiffman, Hickcox, et al., 1996; Shiffman, Paty, Gnys, incorporate time. Each example briefly states the research
Kassel, & Hickcox, 1996; Shiffman et al., 2006). The studies question, explains how data were selected and structured
combined event-contingent and signal-contingent assessments (including the unit of aggregation or analysis), how the data
throughout the day, as well as daily assessments at waking and were analyzed, and what the results showed. As Figure 1 illus-
in the evening. Subjects used palmtop computers to monitor trates, time is often the key organizing principle for EMA data;
S78
Nicotine & Tobacco Research, Volume 16, Supplement 2 (May 2014)
Figure 1. The figure shows the data stream for five subjects in the study, indicated as horizontal blocks A–E. Each point repre-
sents an observation of a particular type (see legend). The x-axis is continuous time, marked by days in the study. The end of Day
17, indicated by the vertical dotted line, was designated as the Target Quit, the time after which subjects were expected to abstain.
The different markers indicate the different kinds of records, as identified in the legend. Thus, the graph shows the timeline of events
for each subject. The bottom-most subject (A) did not quit, as indicated by the presence of cigarette records in the dates following
the quit date. Subject B quit, as indicated by the absence of cigarette records after the quit date, but then frequently lapsed, as indi-
cated by many lapse records, and eventually relapsed, as indicated by the return of smoking records around Day 28. Subject C quit,
and lapsed, but did not relapse while in the study. Subject D did not lapse until Day 23 but had frequent temptations in the preceding
days. Subject E quit and did not report any lapses, though some temptations were reported, at progressively sparser frequency. Daily
milestones of waking up and going to bed and daily waking and evening assessments are not shown.
EMA data are anchored in time and create a timeline of sub- when they quit. Since this was very much a subject-level ques-
ject experiences and events over time, allowing investigators to tion, data were collapsed over time, averaging craving ratings
explore behavioral process over time. for each subject from all signal-contingent assessments they
completed in the first 2 days after quitting in order to derive a
Illustrative Analyses of EMA Data single craving score. Using simple Pearson correlation across
subjects, Drive correlated with postquit craving significantly,
This section presents 11 different analyses of the EMA data but modestly, at r = .20 (Shiffman et al., 2004).
from the sample studies, each illustrating a different selection As this was a between-subjects analysis, it seems not to
and organization of the data and a different approach to statisti- incorporate any effect of time. Yet, the analysis is actually
cal modeling, in order to answer a different kind of research quite time dependent, in that the period over which craving
question. The examples are distinguished by how they concep- was assessed was defined by time: the 2 days after the sub-
tualize the role of time and represent it in the analysis. The ject had first abstained for 24 hr (note that time here is not
illustrative examples are summarized in Table 1. anchored in the calendar but is anchored in a subject-specific
event observed via EMA). The time sampling via signal-con-
Collapsing Time: Between-Subjects Analyses tingent assessments was also crucial in assuring representative
sampling of craving during those 2 days, and the aggregation
Time need not always figure explicitly in EMA analyses. Some
of multiple assessments increases the reliability of the craving
research questions may be about differences between individu-
measure. Thus, the time anchoring of EMA assessments played
als, with no reference to change over time. An example comes
a role even in this “timeless” analysis.
from assessment of nicotine dependence. The Drive subscale of
the Nicotine Dependence Syndrome Scale (Shiffman, Waters,
Collapsing Time Into Events: Contrasting Events
& Hickcox, 2004) purports to assess the tendency to experience
craving when smokers abstains from smoking, a core construct The prior example showed how time-bound observations could
in dependence. The research question was whether smok- usefully be collapsed to contrast subjects. Observations may
ers scoring higher on Drive experience more intense craving also be collapsed over time when contrasting events or contexts
S79
Conceptualizing analyses of EMA data
within persons. This example focuses on the role played by 2007) do not focus on single smoking events, but rather on the
alcohol consumption in triggering smoking (Shiffman et al., smoking “intensity” or rate—that is, cigarettes per unit time—
2002). The research question was whether smoking was more as a function of covariates. The analysis also takes into account
likely to occur when the individual had been drinking. To the sampling scheme for EMA data. One question addressed
address this, the analysis used a case–control design (actually, using point process analysis of the data on ad libitum smoking,
case-crossover), contrasting the proportion of occasions that prior to quitting, concerned the relationship between smoking
each subject was drinking when they were smoking (cases, rate and restlessness. Some evidence suggests that restlessness
from event assessments) versus when not smoking (con- may be a particularly good and specific indicator of nicotine
trols, from signal-contingent assessments). The analysis used withdrawal (Shiffman, Paty, et al., 1996), in contrast to broader
Generalized Estimating Equations (GEEs; Zeger et al., 1988), measures of negative affect, which are heavily affected by
a regression-based method that accounts for nesting of obser- other influences. Accordingly, it was expected that smoking
vations within subjects, to contrast the probability of drinking rate would show specific increases as restlessness increased.
prior to smoking versus nonsmoking episodes. Smoking was A point process analysis using individual observations of
associated drinking: Subjects reported drinking on 11% of the event-contingent smoking and signal-contingent nonsmok-
occasions when they were smoking versus 6% of nonsmok- ing data (i.e., with no aggregation) confirmed the hypothesis:
ing occasions (OR = 2.10, 95% CI: 1.80–2.45). This analysis Smoking rate increased by 16% for every 1-point increase in
focuses on differences between events rather than between sub- the 4-point restlessness rating (Shiffman & Rathbun, 2011). In
jects: It contrasts, across persons, the likelihood of drinking contrast, a more global measure of negative affect was unre-
over multiple smoking and nonsmoking occasions. Time is not lated to the smoking rate. Considering time as part of a calcu-
explicitly referenced in the analysis, yet the entire analysis is lation of smoking rate allows for more powerful analyses of
based on contrasting data collected at two different “times”— influences on smoking.
when smoking and when not smoking.
Sequences of Events Over Time
Cigarette Consumption per Unit Time
Most of the prior analyses do not take into account the par-
An alternative approach to analyzing the relationship between ticular sequence in which recorded events or assessments took
situational variables and smoking more explicitly incorporates place. But sometimes the sequencing can be very important
time. Point process analyses (Rathbun, Shiffman, & Gwaltney, and can inform our understanding of process. In this example,
S80
Nicotine & Tobacco Research, Volume 16, Supplement 2 (May 2014)
the research question was whether initial lapses to smoking eight time blocks of equal length. These temporal profiles were
occurred in the context of emotional distress. The very first then submitted to cluster analysis to identify common circa-
lapse is important because it breaks the momentum of absti- dian patterns. Figure 2 shows the rate of cigarette consumption
nence and often leads to relapse (Kenford et al., 1994). In the (relative to each person’s daily average), by time blocks, for
study, subjects who lapsed recorded their emotional state at the the different clusters. Most of the groups in this heavy-smoking
start of the episode, and their ratings indicated emotional dis- sample smoked more in the morning; “Flatline” smokers were
tress (Shiffman, Paty, et al., 1996). However, such event data the exception. As hypothesized, Flatliners were lighter and less
are interpretable only by contrast to data from a control obser- dependent smokers. Moreover, using subject-level survival
vation, which this EMA design provides through the randomly analysis, the analysis showed that the cluster groups differed in
sampled, signal-contingent (nonlapse) assessments. Unlike in time to lapse (with or without a nicotine patch), confirming that
the prior example, the selection of control observations must be circadian patterns of consumption are relevant to cessation out-
sensitive to temporal sequence because the subject’s emotional come. This analysis focused on between-subject differences,
state after the lapse could be influenced by the lapse itself, con- but those differences were defined by how subjects’ behavior
founding any analysis based on postlapse records. varied by time of day. This analysis was also notable in that
For this analysis, then, affect preceding the first lapse was the EMA data consisted solely of event records, without any
compared to the affect in a single signal-contingent assess- subjective self-report data.
ment that preceded the lapse (see Shiffman, Paty, et al., 1996
for information on how the assessment was selected). Since Time as Defined by an Event
each subject contributed only one observation of each type, the
analysis used a simple dependent t test. Comparison of affect For some hypotheses, the passage of time itself is not of inter-
ratings confirmed that lapse episodes were associated with ele- est, but rather the occurrence of key events, which define
vated distress, compared to the preceding random assessment time simply as before the event or after. Marlatt and Gordon’s
(T-score [M = 50, SD = 10] of 57.8 vs. 50.0, p < .0001). Besides (1985) theory of relapse emphasizes smokers’ psychologi-
avoiding confounding, the fact that the less-distressing ran- cal reactions to lapses as the determining factor in subjects’
dom assessment came first also allowed an important further subsequent trajectory and outcome. They hypothesized that a
inference. Because withdrawal intensity decreases over time lapse causes smokers’ SE to decline, putting them on a course
for most subjects (though see Piasecki et al., 2000; Piasecki, towards further smoking, whereas successfully avoiding smok-
Jorenby, Smith, Fiore, & Baker, 2003), the increase in distress ing when tempted would cause SE to increase, putting them
seen in the later (lapse) observation could not readily be attrib- on a trajectory towards continued abstinence. To address this
uted to nicotine withdrawal. Thus, the example illustrates the hypothesis, the analysis (Shiffman, Hickcox, et al., 1997) iden-
importance of establishing temporal ordering of events and tified each smoker’s first lapse episode, assessment of which
experiences. It also illustrates an exception to the earlier state- included a rating of their SE after the lapse. Each smoker’s
ment that EMA researchers are seldom interested in any one
moment or event. The first lapse is considered to be an impor-
tant unique milestone that dramatically changes the smoker’s
trajectory towards abstinence or relapse. Thus, EMA research-
ers may not only be interested in events and experiences over
time but also in experience at particular unique moments in
time.
S81
Conceptualizing analyses of EMA data
prelapse SE was estimated by looking back at ratings made not have abstinence data going further back). Figure 3 shows
at a single randomly scheduled, signal-based assessment—the that there was no significant increase in negative affect over the
one that most closely preceded the lapse (averaging about 8 hr preceding days. Analyses of stress measures collected just once
prior). A single temptation episode was selected for analy- each evening yielded similar results. This shows that day-level
sis from a prior day close to the lapse (temporal precedence variation in affect does not influence lapse risk.
being deemed important) and was similarly paired with a A second analysis was more fine grained and more proxi-
single preceding random assessment. This set up traditional, mal: It examined data on the day of the lapse itself and did
balanced repeated-measures analyses contrasting pre- versus not aggregate the data, but instead looked at the individual,
postepisode SE levels, in lapses and temptations (this study unaggregated assessments on the lapse day, arrayed by time
also included GEE analyses of multiple temptations). These (4 min to 18 hr), running back from the time of the lapse.
analyses make explicit use of time, defining time by anchoring GEE methods were used to examine temporal trends over that
it to lapse and temptation events to create pre- versus postevent interval. As shown in Figure 3, on the lapse day itself, emo-
measures. The data showed that SE did indeed decline after tional distress increased in the hours leading up to the lapse
a lapse but, sadly, did not increase after smokers successfully episode. Although retrieved in retrospect, these data were col-
resisted temptation (Shiffman, Hickcox, et al., 1997). lected prospectively and thus overcome concern that the sub-
jects’ post facto response to the lapse would bias their recall
of their mood. Running time backwards yielded a prospective
Time Following an Anchoring Event
timeline that could shed light on the precursors of later events.
The idea of defining time by anchoring it in an event of interest The analyses also illustrate the important effects of time scale:
can be extended beyond a simple pre- and postevent analysis. Emotional distress in the days before a lapse did not matter,
Some events are seen as leading to a change in experience over but emotional distress in the hours before a lapse did. This
time. One research question concerns the trajectory of craving highlights the importance of thinking carefully about different
intensity over time after a person quits smoking. Some theory units of analysis and different timeframes to assess the effects
suggests craving should progressively increase as abstinence of interest and shows the potential for EMA to provide data
grows longer. To construct such a timeline, time is anchored fine grained enough to examine the effects of acute changes
by the achievement of abstinence; the time axis cannot be in experience over a period of hours. However, even data col-
anchored by the calendar or study schedule, since individuals lection as intensive as that in this study has limited temporal
quit at different times (Shiffman et al., 2006). For the analysis, resolution. In this study, 20% of subjects had no assessments
craving ratings from random, signal-contingent time sampling preceding the lapse on the lapse day, and those who did have
assessments were aggregated by day to form a series of days assessments averaged only four assessments in the 9 hr (aver-
beginning with each subject’s initial abstinence and ending age) preceding the lapse episode, limiting the detail in which
when they reported lapsing to smoking (Shiffman, Engberg, the timecourse of affect could be described.
et al., 1997). Since the research question concerned the course
of craving over days and weeks, not its pattern within a day, it
was appropriate to aggregate data to the day level, making the
trajectory more interpretable and smoothing out circadian vari-
ations. Anchoring the timeline on the quit day and censoring on
the lapse day allowed the trend to be interpreted as referring to
the course of craving during abstinence. Using GEE methods,
the analysis showed that “background” craving (i.e., “steady-
state” craving in the absence of provocatuve cues; Shiffman,
1989) did not rise over time but actually declined rather steeply
to low levels as abstinence progressed.
S82
Nicotine & Tobacco Research, Volume 16, Supplement 2 (May 2014)
Time-to-Event Analyses: Time as Risk limited window during which other interventions might be able
to slow or reverse the slide towards relapse.
In the examples above, the analysis documents subject-
This complex analysis includes time in multiple ways. The
reported experience collected over time to display the tra-
x-axis in Figure 4 does not represent time, per se, but succes-
jectory of experiences over that interval. Time can figure
sive lapse episodes; in effect, it represents time as a sequence
into EMA-based analyses in another way: as an opportunity
of events, without regard to their spacing in time. The y-axis
to observe the occurrence and timing of events of interest.
represents, for each successive episode, the median time-to-
Important research questions revolve around the process of
relapse; each point in the graph is itself an estimate derived
progression from one lapse—particularly the very first—to
from the recurrent event survival analysis. Note, too, that no
the next. Although it is known that a lapse almost always pro-
EMA self-report data are used in this analysis—just the EMA
gresses to relapse (Kenford et al., 1994), it is not known what
data on the timing of events and the subject-level treatment
influences drive that progression. Marlatt and Gordon (1985)
assignment are used. Yet, the analysis is almost entirely about
hypothesized that smoker demoralization in response to a lapse
time, while also addressing treatment effects.
can lead to further lapses. This research question was evaluated
using smokers’ reports of whether they felt like giving up after
experiencing their first lapse. Change in Effects Over Time
One way to assess how this affected progression to a subse- The preceding analysis illustrates an important principle: The
quent lapse is to use survival analysis or time-to-event methods effect of one variable on another can vary over time. Models
(Hosmer, Lemeshow, & May, 2008), which analyze the risk known as time-varying effects models (TVEMs; Li et al., 2006)
of an event, such as a lapse. Events occurring sooner indicate are designed to capture such effect moderation in very flex-
greater risk (per unit time), and the analysis takes into account ible ways. The analysis does not force temporal trends to fit
that some events are unobserved or censored (e.g., a lapse could a particular parametric function but allows for very flexible
occur after the study ended). A survival (or time-to-event) anal- nonparametric fitting. Marlatt and Gordon (1985) hypoth-
ysis (Cox proportional hazards) related subjects’ readiness to esized that low SE would promote craving. However, both SE
give up after a lapse to the length of time they maintained absti- and craving vary substantially over the course of a quit effort,
nence before lapsing again (or not). The analysis showed that leading Shiyko, Lanza, Tan, Li, and Shiffman (2012) to exam-
subjects who felt like giving up did indeed progress to the next ine whether the relationship between SE and craving might
lapse more quickly (Shiffman, Hickcox, et al., 1996). Note that vary over time, perhaps in different ways for successful versus
this analysis does not use any data collected during the interval unsuccessful quitters.
from the first lapse to the second: It just uses the record of Using event-level data, a TVEM analysis indeed showed
the second lapse (if any) to define the time-to-relapse. Thus, differences. As shown in Figure 5, among successful quitters,
in such analyses, time is a crucial focus, but just as a basis for higher SE was associated with lower urges throughout the
understanding the risk of an event such as a recurring lapse. In interval. The relationship was more dynamic among relapsers:
such analyses, the ability of EMA to provide precise timing SE and urges were unrelated in the first 2 days of quitting, but
for the relapse was crucial, as was the ability to get data on the association grew stronger over time, reaching the degree of
subjects’ immediate response to the lapse. association seen among successful quitters about a week later,
S83
Conceptualizing analyses of EMA data
S84
Nicotine & Tobacco Research, Volume 16, Supplement 2 (May 2014)
on matters related to smoking cessation. GlaxoSmithKline Gwaltney, C. J., Shiffman, S., & Sayette, M. A. (2005).
Consumer Healthcare provided the nicotine patches used in Situational correlates of abstinence self-efficacy.
the patch study reported in this paper. Journal of Abnormal Psychology, 114, 649–660.
doi:10.1037/0021-843X.114.4.649
Holt, L. J., Litt, M. D., & Cooney, N. L. (2012). Prospective
analysis of early lapse to drinking and smoking among
Acknowledgments individuals in concurrent alcohol and tobacco treat-
The author is grateful for helpful comment on a draft from ment. Psychology of Addictive Behaviors, 26, 561–572.
doi:10.1037/a0026039
Ellen Beckjord, Thomas Kirchner, Michael Dunbar, Sarah
Hosmer, J. E., Lemeshow, S., & May, S. (2008). Applied sur-
Scholl, Arthur Stone, and Susan Murphy. Many coauthors con- vival analysis: Regression modeling to time to event data.
tributed to the work cited herein, and their contributions are New York, NY: Wiley.
gratefully acknowledged. Hufford, M. R. (2007). Special methodological challenges and
opportunities in Ecological Momentary Assessment. In A.
A. Stone, S. Shiffman, A. A. Atienza, & L. Nebeling (Eds.),
References The science of real-time data capture: Self-reports in health
research (pp. 54–75). New York, NY: Oxford University
Ali, A. A., Hossain, S. M., Hovsepian, K., Rahman, M., Plarre, Press.
K., & Kumar, S. (2012). mpuff: Automated detection of cig- Hufford, M. R., & Shiffman, S. (2002). Methodological issues
arette smoking puffs from respiration measurements. IPSN affecting the value of patient-reported outcomes data. Expert
‘12: Proceedings of the 11th International Conference on Review of Pharmacoeconomics & Outcomes Research, 2,
Information Processing in Sensor Networks, Association 119–128. doi:10.1586/14737167.2.2.119
for Computing Machinery, Beijing, China. 269–280. Kassel, J. D., Stroud, L. R., & Paronis, C. A. (2003). Smoking,
doi:10.1145/2185677.2185741 stress, and negative affect: Correlation, causation, and con-
Beckham, J. C., Wiley, M. T., Miller, S. C., Dennis, M. F., text across stages of smoking. Psychological Bulletin, 129,
Wilson, S. M., McClernon, F. J., & Calhoun, P. S. (2008). Ad 270–304. doi:1037/0033-2909.129.2.270
lib smoking in post-traumatic stress disorder: An electronic Kenford, S. L., Fiore, M. C., Jorenby, D. E., Smith, S. S.,
diary study. Nicotine & Tobacco Research, 10, 1149–1157. Wetter, D., & Baker, T. B. (1994). Predicting smoking ces-
doi:10.1080/14622200802123302 sation. Who will quit with and without the nicotine patch.
Benowitz, N. L. (2008). Clinical pharmacology of nicotine: Journal of the American Medical Association, 271, 589–594.
Implications for understanding, preventing, and treating doi:10.1001/jama.1994.03510320029025
tobacco addiction. Clinical Pharmacology and Therapeutics, Kirchner, T. R., Cantrell, J., Anesetti-Rothermel, A., Ganz, O.,
83, 531–541. doi:10.1038/clpt.2008.3 Vallone, D. M., & Abrams, D. B. (2013). Geospatial expo-
Bolt, D. M., Piper, M. E., Theobald, W. E., & Baker, T. B. sure to point-of-sale tobacco: Real-time craving and smok-
(2012). Why two smoking cessation agents work better than ing-cessation outcomes. American Journal of Preventive
one: Role of craving suppression. Journal of Consulting and Medicine, 45, 379–385. doi: 10.1016/j.amepre.2013.05.016
Clinical Psychology, 80, 54–65. doi:10.1037/a0026366 Kirchner, T. R., Shiffman, S., & Wileyto, E. P. (2012). Relapse
Bradburn, N. M., Rips, L. J., & Shevell, S. K. (1987). dynamics during smoking cessation: Recurrent abstinence
Answering autobiographical questions: The impact of mem- violation effects and lapse-relapse progression. Journal of
ory and inference on surveys. Science, 236, 157–161. Abnormal Psychology, 121, 187–197. doi:10.1037/a0024451
Carter, B. L., Paris, M. M., Lam, C. Y., Robinson, J. D., Li, R., Root, T. L., & Shiffman, S. (2006). A local linear esti-
Traylor, A. C., Waters, A. J., … Cinciripini, P. M. (2010). mation procedure of functional multilevel modeling. In T.
Real-time craving differences between Black and White A. Walls & J. L. Schafer (Eds.), Models for intensive longi-
smokers. American Journal of Addiction, 19, 136–140. tudinal data (pp. 63–83). New York, NY: Oxford University
doi:10.1111/j.1521-0391.2009.00020.x Press.
Chandra, S., Shiffman, S., Scharf, D. M., Dang, Q., & Magnan, R. E., Köblitz, A. R., McCaul, K. D., & Dillard, A.
Shadel, W. G. (2007). Daily smoking patterns, their deter- J. (2013). Self-monitoring effects of ecological momen-
minants, and implications for quitting. Experimental tary assessment on smokers’ perceived risk and worry.
and Clinical Psychopharmacology, 15, 67–80. Psychological Assessment, 25, 416–423. doi:10.1037/
doi:10.1037/1064-1297.15.1.67 a0031232
Collins, L. M. (2006). Analysis of longitudinal data: The inte- Marlatt, G. A., & Gordon, J. R. (1985). Relapse prevention:
gration of theoretical model, temporal design, and statistical Maintenance strategies in the treatment of addictive behav-
model. Annual Review of Psychology, 57, 505–528. iors. New York, NY: Guilford Press.
Cooney, N. L., Litt, M. D., Cooney, J. L., Pilkey, D. T., McCarthy, D. E., Piasecki, T. M., Fiore, M. C., & Baker, T. B.
Steinberg, H. R., & Oncken, C. A. (2007). Alcohol and (2006). Life before and after quitting smoking: An electronic
tobacco cessation in alcohol-dependent smokers: Analysis diary study. Journal of Abnormal Psychology, 115, 454–466.
of real-time reports. Psychology of Addictive Behaviors, 21, doi:10.1037/0021-843X.115.3.454
277–286. doi:10.1037/0893-164X.21.3.277 McCarthy, D. E., Piasecki, T. M., Jorenby, D. E., Lawrence,
Epstein, D. H., Marrone, G. F., Heishman, S. J., Schmittner, D. L., Shiffman, S., & Baker, T. B. (2010). A multi-level
J., & Preston, K. L. (2010). Tobacco, cocaine, and heroin: analysis of non-significant counseling effects in a rand-
Craving and use during daily life. Addictive Behaviors, 35, omized smoking cessation trial. Addiction, 105, 2195–2208.
318–324. doi:10.1016/j.addbeh.2009.11.003 doi:10.1111/j.1360-0443.2010.03089.x
Ferguson, S. G., Shiffman, S., & Gwaltney, C. J. (2006). McCarthy, D. E., Piasecki, T. M., Lawrence, D. L., Jorenby,
Does reducing withdrawal severity mediate nicotine D. E., Shiffman, S., & Baker, T. B. (2008). Psychological
patch efficacy? A randomized clinical trial. Journal of mediators of bupropion sustained-release treatment
Consulting and Clinical Psychology, 74, 1153–1161. for smoking cessation. Addiction, 103, 1521–1533.
doi:10.1037/0022-006X.74.6.1153 doi:10.1111/j.1360-0443.2008.02275.x
S85
Conceptualizing analyses of EMA data
Mermelstein, R., Hedeker, D., & Wesintein, S. (2010). Shiffman, S. (1989). Conceptual issues in the study of relapse.
Ecological momentary assessment of mood-smoking rela- In M. Gossop (Ed.), Relapse and addictive behaviour (pp.
tionships in adolescent smokers. Washington, DC: American 149–179). Kent, England: Routledge.
Psychological Association. Shiffman, S. (2007). Designing protocols for Ecological
Minami, H., McCarthy, D. E., Jorenby, D. E., & Baker, Momentary Assessment. In A. A. Stone, S. Shiffman, A.
T. B. (2011). An Ecological Momentary Assessment Atienza, & L. Nebeling (Eds.), The Science of Real-time
analysis of relations among coping, affect and smok- Data Capture: Self-reports in Health Research (pp. 27–53).
ing during a quit attempt. Addiction, 106, 641–650. New York, NY: Oxford University Press.
doi:10.1111/j.1360-0443.2010.03243.x Shiffman, S. (2009). Ecological momentary assessment (EMA)
Niaura, R. S., Rohsenow, D. J., Binkoff, J. A., Monti, P. M., in studies of substance use. Psychological Assessment, 21,
Pedraza, M., & Abrams, D. B. (1988). Relevance of cue 486–497. doi:10.1037/a0017074
reactivity to understanding alcohol and smoking relapse. Shiffman, S. (in press). Ecological momentary assessment. In
Journal of Abnormal Psychology, 97, 133–152. K. Sher (Ed.), The Oxford handbook of substance use disor-
Paty, J. A., Kassel, J. D., & Shiffman, S. (1992). The impor- ders: Oxford University Press.
tance of assessing base rates for clinical studies: an exam- Shiffman, S., Elash, C. A., Paton, S. M., Gwaltney, C. J., Paty,
ple of stimulus control of smoking. In M. W. De Vries J. A., Clark, D. B., … DiMarino, M. E. (2000). Comparative
(ed.), The Experience of Psychopathology: Investigating efficacy of 24-hour and 16-hour transdermal nicotine patches
Mental Disorders in Their Natural Settings. New York, NY: for relief of morning craving. Addiction, 95, 1185–1195.
Cambridge University Press; 347–352. doi:10.1046/j.1360-0443.2000.95811855.x
Piasecki, T. M., Jorenby, D. E., Smith, S. S., Fiore, M. C., Shiffman, S., Engberg, J. B., Paty, J. A., Perz, W. G.,
& Baker, T. B. (2003). Smoking withdrawal dynam- Gnys, M., Kassel, J. D., & Hickcox, M. (1997). A
ics: II. Improved tests of withdrawal-relapse rela- day at a time: Predicting smoking lapse from daily
tions. Journal of Abnormal Psychology, 112, 14–27. urge. Journal of Abnormal Psychology, 106, 104–116.
doi:10.1037/0021-843X.112.1.14 doi:10.1037/0021-843X.106.1.104
Piasecki, T. M., Niaura, R., Shadel, W. G., Abrams, D., Goldstein, Shiffman, S., Gwaltney, C. J., Balabanis, M. H., Liu, K. S., Paty,
M., Fiore, M. C., Baker, T. B. (2000). Smoking with- J. A., Kassel, J. D., … Gnys, M. (2002). Immediate ante-
drawal dynamics in unaided quitters. Journal of Abnormal cedents of cigarette smoking: An analysis from ecological
Psychology, 109, 74–86. doi:10.1037/0021-843X.109.1.74 momentary assessment. Journal of Abnormal Psychology,
Piasecki, T. M., Wood, P. K., Shiffman, S., Sher, K. J., & 111, 531–545. doi:10.1037/0021-843X.111.4.531
Heath, A. C. (2012). Responses to alcohol and ciga- Shiffman, S., Hickcox, M., Paty, J. A., Gnys, M., Kassel, J. D.,
rette use during ecologically assessed drinking epi- & Richards, T. (1996). Progression from a smoking lapse
sodes. Psychopharmacology, 223, 331–344. doi:10.1007/ to relapse: Prediction from abstinence violation effects,
s00213-012-2721-1 nicotine dependence, and lapse characteristics. Journal
Piper, M. E., Bolt, D. M., Kim, S., Japuntich, S. J., Smith, S. S., of Consulting and Clinical Psychology, 64, 993–1002.
Niederdeppe, J., … Baker, T. B. (2008). Refining the tobacco doi:10.1037/0022-006X.64.5.993
dependence phenotype using the Wisconsin Inventory Shiffman, S., Hickcox, M., Paty, J. A., Gnys, M., Kassel,
of Smoking Dependence Motives. Journal of Abnormal J. D., & Richards, T. (1997). The abstinence viola-
Psychology, 117, 747–761. doi:10.1037/a0013298 tion effect following smoking lapses and temptations.
Piper, M. E., Cook, J. W., Schlam, T. R., Jorenby, D. E., & Cognitive Therapy and Research, 21, 497–523. doi:10.10
Baker, T. B. (2011). Anxiety diagnoses in smokers seeking 23/A:1021853301255
cessation treatment: Relations with tobacco dependence, Shiffman, S., Hufford, M., Hickcox, M., Paty, J. A., Gnys,
withdrawal, outcome and response to treatment. Addiction, M., & Kassel, J. D. (1997). Remember that? A comparison
106, 418–427. doi:10.1111/j.1360-0443.2010.03173.x of real-time versus retrospective recall of smoking lapses.
Rabe-Hesketh, S., & Skrondal, A. (2008). Multilevel and lon- Journal of Consulting and Clinical Psychology, 65, 292–
gitudinal modeling using Stata. College Station, TX: STATA 300. doi:10.1037/0022-006X.65.2.292.a
Press. Shiffman, S., Paty, J. A., Gnys, M., Kassel, J. D., & Hickcox,
Rathbun, S. L., Shiffman, S., & Gwaltney, C. J. (2007). M. (1996). First lapses to smoking: Within-subjects analy-
Modeling the effects of partially observed covariates on ses of real-time reports. Journal of Consulting and Clinical
Poisson process intensity. Biometrika, 94, 153–165. Psychology, 64, 366–379. doi:10.1037/0022-006X.64.2.366
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear Shiffman, S., & Rathbun, S. L. (2011). Point process analyses
models: Applications and data analysis methods (2nd ed.). of variations in smoking rate by setting, mood, gender, and
Newbury Park, CA: Sage Publications. dependence. Psychology of Addictive Behaviors, 25, 501–
Raudenbush, S. W., Bryk, A. S., Cheong, J., & Congdon, R. 510. doi:10.1037/a0022178
(2010). HLM7 manual. Lincolnwood, IL: SSI. Shiffman, S., Scharf, D. M., Shadel, W. G., Gwaltney,
Schwartz, J. E., & Stone, A. A. (2007). Analysis of real-time C. J., Dang, Q., Paton, S. M., & Clark, D. B. (2006).
momentary data: A practical guide. In A. A. Stone, S. Analyzing milestones in smoking cessation: Illustration
Shiffman, A. Atienza, & L. Nebeling (Eds.), The science of in a nicotine patch trial in adult smokers. Journal of
real-time data capture (pp. 76–113). New York, NY: Oxford Consulting and Clinical Psychology, 74, 276–285.
University Press. doi:10.1037/0022-006X.74.2.276
Shadel, W. G., Martino, S. C., Setodji, C., & Scharf, D. (2012). Shiffman, S., Stone, A. A., & Hufford, M. R. (2008).
Momentary effects of exposure to prosmoking media on col- Ecological momentary assessment. Annual Review of
lege students’ future smoking risk. Health Psychology, 31, Clinical Psychology, 4, 1–32. doi:10.1146/annurev.
460–466. doi:10.1037/a0027291 clinpsy.3.022806.091415
Shapiro, D., Jamner, L. D., Davydov, D. M., & James, P. Shiffman, S., & Waters, A. J. (2004). Negative affect
(2002). Situations and moods associated with smoking in and smoking lapses: A prospective analysis. Journal
everyday life. Psychology of Addictive Behaviors, 16, 342– of Consulting and Clinical Psychology, 72, 192–201.
345. doi:10.1037/0893-164X.16.4.342 doi:10.1037/0022-006X.72.2.192
S86
Nicotine & Tobacco Research, Volume 16, Supplement 2 (May 2014)
Shiffman, S., Waters, A., & Hickcox, M. (2004). The Nicotine Stone, A. A., & Shiffman, S. (1994). Ecological momen-
Dependence Syndrome Scale: A multidimensional measure tary assessment (EMA) in behavioral medicine. Annals of
of nicotine dependence. Nicotine & Tobacco Research, 6, Behavioral Medicine, 16, 199–202.
327–348. doi:10.1080/1462220042000202481 Stone, A. A., Shiffman, S., Atienza, A., & Nebeling, L.
Shiyko, M. P., Lanza, S. T., Tan, X., Li, R., & Shiffman, S. (Eds.). (2007). The science of real-time data capture:
(2012). Using the time-varying effect model (TVEM) to Self-reports in health research. New York, NY: Oxford
examine dynamic associations between negative affect and University Press.
self confidence on smoking urges: Differences between suc- Walls, T. A., & Schafer, J. L. (2006). Models for intensive lon-
cessful quitters and relapsers. Prevention Science, 13, 288– gitudinal data. New York, NY: Oxford University Press.
299. doi:10.1007/s11121-011-0264-z Warthen, M. W., & Tiffany, S. T. (2009). Evaluation of cue
Singer, J. D. (1998). Using SAS PROC MIXED to fit multi- reactivity in the natural environment of smokers using eco-
level models, hierarchical models, and individual growth logical momentary assessment. Experimental and Clinical
models. Journal of Educational and Behavioral Statistics, Psychopharmacology, 17, 70–77.doi:10.1037/a0015617
24, 323–355. doi:10.2307/1165280 West, R., & Schneider, N. (1987). Craving for cigarettes.
Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data British Journal of Addiction, 82, 407–415.
analysis: Modeling change and event occurrence. New York, Wheeler, L., & Reis, H. (1991). Self-recording of everyday life
NY: Oxford University Press. events: Origins, types, and uses. Journal of Personality, 59,
Sinha, R. (2001). How does stress increase risk of drug 339–354.
abuse and relapse? Psychopharmacology, 158, 343–359. Witkiewitz, K., Desai, S. A., Steckler, G., Jackson, K.
doi:10.1007/s002130100917 M., Bowen, S., Leigh, B. C., & Larimer, M. E. (2012).
Snijders, T., & Bosker, R. (2011). Multilevel analysis: An intro- Concurrent drinking and smoking among college students:
duction to basic and advanced multilevel modeling. London, An event-level analysis. Psychology of Addictive Behaviors,
UK: Sage. 26, 649–654. doi:10.1037/a0025363
Stolerman, I. P., & Jarvis, M. J. (1995). The scientific case Zeger, S. L., Liang, K. Y., & Albert, P. S. (1988). Models
that nicotine is addictive. Psychopharmacology (Berl), 117, for longitudinal data: A generalized estimating equation
2–10. doi:10.1007/BF02245088 approach. Biometrics, 44, 1049–1060.
S87