Load prediction models in Web-based systems

Mauro Andreolini Sara Casolari

Department of Information Engineering Department of Information Engineering
University of Modena and Reggio Emilia University of Modena and Reggio Emilia
[email protected] [email protected]

ABSTRACT from transient peaks, for understanding load trends and for antic-
Run-time management of modern Web-based services requires the ipating future conditions, that are of utmost importance for taking
integration of several algorithms and mechanisms for job dispatch- correct decisions.
ing, load sharing, admission control, overload detection. All these In this paper, we address the important issue of predicting future
algorithms should take decisions on the basis of present and/or fu- load conditions of a resource, that is at the basis of several run-time
ture load conditions of the system resources. In particular, we ad- management tasks. We will see that the direct use of measured
dress the issue of predicting future resource loads under real-time raw data does not allow us to solve this problem, because resource
constraints in the context of Internet-based systems. In this situa- measures obtained from load monitors of Internet-based servers are
tion, it is extremely difficult to deduce a representative view of a extremely variable even at different time scales, and tend to become
system resource from collected raw measures that show very large obsolete rather quickly [16]. Consequently, long time measurement
variability even at different time scales. For this reason, we propose intervals reduce the effectiveness of the resource measure as a suit-
a two-step approach that first aims to get a representative view of able indicator of the real load conditions. Moreover, the workload
the load trend from measured raw data, and then applies a load pre- reaching the Internet-based systems that we consider in this paper
diction algorithm to load trends. This approach is suitable to sup- is typically characterized by heavy-tailed distributions [4, 11, 15]
port different decision systems even for highly variable contexts and by flash crowds [23] that contribute to augment the skewness
and is characterized by a computational complexity that is com- of the raw data. These characteristics of load resource measures
patible to run-time decisions. The proposed models are applied to in Internet-based systems make really difficult, if not impossible,
a multi-tier Web-based system, but the results can be extended to to forecast the behavior of future resource measures because they
other Internet-based contexts where the systems are characterized appear as unrelated to the previous sample, and to deduce a clear
by similar workloads and resource behaviors. trend about the load behavior of a resource, for example to find out
whether a resource is offloading, overloading or stabilizing.
We have verified that, in a heavy-tailed context, it is of little value
1. INTRODUCTION to let a load predictor work directly on resource measures, because
The most critical Internet-based services are provided by dis- they give only a limited and instantaneous view of the resource and
tributed infrastructures that have to satisfy scalability and availabil- do not capture its behavioral trend that is of major interest for pre-
ity requirements, and have to avoid performance degradation and diction. Hence, we can anticipate that in Web-based systems it is
system overload. Managing these systems requires a large set of not convenient to base load prediction directly on resource mea-
algorithms, for load balancing and load sharing [2, 8, 32], overload sures, as done in different contexts [1, 5, 12], but it is preferable
and admission control [14, 20, 28], job dispatching and redirec- to operate on a “representation” of the load behavior of system re-
tion even at a geographical scale [9]. The advent of self-adaptive sources. To this purpose, we propose and compare different func-
systems and autonomic computing [21, 25, 39] will further increase tions, called load trackers, that may offer a representative view of
the necessity for management algorithms requiring a run-time eval- the load conditions to the load predictors, thus achieving the two-
uation of the load conditions of hardware and software system re- step approach shown in Figure 1. In this paper, we consider three
sources, and the possibility of predicting future load values of the classes of load trackers that are suitable to support different deci-
system components. Almost all algorithms and mechanisms evalu- sion systems and are characterized by a computational complexity
ate the load conditions of a system resource through the periodic that is compatible to run-time decisions: two of them are based on
sampling of raw data that we call resource measures. While a linear models (simple and exponential moving average), and one is
measure offers an instantaneous view of the load conditions of a based on a non-linear model (cubic spline). We compare the effi-
resource, it is of little help for distinguishing overload conditions cacy of the proposed load trackers for the support of load prediction
goals. To this purpose, we define also metrics for evaluating the ef-
ficacy of a load tracker that supports load prediction, according to
the accuracy of the load tracker to represent the real load, and to the
Permission to make digital or hard copies of all or part of this work for error between the real and predicted load values. Our results show
personal or classroom use is granted without fee provided that copies are that the precision of the predictor depends largely on the choice of
not made or distributed for profit or commercial advantage and that copies the load tracker. Smoother load trackers typically lead to a better
bear this notice and the full citation on the first page. To copy otherwise, to prediction because of their small prediction error and low compu-
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
tational costs. Thanks to the two-step approach, and an adequate
Valuetools’06, October 11-13, 2006, Pisa, Italy load tracker, even simple linear-based predictor models are able to
Copyright 2006 ACM 1-59593-504-5/06/10...$5.00 $.5.00.
 model [37] that is becoming a de facto standard for the performance
analysis of Web-based systems (e.g., [7, 10, 18]). The first node ex-
ecutes the HTTP server and the application server that is deployed
through the Tomcat [36] servlet container; the second node runs
the MySQL [29] database server. Requests are generated through a
set of emulated browsers, where each browser is implemented as a
Java thread that emulates an entire user session. We use three TPC-
W like workload mixes that represent different scenarios shown in
Figure 3.

• Step scenario. This scenario describes a sudden load incre-

ment from a relatively unloaded to a more loaded system.
The population is kept at 300 emulated browsers for 5 min-
utes, then suddenly increased to 700 emulated browsers for
other 5 minutes.

• Staircase scenario. This scenario represents a gradual incre-

ment in the population up to 600 emulated browsers, that is
followed by a similar decrease.

• Alternating scenario. This scenario describes an alternat-

ing increment and decrease of the load between 300 and 600
emulated browsers, every two minutes.

There are many critical resources in each server node that can be
measured through several system monitors [22,31,34]. These tools
Figure 1: A framework for supporting run-time decisions in yield instantaneous resource measures at regular time intervals. We
Web-based systems have carried out a very large set of experiments for analyzing the
typical behavior of commonly measured resources, such as CPU
utilization, disk and network throughput, number of open sockets,
achieve good predictions. number of open files, UNIX load, amount of used main memory,
Previous literature [5, 12, 17, 38] uses linear models to predict real for different resource measure intervals and workload scenarios.
resource load measures. However, we should consider that linear Here, we report a subset of results that are representative of the
models have various drawbacks in the context of highly variable main observations and conclusions about the behavior of resource
Web-based systems where decisions should be taken in real-time. measures. In the Figures 4 we describe the CPU and disk mea-
For example, linear (auto)regression works well provided that the sures of the back-end node of the multi-tier architecture shown in
resource measures are highly correlated, which is not always the Figure 2. In particular, we report:
case in heavy-tailed systems. Furthermore, linear (auto)regressive
models require a training period to compute the parameters that • two rates for resource measurement: 1 second and 5 seconds
may not be available to decision systems subject to run-time con- (Figure 4 (a) and (b), respectively);
straints.
The experimental results are based on realistic scenarios in the con- • two resource metrics: disk throughput (blocks/sec.) and CPU
text of a multi-tier Web-based system, but the proposed methodol- utilization (Figure 4 (c) and (d), respectively);
ogy is not tied to specific application contexts (such as in [12, 13])
• three user scenarios: step, staircase, alternating (Figure 4 (c),
and can be applied to other Internet-based systems as well.
(e) and (f)).
The paper is organized as follows. Section 2 gives a motivation for
this paper by showing the extreme variability of resource measures
at different time scales and under different Web related workload
scenarios. It also describes the test-bed architecture and workload
models that we use in this paper. Section 3 proposes the load track-
ers and their properties for load prediction goals that are detailed
in Section 4. Section 5 shows the experimental results. Section 6
compares the results of this paper with respect to the state of the
art. Section 7 concludes the paper with some final remarks.

2. RESOURCE MEASURES
In this section, we consider the behavior of measured resource
loads in the context of a dynamic Web-based system from which
we can see that the view of a resource load obtained by monitors
is extremely oscillatory at different time scales and for different
workload scenarios. The considered test-bed refers to a typical
multi-tier logical architecture (Figure 2) that is based on the im-
plementation presented in [7]. The workload refers to the TPC-W Figure 2: Architecture of a multi-tier Web-based system
 (a) Step scenario (b) Staircase scenario (c) Alternating scenario

Figure 3: Considered workload scenarios

1000 1 1
Disk throughput/DB/Staircase scenario/1s CPU utilization/DB/Step scenario/1s CPU utilization/DB/Staircase scenario/1s

800 0.8 0.8

Disk throughput

CPU utilization

CPU utilization
600 0.6 0.6

400 0.4 0.4

200 0.2 0.2

0 0 0
0 100 200 300 400 500 600 0 100 200 300 400 500 600 0 100 200 300 400 500 600
Time [s] Time [s] Time [s]

(a) Sample rate: 1s (c) Resource: CPU (e) Scenario: prediction

1000 1000 1
Disk throughput/DB/Staircase scenario/5s Disk throughput/DB/Step scenario/1s CPU utilization/DB/Alternating scenario/1s

800 800 0.8

Disk throughput

Disk throughput

CPU utilization
600 600 0.6

400 400 0.4

200 200 0.2

0 0 0
0 100 200 300 400 500 600 0 100 200 300 400 500 600 0 100 200 300 400 500 600
Time [s] Time [s] Time [s]

(b) Sample rate: 5s (d) Resource: disk (f) Scenario: trend

Figure 4: Resource measurements

All these figures share the common trait that the view of a re- correlation. This variability is critical even for some performance
source obtained from monitored measures is extremely variable to control mechanisms [1, 24] that follow a feedback control theory
the extent that run-time decisions based on these values may be approach, where the major risk is represented by the instability and
risky when not completely wrong. Even if a load predictor is able self-induced oscillations.
to guess precisely a load value in the future, is this value really Another question arises about the computational complexity of the
useful for taking a correct decision? For example, let us suppose load predictor models. In some literature, the high skewness of load
that a load predictor evaluates a future CPU utilization measure measurements is addressed by sophisticated load predictor models
with high precision, and this predicted value is used by an admis- that may require training periods and/or off-line analyses. These
sion controller to decide about the acceptance/refusal of requests models are not suitable to support the run-time decision systems
for the satisfaction of some service level agreements. Actually, a considered in this paper. In other words, we should not look for the
predicted value is a single point in the space, and no one in a highly most precise load predictor, but for the best predictor that is able to
variable context would have faith in it for taking critical decisions. work subject to real time constraints.
If we do not figure out the behavioral trend from a sequence of
measures, even accurate predicted values share the same charac-
teristics of the resource measures: instability, variability, low auto-
 2
3. LOAD TRACKER FUNCTIONS where the constant α = n+1 is the smoothing factor. The initial
−
→
In this section, we describe the first phase of the two-step ap- EM A(Sn (tn )) value is initialized to the arithmetical mean of the
proach that aims to get a representative view of the load trend from first n measures:
measured raw data instead of letting a load predictor work on raw P
sj
measures of resource loads. To this purpose, we propose a new −
→ 0≤j≤n
function, called load tracker, that filters out the noises presented EM A(Sn (tn )) = (3)
n
by a sequence of uncorrelated resource measures and yields a more
As the last resource measures give a contribution higher than the
regular view about the load trend of a resource. −
→
We consider a resource measure si sampled at time ti , and a set of first measures of the set Sn (ti ), a load tracker based on a EMA
−
→ model is able to react rather quickly to changes in the resource load
previously collected n measures, that is, Sn (ti ) = (si−n , . . . , si ).
−→ conditions similarly to a short-term SMA, with the advantage that
We define load tracker a function LT (Sn (ti )) : Rn → R that EMA is subject to less oscillations than a short-term SMA.
−→
takes as its input Sn (ti ) and gives a “representation” of the re-
source load conditions li at time ti . Multiple applications of the 3.2 Non-linear load trackers
load tracker give a sequence of load values that should exclude out- The necessity to consider a non-linear tracker depends on some
of-scale resource measures and yield a regular trend of the resource limits shown by the linear load descriptors. We will see that the
load conditions. For the purposes of this paper, we consider and simple SMA and EMA models introduce a delay in load trend de-
compare two linear and one non-linear load trackers as supports −→
scription when |Sn (ti )| increases. Even the more sophisticated
for load predictors. ARMA and ARIMA models are not a valid alternative because
they are strongly dependent on the considered resource metrics and
3.1 Linear load trackers workload characteristics.
We consider the class of moving averages as linear load trackers. In this paper, we consider the cubic spline function [33] as a rep-
Indeed, moving averages smooth out resource measures, reduce the resentative example of a non-linear tracker. Lower-order curves
effect of out-of-scale values, are fairly easy to compute at run-time, (with a degree less than 3) do not react quickly to load changes,
and are commonly used as trend indicators [26]. We focus on two while higher-order curves (with a degree higher than 3) are con-
classes of moving average: the Simple Moving Average (SMA) and sidered unnecessary complex, introduce undesired wiggles and are
the Exponential Moving Average (EMA) that use uniform and non- computationally too expensive for a run-time context. For the defi-
uniform weighted distributions of the past measures, respectively. nition of the cubic spline, let us choose some control points (tj , sj )
We do not consider other popular linear auto regressive models, in the set of measured load values, where tj is the time of measure-
such as ARMA and ARIMA [17, 38], because they would require ment of sj . A cubic spline function CSJ (t), based on J control
frequent updates of their parameters in the case of highly variable points, is a set of J − 1 piecewise third-order polynomials pj (t),
systems. For these reasons, the auto regressive models are typically where j ∈ [1, J − 1], that satisfy the following properties.
created offline after examination of all available data, or applied Property 1. The control points are connected through third-order
to workloads characterized by smaller variability and high auto- polynomials:
correlation of load measures. These operations are computationally (
too expensive and inadequate to support run-time decision systems, CS J (tj ) = sj j = 1, . . . , J,
(4)
especially in a context where auto-correlation values are low. CS J (t) = pj (t) tj < t < tj+1 , j = 1, . . . , J
Simple Moving Average (SMA). It is the unweighted mean of
−→ Property 2. To guarantee a C 2 behavior at each control point, the
the n resource measures of the vector Sn (ti ), evaluated at time ti :
first order and second order derivatives of pj (t) and pj+1 (t) are set
P
sj equal at time tj+1 :
−
→ i−n≤j≤i ( dp (t )
SM A(Sn (ti )) = (1) j j+1 dp (t
= j+1dt j+1 ,
)
n 2
dt
2 (5)
d pj (tj+1 ) d pj+1 (tj+1 )
−
→ dt2
= dt2
A SMA-based load tracker evaluates SM A(Sn (ti )) for each con-
sidered sample si during the entire observation period. As SMAs If we apply both properties, we obtain the following definition of
assign an equal weight to every resource measure, they have to CS J (t):
solve a clear trade-off. Load trackers based on short-term moving zj+1 (t − tj )3 + zj (tj+1 − t)3
−
→ CS J (t) =
averages (that is, working on a small set of load measures Sn (ti )) 6hi
are more responsive to variations of the load conditions, but at the sj+1 hj
expense of increased oscillations. On the other hand, long-term +( − zj+1 )(t − tj ) (6)
hi 6
moving averages are able to smooth out all minor fluctuations, and
sj hj
tend to show only long-term trends [3]. Typically, the SMA mod- +( − zj )(tj+1 − t)
els tend to introduce a significant delay in the trend representation hj 6
−
→
when the size of the set |Sn (ti )| increases. The EMA models are where hj = tj+1 − tj . The zj coefficients are solved by the fol-
often proposed to limit this delay effect, without incurring in oscil- lowing system of equations:
lation risks characterizing short-term moving averages. 8
Exponential Moving Average (EMA). It is the weighted mean <z0 = 0
>
−
→ hj−1 zj−1 + 2(hj−1 + hj )zj + hj zj+1 = 6( j+1
s −sj s −sj−1
− jhj−1 )
of the n resource measures of the set Sn (ti ), where the weights > hj
−
→ :
decrease exponentially. A EMA-based load tracker LT (Sn (ti )), zn = 0
for each time ti where i > n, is equal to: (7)
−→
−
→ −
→ The spline-based load tracker LT (Sn (ti )), at time ti is defined as
EM A(Sn (ti )) = α ∗ si + (1 − α) ∗ EM A(Sn (ti−1 )) (2) the cubic spline CSnJ (ti ) that is obtained through a subset of J
 −
→
control points from the vector of the load measures Sn (ti ), having ues may be risky when not completely wrong; hence, in our
dimension n. context we think that the prediction of real resource measures
While the cubic spline is more expensive to compute than the SMA is not really useful for taking correct decisions.
and EMA models, it is commonly used in approximation and smooth-
ing contexts [19, 33, 40], because its computational complexity is • Many proposed load predictors working on real measures are
compatible to support run-time decision systems. Indeed, load not suitable to support a run-time decision system because of
trackers based on moving average models compute a load tracker their computational complexity.
value at each resource measure, while the proposed load tracker
based on the cubic spline function CSnJ returns a new value after n We confirm our hypothesis through a study of the auto-correlation
resource measures. Moreover, the cubic spline has the advantage of function of the CPU utilization measures. The accuracy of a pre-
being reactive to load changes and independent of resource metrics diction algorithm depends on the correlation between consecutive
and workload characteristics. resource measures, and when the auto-correlation functions of the
set of analyzed data rapidly fall, it is more difficult to have an
3.3 Accuracy of load trackers accurate prediction [5, 38]. In these scenarios, even an attractive
All the considered load trackers have the common goal of repre- approach for statistical modeling and forecasting of complex tem-
senting the trend of a set of resource measures obtained from some poral series, such as the Box-Jankins’s ARIMA (Auto-regressive
load monitor. A load tracker is accurate if it is able to perfectly fol- Integrated Moving Average) method [6], tends to produce models
low the ideal trend. For the purposes of this paper, we consider as that do not adapt well to highly variable changes in the workloads.
the ideal trend the indicator of the central tendency of the resource In Figure 6(a), we analyze the auto-correlation function (ACF) of
measures in specific intervals of the experiment where the load is the CPU utilization for the three considered scenarios during an
subject to significant changes. If you do not control the load gen- observation period of 600 seconds. A point (k, y) in this graph
erators, the choice of the most suitable intervals where to evaluate represents the correlation value y between the resource measure si
the central tendency is a problem by itself. Here, we consider as at time ti and the measure si+k at time ti+k . A high value of the
a reference interval the period of time during which we generate auto-correlation function means that the two resource measures are
the same number of users (i.e., emulated browsers). For example, highly correlated, which suggests that the resource measure at time
in the step scenario, we have two reference intervals ([0,300] and ti may be used to predict the load at time ti+k . On the other hand,
[301, 600]); in the staircase scenario, we have five reference inter- a low value of the auto-correlation function indicates a jittery be-
vals ([0,120], [121,240], [241, 360], [361,480] and [481,600]), and havior and an increased difficulty of prediction. From this figure,
similarly for the alternating scenario. we can conclude that the resource measure are uncorrelated for any
The simple mean is a good indicator of the central tendency of a scenario. In similar contexts, using an auto-regressive model such
set of data [26], and we use this value as the ideal trend. In Fig- as ARIMA [38] to predict future resource samples may prove com-
ures 5, we plot the ideal trends for the three considered scenar- putationally expensive, because the model parameters need to be
ios, where the monitored resource is the CPU utilization of the DB updated frequently, and are not suitable to support run-time deci-
server measured every second. We define the accuracy of a load sion systems.
tracker in representing the trend of the resource measures as the Hence, we propose the load tracker results as the basis for load
sum of the distances between each load tracker value li computed prediction. In Figures 6(b) and (c), we report the auto-correlation
at time ti and the corresponding value of the ideal trend idi at the diagram of the load tracker values. Let us suppose that the predic-
same instant, that is: tion is carried at the instant ti = 0, and that the prediction window
X of interest for our study is an interval of 30 seconds. In the inter-
∆LT = |li − idi | (8) val [0 − 30], the ACF of resource measures decreases much more
i
abruptly than the ACFs of the two proposed load trackers. This re-
where i spans over the load tracker values computed during the ob- sult is important, because in a context of consecutive values show-
servation period. Load trackers with small distance values follow ing a high correlation degree, the accuracy of the load prediction
the ideal trend with high accuracy. On the other hand, high dis- is more likely. Thanks to the use of a load tracker that provides
tances from the ideal trend values are due to representation delays high auto-correlation among values, even simple linear predictors
or oscillations. These behaviors lead to load trackers that are less should be sufficient to forecast the future behavior of resource load.
accurate and even less suitable to support load predictors. Indeed, previous studies [5, 27, 35] show that simple linear models,
such as the AR model or the linear interpolation, are adequate for
4. LOAD PREDICTION prediction when the correlation of consecutive resource measures
−
→ is high. For example, Dinda shows that the UNIX load average can
A load predictor is a function LPk (Lq (ti )) : Rq → R that takes
−
→ be predicted best through an auto-regressive model that takes into
as its input the set of q values Lq (ti ) = (li−q , . . . , li ), and returns account the last 16 measures (AR(16)), because of its good pre-
a real number that is the predicted value at time ti+k , where k > 0. dictive power and low computational cost [17].
−
→ −
→
In previous studies, the vector Lq (ti ) consists of a set of real re- In this paper, we consider a set of load predictors LPk (Lq (ti )) that
source measures and the load predictors aim to forecast the future are based on the linear regression of two available load tracker val-
resource measure, at time ti+k . On the other hand, we propose ues. Each predictor in this class is characterized by a couple of
that our load predictor takes as its input a set of load tracker val- values: the predicted window k, that represents the size of the pre-
ues and returns a future load tracker value. In a context, where the diction interval, and the past time window q, where q is the size
resource measures obtained from the load monitors of the Internet- −→
of load tracker vector Lq (ti ), that is the distance between the first
based servers are extremely variable, there are two reasons that jus-
and last load tracker value. Let us consider two load tracker
−
→ values:
tify our choice.
the first li−q and the last li . The load predictor LPk (Lq (ti )) of the
• The behavior of a measured resource appears extremely vari- load tracker is the line that intersects the two points (ti−q , li−q )
able to the extent that run-time decisions based on these val- and (ti , li ) and returns l̂i+k that is the predicted value of the load
 1 1 1
Ideal trend Ideal trend Ideal trend
CPU utilization CPU utilization CPU utilization
0.8 0.8 0.8

CPU utilization
CPU utilization

CPU utilization
0.6 0.6 0.6

0.4 0.4 0.4

0.2 0.2 0.2

0 0 0
0 100 200 300 400 500 600 0 100 200 300 400 500 600 0 100 200 300 400 500 600
Time [s] Time [s] Time [s]

(a) Step scenario (b) Staircase scenario (c) Alternating scenario

Figure 5: Ideal trends for the three scenarios.

1 1 1
ACF step 0.1 0.1
ACF stair -0.1 -0.1
ACF alternating ACF CS step (past 30 measures, J=3) ACF EMA step (past 30 measures)
ACF CS stair (past 30 measures, J=3) ACF EMA stair (past 30 measures)
0.5 0.5 ACF CS alternating (past 30 measures, J=3) 0.5 ACF EMA alternating (past 30 measures)

ACF
ACF

ACF
0 0 0

-0.5 -0.5 -0.5

-1 -1 -1
0 100 200 300 400 500 600 0 100 200 300 400 500 600 0 100 200 300 400 500 600
Lag Lag Lag

(a) ACF of the measured values (b) ACF of the CS330 values (c) ACF of the EMA30 values

Figure 6: Autocorrelation function

tracker li+k at time ti+k : 5. RESULTS

8 −→
>
< LPk (Lq (ti )) = m ∗ (ti+k ) + a
m=
li −li−q
(9) 5.1 Accuracy of the load trend representation
>
: a = l q− m ∗ t This section has the twofold goal of analyzing both the accuracy
i−q i−q
of a load tracker in representing the load trend of a set of resource
The strength of a load predictor depends on its accuracy to evaluate measures, and the accuracy of the considered load predictor to fore-
the future values of the load tracker. The common way to measure cast the future load trend. To this purpose, we use the three classes
the accuracy of a prediction is through the evaluation of the rela- of load trackers proposed in Section 3 and the linear load predictor
tive error between a load tracker value li+k and the corresponding proposed in Section 4.
predicted value l̂i+k . A load predictor characterized by a low pre- Let us first evaluate which of the considered load trackers are rep-
diction error is able to evaluate future load tracker accurately. resentative of the considered ideal trends. We are also interested
−
→ to find out the impact of the parameters of the load tracker func-
Let us consider a load tracker LT (Sn (ti )) and a load predictor −
→
−→ −
→ tions, that is, the size n of the measurement vector Sn (ti ), and the
LPk (Lq (ti )) that, at time ti , forecasts LT (Sn (ti+k )) where k >
0. We define the prediction error i+k at time ti+k as the relative number of control points for the cubic splines. To this purpose, in
error between the actual load tracker value li+k and the predicted Figures 7, we report the the distance ∆LT (see Equation (8)) be-
value l̂i+k : tween each load tracker and the corresponding ideal trend for the
step, staircase and alternating scenarios. In the four Figures 8 we
li+k − l̂i+k give a graphical representation of the behavior of some load track-
i+k = | | (10)
li+k ers with respect to the ideal trend values for the staircase scenario.
Small values of i+k indicate a good accordance between li+k and We first consider in Figure 7(a) the non linear load trackers (CS)
that depend on the dimension of the vector of resource measures
lˆi+k . We will compare the accuracy of the load predictor in Equa-
(n) and on the number of control points (J). We carried out a large
tion (9) applied to the load trackers presented in Section 3 by eval-
set of experiments, and we report here some significant results for
uating the mean ∆P of all prediction errors throughout the entire
n = 9, 30, 60 and J = 3, 6, 10, 20 (shown between brackets in the
observation period:
P figure). In general, the ability of a cubic spline based load tracker
i+k to represent the ideal trend, depends on the scenario that it needs
i
∆P = (11) to describe. A load tracker that represents perfectly the ideal trend
M for only one specific scenario and that is inadequate to describe
where M is the length of the entire observation period. different workloads, can not be considered the “best” load tracker.
For this reason, we analyze the distance ∆LT for all the three con-
sidered workload scenarios. The best performing load tracker is
the cubic spline CS330 , because it shows the smaller distance ∆LT
for every considered workload mix. If we also consider the CS
 10 10
CS (past 9 measures, J=3) EMA (past 30 measures)
CS (past 30 measures, J=10) CS (past 30 measures, J=3)
CS (past 60 measures, J=20) SMA (past 30 measures)
CS (past 30 measures, J=3) EMA (past 60 measures)
8 CS (past 60 measures, J=6) 8 SMA (past 60 measures)

6 6
Distance

Distance
4 4

2 2

0 0
Step Staircase Alternating Step Staircase Alternating

(a) Cubic spline load trackers (b) Linear load trackers and best cubic spline

Figure 7: Distances among the load tracker values and the ideal trend points for the three scenarios

curves in Figures 8(c), we can observe that the behavior of the cubic These figures allow us to conclude that the load predictor based on
splines for different sizes of the measure vector is similar. However, EMA30 performs always better than CS330 . The reason of its lower
when the number of resource measures n is kept constant, the dis- prediction error lies in the reduced oscillations of the linear load
tance ∆LT increases with the number of control points J. With a tracker. When the prediction window is smaller, the accuracy of
higher number of control points, the cubic spline is forced to touch load predictor based on EMA30 is considerably higher than that of
more points; as these points are resource measures with high oscil- load predictor based on CS330 (almost 3 times when q = 5).
lations, the resulting cubic curve presents more ripples. Hence, we We also note that small values of q (for example, q = 1) lead to
conclude that it is better to work with a small number of control higher prediction errors ∆P . The reason is that, with a small value
points. of q, the prediction line takes into account only the very recent trend
In Figure 7(b), we report the distances referring to the linear load of the load tracker. If the load tracker is not perfectly smoothed, the
trackers (EMA and SMA) and the best performing cubic spline, prediction error augments. On the other hand, very high values of
that is, the CS330 load tracker working on a set of 30 load measures q lead to prediction lines that weigh past history more than recent
and 3 control points. If we consider Figures 7(b) and 8, we ob- history, thus causing another increased prediction error. It seems
serve a tradeoff between a reduced delay and a reduced degree of that the optimal values of q lie in the central area of the Figures 9.
oscillations. We also notice that the distance ∆LT increases when Let us choose two values of q that allow us to obtain a small pre-
the
−→linear load trackers use a wider resource measurement vector diction error that is, q = 5 for a prediction window of k = 10
(S60 (ti )). This implies a reduced accuracy in representing the load seconds, and q = 15 for a prediction window of k = 30 seconds.
trend, which holds true for all the considered load trackers. When a Figures 8 show the load tracker values and the predicted values for
larger vector of resource measures is used, the linear load trackers the two prediction windows. These figures confirm that the load
introduce a delay. Similarly, a reduced number of resource mea- predictor based on an EMA model has a lower prediction error than
sures reduces the delays, improves the reactivity to load changes, that based on the cubic spline. The difference is noteworthy for a
but it does not smooth out oscillations. Hence, for a good repre- small prediction window of 10 seconds that is, ∆P = 0.065 and
sentation of the load trend, it is necessary to find a value of n that ∆P = 0.16 for the EMA-based and the CS-based predictors, re-
represents a good tradeoff between a reduced delay and a reduced spectively.
degree of oscillations. The exponential moving average EMA30 is On the other hand, the EMA-based predictor could suffer of higher
the load tracker that presents the smallest distance ∆LT and has delays in representing the load trends with respect to the CS model.
results comparable with those of the cubic spline CS330 . For ex- These delays, that are measured as ∆LT distances, may cause a
3
ample, in the staircase scenario we have ∆LT (CS30 ) = 2.25 and reduced reactivity to evidence variations in the load conditions.
∆LT (EM A30 ) = 2.6. In the rest of this section, we evaluate the Hence, we can conclude that the best choice for the load predic-
accuracy of the linear load predictor that is based on these two load tor should depend on the context to which the decision system is
trackers. applied. When the highest accuracy is more important than the re-
activity, the load predictor based on an EMA load tracker model is
5.2 Accuracy of the load predictor preferable; otherwise, we are oriented to recommend a load predic-
tor based on a Cubic Spline load tracker model.
The linear load predictor presented in Section 4 is actually a class
of load predictors that are based on the linear regression of two load
tracker values. Each predictor in this class is characterized by a
couple of values: the predicted window k and the past time win- 6. RELATED WORK
dow q. In this work we address the issue of predicting future resource
First, we aim to choose a good value for the q parameter of the load loads under real-time constraints in the context of Internet-based
predictor, because it drives the choice of the second point used to systems, where the prediction is an important task for some run-
compute the prediction line. To this purpose, in Figures 9 (a) and time decision systems. Previous literature proposes several strate-
(b) we show the prediction error as a function of q for two predic- gies and algorithms for decision systems such load balancing and
tion windows equal to k = 10 and k = 30 seconds, respectively. load sharing [2, 8, 32], overload and admission control [14, 20, 28],
 1 1
Ideal trend Ideal trend
SMA (past 30 measures) EMA (past 30 measures)
SMA (past 60 measures) EMA (past 60 measures)
0.8 0.8
Load tracker

Load tracker
0.6 0.6

0.4 0.4

0.2 0.2

0 0
0 100 200 300 400 500 600 0 100 200 300 400 500 600
Time [s] Time [s]
(a) The SMA load trackers (b) The EMA load trackers
1 1
Ideal trend Ideal trend
CS (past 30 measures, J=10) CS (past 30 measures, J=10)
CS (past 60 measures, J=20) CS (past 30 measures, J=3)
0.8 0.8

Load tracker
Load tracker

0.6 0.6

0.4 0.4

0.2 0.2

0 0
0 100 200 300 400 500 600 0 100 200 300 400 500 600
Time [s] Time [s]
(c) The CS load trackers (d) The CS330 load tracker

Figure 8: Load tracker behaviors with respect to the ideal trend values in the staircase scenario

1 1
Load predictor based on CS (past 30 measures, J=3) Load predictor based on CS (past 30 measures, J=3)
Load predictor based on EMA (past 30 measures) Load predictor based on EMA (past 30 measures)

0.8 0.8
Mean prediction error

Mean prediction error

0.6 0.6

0.4 0.4

0.2 0.2

0 0
2 4 6 8 10 12 14 16 18 20 10 20 30 40 50
Past time window Past time window
(a) Predicted window: 10 seconds (b) Prediction window: 30 seconds

Figure 9: Prediction errors in the staircase scenario

 1 1
EMA (past 30 measures) CS (past 30 measures, J=3)
Load predictor based on EMA (past 30 measures, q=5) Load predictor based on CS (past 30 measures, q=5)

0.8 0.8

Load tracker
Load tracker

0.6 0.6

0.4 0.4

0.2 0.2

0 0
0 100 200 300 400 500 600 0 100 200 300 400 500 600
Time [s] Time [s]
(a) Prediction, 10s in the future - EMA30 (b) Prediction, 10s in the future - CS330
1 1
EMA (past 30 measures) CS (past 30 measures, J=3)
Load predictor based on EMA (past 30 measures, q=15) Load predictor based on CS (past 30 measures, q=15)

0.8 0.8

Load tracker
Load tracker

0.6 0.6

0.4 0.4

0.2 0.2

0 0
0 100 200 300 400 500 600 0 100 200 300 400 500 600
Time [s] Time [s]
(c) Prediction, 30s in the future - EMA30 (d) Prediction, 30s in the future - CS330

Figure 10: Load predictors

job dispatching [30] and redirection even at a geographical scale [9], noisy nature and oscillatory behavior. In this paper, we show that in
that have the common trait of using the resource measures as the heavy-tailed systems, the decisions based on the direct use of these
indication of server load. Even the proposals that adopt a control values may be risky when not completely wrong.
theoretical approach to control the request rate of a system [1, 24]
with the intent to prevent possible overload and/or provide guar-
anteed levels of performance, use direct resource measures (e.g., 7. CONCLUSIONS
the CPU utilization or the average Web object response time) as a In this paper, we have investigated the properties that must be
feedback signal. In this paper, we show that in a context where the possessed by load predictors to support run-time decisions in the
context of a multi-tier Web-based system that is subject to heavy-
resource measures are characterized by high instability and vari- tailed workloads. Our results show that a more regular representa-
ability, the decisions based on the direct use of these values, may tion of the load conditions of a resource is necessary because the
be risky when not completely wrong. In this paper we propose a measures deriving from load monitors show high oscillations. In
two-step approach that first aims to represent the load trend of a this context, even the most sophisticated time series models are in-
resource and then it uses this load trend descriptor as the input of adequate for run-time decision systems, because they would require
the load predictor model. a frequent update of their parameters. For this reason, we propose
a two-step approach that first evaluates the load trend through a
Other papers [5,17,27,35], in non heavy-tail contexts or in a context load tracker function, and then applies the load predictor to the load
where the correlation of subsequent resource measures is high, pro- trend results, instead of working on direct resource measures.
pose a simple linear model to predict the resource load. However,
in the context of heavy-tailed systems such as the one considered
in this paper and when resource measures present low correlation, 8. REFERENCES
more sophisticated approaches are needed to predict the resource [1] T. Abdelzaher, K. G. Shin, and N. Bhatti. Performance guarantees for
measures, such as ARIMA [38]. These models are more expensive Web server end-systems: A control-theoretical approach. IEEE
Trans. Parallel and Distributed Systems, 13(1):80–96, Jan. 2002.
to compute and update, hence they are difficult to use in a real-time
[2] M. Andreolini, M. Colajanni, and M. Nuccio. Scalability of
setting. Moreover, also the theoretical approach [1, 24] that uses content-aware server switches for cluster-based Web information
the resource measures to prevent overload, may be inadequate in a systems. In Proc. of 12th Int’l World Wide Web Conf. (WWW2003),
heavy-tail scenario where the resource measures have an extremely Budapest, HU, May 2003.
 [3] K. Arun, M. S. Squillante, L. Zhang, and J. Poirier. Analysis and load balancing applications. In Proceedings of Performance 1987,
characterization of large-scale Web server access patterns and pages 515–528, North-Holland, The Netherlands, 1987.
performance. World Wide Web, 2(1-2):85–100, Mar. 1999. [21] A. G. Ganek and T. Corbi. The dawning of the autonomic computing
[4] P. Barford and M. E. Crovella. Generating representative Web era. IBM Systems Journal, 42(1):5–18, 2003.
workloads for network and server performance evaluation. In [22] S. Godard. Sysstat: System performance tools for the Linux OS,
Proceedings of the Joint International Conference on Measurement 2004. http://perso.wanadoo.fr/sebastien.godard/.
and modeling of computer systems (ACM SIGMETRICS [23] J. Jung, B. Krishnamurthy, and M. Rabinovich. Flash crowds and
1998/Performance 1998), pages 151–160, Madison, WI, July 1998. denial of service attacks: characterization and implications for CDNs
[5] Y. Baryshnikov, E. Coffman, G. Pierre, D. Rubenstein, M. Squillante, and Web sites. In Proc. of 11th Int’l World Wide Web Conference
and T. Yimwadsana. Predictability of Web server traffic congestion. (WWW2002), May 2002.
In Proc. of 10th Int’l Workshop of Web Content Caching and [24] A. Kamra, V. Misra, and E. M. Nahum. Yaksha: a self-tuning
Distribution (WCW05), Sep. 2005. controller for managing the performance of 3-tiered sites. In
[6] G. Box, G. Jenkins, and G. Reinsel. Time Series Analysis Forecasting Proceedings of Twelfth International Workshop on Quality of Service
and Control. Prentice-Hall, 1994. (IWQOS2004), pages 47–56, June 2004.
[7] H. W. Cain, R. Rajwar, M. Marden, and M. H. Lipasti. An [25] J. O. Kephart and D. M. Chess. The vision of Autonomic Computing.
architectural evaluation of Java TPC-W. In Proc. of the 7th Int.l IEEE Computer, 36(1):41–50, Jan. 2003.
Symposium on High-Performance Computer Architecture [26] D. J. Lilja. Measuring computer performance. A practitioner’s guide.
(HPCA2001), Nuovo Leone, ME, jan 2001. Cambridge University Press, 2000.
[8] V. Cardellini, E. Casalicchio, M. Colajanni, and P. Yu. The state of [27] Y. Lingyun, I. Foster, and J. M. Schopf. Homeostatic and
the art in locally distributed Web-server system. ACM Computing tendency-based CPU load predictions. In Parallel and distributed
Surveys, 2002. processing Symposium, 2003, pages 9–, 2003.
[9] V. Cardellini, M. Colajanni, and P. Yu. Request redirection [28] M. Mitzenmacher. How useful is old information. IEEE Trans.
algorithms for distributed Web systems. IEEE Trans. Parallel and Parallel and Distributed Systems, 11(1):6–20, Jan. 2000.
Distributed Systems, 14(5), May 2003.
[29] MySQL database server, 2005. – http://www.mysql.com/.
[10] E. Cecchet, A. Chanda, S. Elnikety, J. Marguerite, and
[30] Network Weather Service, 2005. – http://nws.cs.ucsb.edu/ewiki/.
W. Zwaenepoel. Performance comparison of middleware
architectures for generating dynamic Web content. In Proc. of 4th [31] T. Oetiker. Rrdtool: a system for displaying time-series data, 2004.
Middleware Conference, Jun 2003. http://oss.oetiker.ch/rrdtool/.
[11] J. Challenger, P. Dantzig, A. Iyengar, M. Squillante, and L. Zhang. [32] V. S. Pai, M. Aron, G. Banga, M. Svendsen, P. Druschel,
Efficiently serving dynamic data at highly accessed Web sites. W. Zwaenepoel, and E. M. Nahum. Locality-aware request
IEEE/ACM Transactions on Networking, 12(2):233+, 2004. distribution in cluster-based network servers. In Proceedings of the
8th ACM Conference on Architectural Support for Programming
[12] X. Chen and J. Heidemann. Flash crowd mitigation via an adaptive
Languages and Operating Systems, pages 205–216, San Jose, CA,
admission control based on application-level measurement. Technical
Oct. 1998.
Report ISI-TR-557, USC/Information Sciences Institute, May 2002.
[33] D. J. Poirier. Piecewise regression using cubic spline. Journal of the
[13] L. Cherkasova and P. Phaal. Session based admission control: a
American Statistical Association, 68(343):515–524, Sep. 1973.
mechanism for improving performance of commercial Web sites. In
Proceedings of the International Workshop on Quality of Service, [34] procps - the /proc file system utilities, 2005.
London, June 1999. http://procps.sourceforge.net/.
[14] L. Cherkasova and P. Phaal. Session-based admission control: a [35] A. Sang and S. Li. A predictability analysis of network traffic. In
mechanism for peak load management of commercial web sites. Proceedings of Nineteenth Annual Joint Conference of the IEEE
IEEE Transactions on Computers, 51(6), June 2002. Computer and Communications Societies (INFOCOM2000), pages
342–351, 2000.
[15] M. E. Crovella, M. S. Taqqu, and A. Bestavros. Heavy-tailed
probability distributions in the World Wide Web. In A Practical [36] The Tomcat Servlet Engine, 2005. –
Guide To Heavy Tails, pages 3–26. Chapman and Hall, New York, http://jakarta.apache.org/tomcat/.
1998. [37] TPC-W transactional Web e-commerce benchmark, 2004. –
[16] M. Dahlin. Interpreting stale load information. IEEE Trans. Parallel http://www.tpc.org/tpcw/.
and Distributed Systems, 11(10):1033–1047, Oct. 2000. [38] N. Tran and D. Reed. Automatic ARIMA time series modeling for
[17] P. Dinda and D. O’Hallaron. Host load prediction using linear adaptive I/O prefetching. IEEE transaction on parallel and
models. Cluster Computing, 3(4):265–280, december 2000. distributed systems, 15(4):362–377, Apr. 2004.
[18] R. C. Dodge, D. A. Menascé, and D. Barbará. Testing e-commerce [39] J. Wildstrom, P. Stone, E. Witchel, R. Mooney, and M. Dahlin.
site scalability with TPC-W . In Proc. of 2001 Computer Towards self-configuring hardware for distributed computer systems.
Measurement Group Conference, Dec 2001. In Proc. of the Second International Conference on Autonomic
Computing (ICAC2005), June 2005.
[19] R. L. Eubank and E. Eubank. Non parametric regression and spline
smoothing. Marcel Dekker, 1999. [40] G. Wolber and I. Alfy. Monotonic cubic spline interpolation. In
Computer Graphics International, pages 188–195, Canmore, Alta.,
[20] D. Ferrari and S. Zhou. An empirical investigation of load indices for
Canada, July 1999.