LOVELY PROFESSIONAL UNIVERSITY

Algorithm design and analysis Topic: Amortized analysis: Aggregate method

Submitted to: Mr. Vijay Kumar Garg (CSE408)

Submitted by: Navdeep Singh RK1R08B42 10804943

Acknowledgment

First of all I am heartily thankful to my teacher MR. Vijay Kumar Garg who stand a chance for me to get more knowledge about the topic Amortized analysis: Aggregate method I am also thankful to my friends and my parents who supported me in completion of this term paper. And last but not the least very much thankful to the University Providing Internet Facility inside the campus and providing excellent library facilities hence making the work easier.

Navdeep Singh BTECH MTECH (CSE)

Introduction Amortized analysis is a method of analyzing algorithms that considers the entire sequence of operations of the program. It allows for the establishment of a worst-case bound for the performance of an algorithm irrespective of the inputs by looking at all of the operations. At the heart of the method is the idea that while certain operations may be extremely costly in resources, they cannot occur at a high-enough frequency to weigh down the entire program because the number of less costly operations will far outnumber the costly ones in the long run, "paying back" the program over a number of iterations. It is particularly useful because it guarantees worst-case performance rather than making assumptions about the state of the program. Amortized analysis initially emerged from a method called aggregate analysis, which is now subsumed by amortized analysis. However, the technique was first formally introduced by Robert Tarjan in his paper Amortized Computational Complexity, which addressed the need for a more useful form of analysis than the common probabilistic methods used. Amortization was initially used for very specific types of algorithms, particularly those involving binary trees and union operations. However, it is now ubiquitous and comes into play when analyzing many other algorithms as well. In an amortized analysis, the time required to perform a sequence of data-structure operations is averaged over all the operations performed. Amortized analysis can be used to show that the average cost of an operation is small, if one averages over a sequence of operations, even though a single operation might be expensive. Amortized analysis differs from average-case analysis in that probability is not involved; an amortized analysis guarantees the average performance of each operation in the worst case. Key ideas Amortized analysis is an upper bound: it's the average performance of each operation in the worst case. Amortized analysis is concerned with the overall cost of a sequence of operations. It does not say anything about the cost of a specific operation in that sequence. For example, it is invalid to reason, The amortized cost of insertion into a splay tree with n items is O(log n), so when I insert '45' into this tree, the cost will be O(log n). In fact, inserting '45' might require O(n)

operations! It is only appropriate to say, When I insert m items into a tree, the average time for each operation will be O(log n). Amortized analysis is concerned with the overall cost of arbitrary sequences. An amortized bound will hold regardless of the specific sequence; for example, if the amortized cost of insertion is O(log n), it is so regardless of whether you insert the sequence '10,' '160,' '2' or the sequence '2', '160', '10,' '399', etc. The two points above imply that both amortized and worst-case bounds should be understood when choosing an algorithm to use in practice. Because an amortized bound says nothing about the cost of individual operations, it may be possible that one operation in sequence requires a huge cost. Practical systems in which it is important that all operations have low and/or comparable costs may require an algorithm with a worse amortized cost but a better worst-case per operation bound. Amortized analysis can be understood to take advantage of the fact that some expensive operations may pay for future operations by somehow limiting the number or cost of expensive operations that can happen in the near future. If good amortized cost is a goal, an algorithm may be designed to explicitly perform this clean up during expensive operations Comparison to other analysis techniques As mentioned above, worst-case analysis can give overly pessimistic bounds for sequences of operations, because such analysis ignores interactions among different operations on the same data structure (Tarjan 1985). Amortized analysis may lead to a more realistic worst-case bound by taking these interactions into account. Note that the bound offered by amortized analysis is, in fact, a worst-case bound on the average time per operation; a single operation in a sequence may have cost worse than this bound, but the average cost over all operations in any valid sequence will always perform within the bound. Amortized analysis is similar to average-case analysis, in that it is concerned with the cost averaged over a sequence of operations. However, average-case analysis relies on probabilistic assumptions about the data structures and operations in order to compute an expected running time of an algorithm. Its applicability is therefore dependent on certain assumptions about probability distributions on algorithm inputs, which means the analysis is invalid if these assumptions do not hold (or that probabilistic analysis cannot be used at all, if input distribution s cannot be described!) (Cormenet al. 2001, 923). Amortized analysis needs no such assumptio ns. Also, it offers an upper-bound on the worst case running time of a sequence of operations, and this bound will always hold. An average-case bound, on the other hand, does not preclude the possibility that one will get unlucky and encounter an input that requires much more than the expected computation time, even if the assumptions on the distribution of inputs are valid.

These differences between probabilistic and amortized analysis therefore have important consequences for the interpretation and relevance of the resulting bounds. Amortized analysis is closely related to competitive analysis, which involves comparing the worst-case performance of an online algorithm to the performance of an optimal off-line algorithm on the same data. Amortization is useful because competitive analysis's performance bounds must hold regardless of the particular input, which by definition is seen by the online algorithm in sequence rather than at the beginning of processing. Sleator and Tarjan (1985a) offer an example of using amortized analysis to perform competitive analysis. Types:

Amortized analysis is of three types: 1) Aggregate analysis 2) Accounting method 3) Potential method Aggregate analysis, in which we determine an upper bound T(n) on the total cost of a sequence of n operations. The amortized cost per operation is then T(n)/n. Accounting method, in which we determine an amortized cost of each operation. When there is more than one type of operation, each type of operation may have a different amortized cost. The accounting method overcharges some operations early in the sequence, storing the overcharge as "prepaid credit" on specific objects in the data structure. The credit is used later in the sequence to pay for operations that are charged less than they actually cost. The potential method, which is like the accounting method in that we determine the amortized cost of each operation and may overcharge operations early on to compensate for undercharges later. The potential method maintains the credit as the "potential energy" of the data structure instead of associating the credit with individual objects within the data structure. We shall use two examples to examine these three models. One is a stack with the additional operation MULTIPOP, which pops several objects at once. The other is a binary counter that counts up from 0 by means of the single operation INCREMENT. The aggregate analysis method In the aggregate method of amortized analysis, we show that for all n, a sequence of n operations takes worst-case time T(n) in total. In the worst case, the average cost, or amortized cost, per operation is therefore T(n) / n. Note that this amortized cost applies to each operation, even when there are several types of operations in the sequence. The other two

methods we shall study in this chapter, the accounting method and the potential method, may assign different amortized costs to different types of operations. Stack operations In our first example of the aggregate method, we analyze stacks that have been augmented with a new operation. PUSH(S, x) pushes object x onto stack S. POP(S) pops the top of stack S and returns the popped object. Since each of these operations runs in O(1) time, let us consider the cost of each to be 1. The total cost of a sequence of n PUSH and POP operations is therefore n, and the actual running time for n operations is therefore (n). The situation becomes more interesting if we add the stack operation MULTIPOP(S, k), which removes the k top objects of stack S, or pops the entire stack if it contains less than k objects. In the following pseudo code, the operation STACK-EMPTY returns TRUE if there are no objects currently on the stack, and FALSE otherwise. MULTIPOP(S,k) 1 while not STACK-EMPTY(S) and k 2 do POP(S) 3 k k-1

Figure 1 shows an example of MULTIPOP. What is the running time of MULTIPOP(S, k) on a stack of s objects? The actual running time is linear in the number of POP operations actually executed, and thus it suffices to analyze MULTIPOP in terms of the abstract costs of 1 each for PUSH and POP. The number of iterations of the while loop is the number min(s, k) of objects popped off the stack. For each iteration of the loop, one call is made to POP in line 2. Thus, the total cost of MULTIPOP is min(s, k), and the actual running time is a linear function of this cost.

Figure 1 The action of MULTIPOP on a stack S, shown initially in (a). The top 4 objects are popped by MULTIPOP(S, 4), whose result is shown in (b). The next operation is MULTIPOP(S, 7), which empties the stack shown in (c) since there were fewer than 7 objects remaining. Let us analyze a sequence of n PUSH, POP, and MULTIPOP operations on an initially empty stack. The worst-case cost of a MULTIPOP operation in the sequence is O(n), since the stack size is at most n. The worst-case time of any stack operation is therefore O(n), and hence a sequence of n operations costs O(n2), since we may have O(n) MULTIPOP operations costing O(n) each. Although this analysis is correct, the O(n2) result, obtained by considering the worstcase cost of each operation individually, is not tight. Using the aggregate method of amortized analysis, we can obtain a better upper bound that considers the entire sequence of n operations. In fact, although a single MULTIPOP operation can be expensive, any sequence of n PUSH, POP, and MULTIPOP operations on an initially empty stack can cost at most O(n). Why? Each object can be popped at most once for each time it is pushed. Therefore, the number of times that POP can be called on a nonempty stack, including calls within MULTIPOP, is at most the number of PUSH operations, which is at most n. For any value of n, any sequence of n PUSH, POP, and MULTIPOP operations takes a total of O(n) time. The amortized cost of an operation is the average: O(n)/n = O(1). We emphasize again that although we have just shown that the average cost, and hence running time, of a stack operation is O(1), no probabilistic reasoning was involved. We actually showed a worst-case bound of O(n) on a sequence of n operations. Dividing this total cost by n yielded the average cost per operation, or the amortized cost.

Incrementing a binary counter As another example of the aggregate method, consider the problem of implementing a k-bit binary counter that counts upward from 0. We use an array A[0 . . k - 1] of bits, where length[A] = k, as the counter. A binary number x that is stored in the counter has its lowest-order bit in A[0] and its highest-order bit in A[k - 1], so that . Initially, x = 0, and thus A[i] = k 0 for i = 0, 1, . . . , k - 1. To add 1 (modulo 2 ) to the value in the counter, we use the following procedure.

Amortized cost per operation is T(n)/n

Figure 2 An 8-bit binary counter as its value goes from 0 to 16 by a sequence of 16 INCREMENT operations. Bits that flip to achieve the next value are shaded. The running cost for flipping bits is shown at the right. Notice that the total cost is never more than twice the total number of INCREMENT operations. INCREMENT (A) 1 i 0 2 while i< length[A] and A[i] = 1 3 do A[i] 0 4 i i+1 5 if i < length[A] 6 then A[i] 1 This algorithm is essentially the same one implemented in hardware by a ripple-carry counter Figure 2 shows what happens to a binary counter as it is incremented 16 times, starting with the initial value 0 and ending with the value 16. At the start of each iteration of the while loops in lines 2-4, we wish to add a 1 into position i. If A[i] = 1, then adding 1 flips the bit to 0 in position i and yields a carry of 1, to be added into position i + 1 on the next iteration of the loop. Otherwise, the loop ends, and then, if i < k, we know that A[i] = 0, so that adding a 1 into position i, flipping the 0 to a 1, is taken care of in line 6. The cost of each INCREMENT operation is linear in the number of bits flipped. As with the stack example, a cursory analysis yields a bound that is correct but not tight. A single execution of INCREMENT takes time (k) in the worst case, in which array A contains all 1's. Thus, a sequence of n INCREMENT operations on an initially zero counter takes time O(nk) in the worst case. We can tighten our analysis to yield a worst-case cost of O(n) for a sequence of n INCREMENT'S by observing that not all bits flip each time INCREMENT is called. As Figure 2 shows, A[0] does flip each time INCREMENT is called. The next-highest-order bit, A[1], flips only every other time:

a sequence of n INCREMENT operations on an initially zero counter causes A[1] to flip n/2 times. Similarly, bit A[2] flips only every fourth time, or n/4 times in a sequence of n INCREMENT'S. In general, for i = 0, 1, . . . , lg n , bit A[i] flips n/2i times in a sequence of n INCREMENT operations on an initially zero counter. For i > lg n , bit A[i] never flips at all. The total number of flips in the sequence is thus

by equation (3.4). The worst-case time for a sequence of n INCREMENT operations on an initially zero counter is therefore O(n), so the amortized cost of each operation is O(n)/n = O(1).

Conclusion Amortized analysis is a useful tool that complements other techniques such as worst-case and average-case analysis. It has been applied to a variety of problems, and it is also crucial to appreciating structures such as splay trees that have been designed to have good amortized bounds. To understand the application of amortized analysis to common problems, it is essential to know the basics of both the accounting method and the potential method. The resources presented here supply many examples of both methods applied to real problems. To perform an amortized analysis, one should choose either the accounting method or the potential method. The approaches yield equivalent results, but one might be more intuitively appropriate to the problem under consideration. There is no magic formula for arriving at a potential function or accounting credit scheme that will always work; the method used depends on the desired bounds and the desired complexity of the analysis. Some strategies that sometimes work, however, include enumerating the ways in which an algorithm might operate on a data structure, then performing an analysis for each case; computing the potential of a data structure as a sum of local potentials so that one can reason about the effects of local changes while ignoring irrelevant and unchanging components of the structure; designing the potential method around the desired form of the result (e.g., relating the potential to the lo g of the subtree size for splay trees, as the desired outcome is a logarithmic bound); and reason ing about each type of operation in a sequence individually before coming up with a bound on a n arbitrary sequence of operations. An understanding of amortized analysis is essential to success in an algorithms course, to understanding the implication of theoretical bounds on real-world performance, and to thoroughly appreciating the design and purpose of certain data structures. The reader is theref

ore again urged to consult any of the sources mentioned here to improve his or her understanding of amortized analysis and to explore these algorithms in greater depth.

References:1) http://www.eli.sdsu.edu/courses/fall95/cs660/notes/amortized/Amortized.html#RTF ToC3 2) http://staff.ustc.edu.cn/~csli/graduate/algorithms/book6/chap18.htm 3) http://www.bowdoin.edu/~ltoma/teaching/cs231/fall11/Lectures/13amortized/amortized.pdf