Caching on the World Wide Web - Charu Aggarwal

16 Pages·2011·0.18 MB·English

Caching on the World Wide Web
Charu Aggarwal(cid:2) Joel Wolf and Philip Yu
IBM T(cid:3)J(cid:3) Watson Research Center(cid:2) Yorktown Heights(cid:2) New York
Abstract
Withtherecentexplosioninusageoftheworldwideweb(cid:2)theproblemofcachingwebobjects
has gained considerable importance(cid:3) Caching on the web di(cid:4)ers from traditional caching in
several ways(cid:3) The non(cid:5)homogeneity of the object sizes is probably the most important such
di(cid:4)erence(cid:3) In this paper we give an overview of caching policies designed speci(cid:6)cally for web
objects and provide a new algorithm of our own(cid:3) This new algorithm can be regarded as a
generalization of the standard LRU algorithm(cid:3) We examine the performance of this and other
web caching algorithmsvia event and trace driven simulation(cid:3)
(cid:2) Introduction
The recent increase in popularity of the world wide web has led to a considerable increase in the
amount of tra(cid:2)c over the internet(cid:3) As a result(cid:4) the web has now become one of the primary
bottlenecks to network performance(cid:3) When objects are requested by a user who is connected to a
server on a slow network link(cid:4) there is generally considerable latency noticeable at the client end(cid:3)
Further(cid:4)transferringtheobjectoverthenetworkleads toanincrease in thelevel oftra(cid:2)c(cid:3) This has
the e(cid:5)ect of reducing the bandwidth available for competing requests(cid:4) and thus increasing latencies
for other users(cid:3) In order to reduce access latencies(cid:4) it is desirable to store copies of popular objects
closer to the user(cid:3)
Consequently(cid:4)webcachinghasbecomeanincreasingly importanttopic(cid:6)(cid:7)(cid:4)(cid:8)(cid:4)(cid:7)(cid:9)(cid:4)(cid:7)(cid:8)(cid:4)(cid:7)(cid:10)(cid:4)(cid:11)(cid:11)(cid:12)(cid:3) Caching
canbe implemented atvariouspointsin thenetwork(cid:3) Ononeend ofthespectrum(cid:4) thereis typically
a cache in the web server itself(cid:3) Further(cid:4) it is increasingly common for a university or corporation
to implement specialized servers in the network called caching proxies(cid:3) Such proxies act as agents
on behalf of the client in order to locate a cached copy of a object if possible(cid:3) More information on
caching proxies maybe found in (cid:6)(cid:7)(cid:13)(cid:12)(cid:3) Usually caching proxies and web serversbehave as secondary
or higher level caches(cid:4) because they are concerned only with misses left over from client caches(cid:3)
Such client caches are built into the web browsers themselves(cid:3) They may store only those accesses
from the current invocation (cid:14)so(cid:15)called non(cid:2)persistent cache(cid:16)(cid:4) or they may retain objects between
invocations(cid:3) Mosaic(cid:4) for example(cid:4) uses a non(cid:15)persistent cache(cid:3)
Inthis paper(cid:4)weshall discussgeneral mainmemorycachereplacement policies designed speci(cid:17)cally
for use by web caches(cid:3) The results are applicable to web server(cid:4) proxy and client caches(cid:3)
One of the key complications in implementing cache replacement policies for web objects is that
the objects to be cached are not necessarily of homogeneous size(cid:3) For example(cid:4) if two objects are
accessed with equal frequency(cid:4) the hit ratio is maximized when the replacement policy is biased
towards the smaller object(cid:3) This is because it is possible to store a larger number of objects of
smaller size(cid:3) In the standard least recently used (cid:3)LRU(cid:4) caching algorithm for equal sized objects
we maintain a list of the objects in the cache which is ordered based on the time of last access(cid:3)
In particular(cid:4) the most recently accessed object is at the top of the list(cid:4) while the least recently
accessed object is at the bottom(cid:3) When a new object comes in and the cache is full(cid:4) one object in
the cache must be pruned in order to make room for the newly accessed object(cid:3) The object chosen
is the one which was least recently used(cid:3) Clearly the LRU policy needs to be extended to handle
objects of varying sizes(cid:3)
In addition to non(cid:15)homogeneous object sizes(cid:4) there are several other special features of the web
whichneedtobeconsidered(cid:3) First(cid:4)thehitratiomaynotbethebestpossible measureforevaluating
the quality of a web caching algorithm(cid:3) For example(cid:4) the transfer time cost for transfering a large
objectis morethan thatforasmall object(cid:4) thoughthe relationship is typically notstraightforward(cid:3)
It will depend(cid:4) for instance(cid:4) on the distance of the object from the web server(cid:3) Furthermore(cid:4) web
objects will typically have expiration times(cid:3) So(cid:4) when considering which objects to replace when a
new object enters a web cache(cid:4) we must consider not only the relative frequency(cid:4) but also factors
such as object sizes(cid:4) transfer time savings and expiration times(cid:3)
A related issue to that of replacement is admission control(cid:3) In other words(cid:4) when should we allow
(cid:7)
an object to enter the cache at all(cid:18) It may not always be favorable to insert an object into the
cache(cid:4) because it may lower the probability of a hit to the cache(cid:3)
We list below some cache replacement policies evaluated in (cid:6)(cid:7)(cid:4) (cid:11)(cid:19)(cid:12)(cid:3) Each of them can be combined
with an admission policy(cid:3)
(cid:14)(cid:7)(cid:16) LRU(cid:2) In the most straightforward extension of LRU for handling non(cid:15)homogeneous sized
objects(cid:4) one would prune o(cid:5) as many of the least recently used objects as is necessary to
have su(cid:2)cient space for the newly accessed object(cid:3) This may involve zero(cid:4) one or many
replacements(cid:3) Thus(cid:4) this extension of LRU takes size into account only peripherally while
performing the cache replacement decisions(cid:3) As we shall see(cid:4) such a replacement policy turns
out to be naive in practice(cid:3)
(cid:14)(cid:11)(cid:16) LRUMIN(cid:2)Thispolicy isbiasedinfavorofsmallersizedobjectssoastominimize thenumber
of objects replaced(cid:3) Let the size of the incoming object be S(cid:3) Suppose that this object will
not(cid:17)tin thecache(cid:3) Ifthereareanyobjectsin the cachewhich havesize atleast S(cid:4)weremove
the least recently used such object from the cache(cid:3) If there are no objects with size at least
S(cid:4) then we start removing objects in LRU order of size at least S(cid:2)(cid:11)(cid:4) then objects of size at
least S(cid:2)(cid:9)(cid:4) and so on until enough free cache space has been created(cid:3)
(cid:14)(cid:20)(cid:16) SIZE policy(cid:2) In this policy(cid:4) the objects are removed in order of size(cid:4) with the largest object
removed (cid:17)rst(cid:3) Ties based on size are somewhatrare(cid:4) but when they occur they are broken by
considering the time since last access(cid:3) Speci(cid:17)cally(cid:4) objects with higher time since last access
are removed (cid:17)rst(cid:3)
Note that all of these policies take into account either the size or the time since last access or both(cid:3)
It was concluded in (cid:6)(cid:7)(cid:4) (cid:11)(cid:19)(cid:12) that policies which take into account the size tend to perform better
than those which do not(cid:3) This is because removing larger objects makes room for multiple smaller
ones(cid:3)
In this paper we devise a web cache replacement policy which appears to achieve performance
better than any of the above schemes(cid:3) We describe a corresponding admission control policy as
well(cid:3) The scheme we propose is quite general purpose(cid:4) is easy to implement(cid:4) and works well on
many di(cid:5)erent kinds of workloads(cid:3)
We brie(cid:21)y survey and categorize some additional cache replacement schemes for the web(cid:3) The list
below is certainly not exhaustive(cid:4) though many replacement algorithms can be classi(cid:17)ed into one
or more of the following categories(cid:3)
(cid:14)(cid:7)(cid:16) Direct extensions of traditional policies(cid:2) Besides LRU(cid:4) traditional policies such asLeast
Frequently Used (cid:3)LFU(cid:4) and First In First Out (cid:3)FIFO(cid:4) are well(cid:15)known cache replacement
strategies for paging scenarios (cid:6)(cid:11)(cid:9)(cid:12)(cid:3) Just as with LRU(cid:4) it is possible to extend these policies
to handle objects of non(cid:15)homogeneous size(cid:3) The policy in (cid:6)(cid:11)(cid:11)(cid:12) can be regarded as an LRU
extension(cid:4) though time since last access is rounded to the nearest day(cid:3) The di(cid:2)culty with
such policies in general is that they fail to pay su(cid:2)cient attention to object sizes(cid:3)
(cid:14)(cid:11)(cid:16) Key(cid:3)based policies(cid:2) The idea in key(cid:15)based policies is to sort objects based upon a primary
key(cid:4) break ties based on a secondary key(cid:4) break remaining ties based on a tertiary key(cid:4) and
(cid:11)
Name Primary Key Secondary Key Tertiary Key
LRU Time Since Last Access
FIFO Entry Time of Object in Cache
LFU Frequency of Access
SIZE Size Time Since Last Access
LOG(cid:11)(cid:15)SIZE blog(cid:0)(cid:14)Size(cid:16)c Time Since Last Access
HYPER(cid:15)G Frequency of Access Time Since Last Access Size
Table (cid:7)(cid:22) Some examples of Key Based Policies
so on(cid:3) This class of policies by proposed in (cid:6)(cid:11)(cid:19)(cid:12)(cid:3) For example(cid:4) a policy called LOG(cid:5)(cid:2)SIZE
discussed in (cid:6)(cid:11)(cid:19)(cid:12) uses blog(cid:0)fSizegc as the primary key and the time since last access as the
secondary key(cid:3) The HYPER(cid:2)G policy uses frequency of access asthe primary key(cid:4) breaks ties
using the recency of last use(cid:4) and then (cid:17)nally uses size as the tertiary key(cid:3) Some additional
examples of key(cid:15)based policies are illustrated in Table (cid:7)(cid:3) The idea in using the key based
policies is to prioritize some replacement factors over others(cid:3) However(cid:4) such prioritization
may not always be ideal(cid:3)
(cid:14)(cid:20)(cid:16) Function(cid:3)based replacement policies(cid:2) The idea in function(cid:15)based replacement policies is
toemploy a potentially general function of the di(cid:5)erent factorssuch as time since last access(cid:4)
entrytimeoftheobjectin thecache(cid:4) transfertimecost(cid:4)objectexpiration timeand soon(cid:3) For
example(cid:4) the algorithm described in (cid:6)(cid:13)(cid:12) employs a weighted rational function of the transfer
time cost(cid:4) the size and the time since last access(cid:3) The algorithm described in (cid:6)(cid:11)(cid:13)(cid:12) employs a
weighted exponential function of the access frequency(cid:4) the size(cid:4) the latency to the server and
the bandwidth to the server(cid:3) The Least Normalized Cost Replacement (cid:3)LNC(cid:2)R(cid:4) algorithm
described in (cid:6)(cid:11)(cid:20)(cid:12) employs a rational function of the access frequency(cid:4) the transfer time cost
and the size(cid:3) This algorithm is certainly the most similar to our own scheme(cid:4) which is also
function(cid:15)based(cid:3)
The paper is organized as follows(cid:3) In Section (cid:11) we concentrate on formulating a theoretical op(cid:15)
timization model for web caching which generalizes LRU(cid:3) We devise an optimization model and
show how this can be approximately solved by a simple heuristic(cid:3) We call the policy derived here
Size(cid:2)adjusted LRU(cid:4) or SLRU(cid:3) In Section (cid:20) we show how to make this heuristic more easily imple(cid:15)
mentable in practice(cid:3) We call the resulting policy the Pyramidal Selection Scheme(cid:4) or PSS(cid:3)Section
(cid:9) describes web(cid:15)speci(cid:17)c extensions to the above two algorithms in order to handle general access
costs and expiration times(cid:3) In Section (cid:23) we discuss an admission control policy(cid:3) Results of event
and trace driven simulations are presented in Section (cid:19)(cid:3) In particular(cid:4) we compare PSS with the
LRU(cid:4) LRUMIN and SIZE schemes(cid:3) Finally in Section (cid:13)(cid:4) we present a summary and conclusion(cid:3)
(cid:3) Generalized LRU Replacement
When an object is to be inserted into the cache(cid:4) more than one object may need to be removed
in order to create su(cid:2)cient space(cid:3) In the LRU extension discussed in (cid:6)(cid:7)(cid:4) (cid:11)(cid:19)(cid:12)(cid:4) objects are greedily
removed from the cache in the order of recency of last access until enough space is created for
(cid:20)
the incoming object(cid:3) But such a policy is not the only possible LRU generalization for handling
objects of non(cid:15)uniform size(cid:3) In this section we describe the theoretical foundations of another such
policy(cid:3) Speci(cid:17)cally(cid:4) we de(cid:17)ne and heuristically solve an optimization problem which mimics but
generalizes the LRU criteria for uniform sized objects(cid:3)
First we shall need some notation(cid:3) We assume that there are N objects(cid:4) and that object i has size
Si(cid:3) A counter is maintained and incremented each time there is a request for an object(cid:3) The set
of objects in the cache at the kth iteration is denoted by C(cid:14)k(cid:16)(cid:3) Let ik denote the object accessed
at the kth iteration(cid:3) If ik is present in the cache in the (cid:14)k(cid:2)(cid:7)(cid:16)st iteration we have a hit(cid:4) and it
does not need to be brought into the cache(cid:3) On the other hand(cid:4) if ik is not present then we have a
miss(cid:3) Assuming ik satis(cid:17)es the admission control requirements(cid:4) we have to decide which objects to
purge from the cache(cid:3) Let R (cid:3) (cid:24) denote the amount of additional space in the cache which must
be created in order to accommodate ik(cid:4) an easy calculation(cid:3) Consider the decision variable yi for
object i de(cid:17)ned to be (cid:7) if we wish to purge it(cid:4) and (cid:24) if we want to retain it(cid:3) The decision variable
yi is de(cid:17)ned only for objects which are present in the cache(cid:3) We assume that (cid:25)Tik is the number
of accesses since the last time object i was accessed(cid:3) This number is well(cid:15)de(cid:17)ned for all objects
which have been accessed before(cid:3) We shall refer to (cid:7)(cid:2)(cid:25)Tik as the dynamic frequency of object i at
iteration k(cid:3)
Note that the LRU policy for uniform size objects removes the object with the smallest dynamic
frequency fromthe cache(cid:4) and thustends toretain the objectswithhigh frequency ofaccess(cid:3) While
the dynamic frequency is an imperfect estimator of the true frequency of an object(cid:4) LRU turns out
to be a very robust algorithm in practice(cid:4) at least for the case of uniform size objects(cid:3)
Roughly speaking(cid:4) for non(cid:15)uniform sized objects we would like the sum of dynamic frequencies for
the outgoing objects to be as small as possible(cid:3) Speci(cid:17)cally(cid:4) we have the following model(cid:22)
Minimize X yi(cid:2)(cid:25)Tik
i(cid:0)C(cid:2)k(cid:3)
such that X Si(cid:4)yi (cid:3) R
i(cid:0)C(cid:2)k(cid:3)
and yi (cid:5) f(cid:24)(cid:3)(cid:7)g(cid:4)
The above mathematical programming problem is a version of the knapsack problem (cid:6)(cid:9)(cid:12)(cid:3) (cid:14)Said
precisely(cid:4) the objects we place in the knapsack are actually those which will be purged from the
cache(cid:3)(cid:16) The knapsack problem is known to be NP(cid:15)hard(cid:3) However(cid:4) there exist fast heuristics which
do well in practice(cid:3) One well(cid:15)known knapsack problem heuristic is the following greedy policy(cid:22)
Order the objects by the ratio of cost to size(cid:3) Then choose the objects with the best cost(cid:15)to(cid:15)size
ratio(cid:4) one by one(cid:4) until no more can (cid:17)t into the knapsack(cid:3) The cost(cid:15)to(cid:15)size ratio for the object
i is (cid:7)(cid:2)(cid:14)Si(cid:4)(cid:25)Tik(cid:16)(cid:3) So(cid:4) we reindex the objects (cid:7)(cid:3)(cid:11)(cid:3)(cid:4)(cid:4)(cid:4)(cid:3)jC(cid:14)k(cid:16)j in order of non(cid:15)decreasing values of
Si(cid:4)(cid:25)Tik(cid:3) After sorting we have(cid:22)
S(cid:4)(cid:4)(cid:25)T(cid:4)k (cid:6) S(cid:0)(cid:4)(cid:25)T(cid:0)k (cid:6) (cid:4)(cid:4)(cid:4)(cid:6) SjC(cid:2)k(cid:3)j(cid:4)(cid:25)TjC(cid:2)k(cid:3)jk(cid:4)
Then we greedily pick thehighest index objectsone by one and purge themfromthe cache until we
(cid:9)
Figure (cid:7)(cid:22) The Pyramidal Selection Scheme
havecreatedsu(cid:2)cientspace fortheincoming object(cid:3) Wecall thisreplacement schemeSize(cid:2)adjusted
LRU(cid:4) or SLRU(cid:3)
(cid:4) The Pyramidal Selection Scheme
WeshouldnotethattheSLRUpolicydescribed intheprevioussectionmaybesomewhatunrealistic
to implement in practice(cid:4) because of the di(cid:2)culty in comparing the product of the size and the
time since last access for every object in the cache(cid:3) A somewhat more practical variant(cid:4) known as
the Pyramidal Selection Scheme(cid:4) or PSS(cid:4) will be described in this section(cid:3)
The primary idea behind the PSS scheme is that we make a pyramidal classi(cid:17)cation of objects
i(cid:2)(cid:4) i
depending upon their size(cid:3) All objects of group i will have sizes ranging between (cid:11) and (cid:11) (cid:2)(cid:7)(cid:3)
Thus there will be N (cid:26) dlog(cid:14)M(cid:27)(cid:7)(cid:16)e di(cid:5)erent groups of objects(cid:4) where M is the cache size(cid:3) For the
cache(cid:4) as illustrated in Figure (cid:7)(cid:4) the objects in each group i are maintained as a separate LRU list(cid:3)
Whenever we need to decide which object to eject from the cache(cid:4) we compare the Si(cid:4)(cid:25)Tik values
of only the least recently used objects in each group(cid:3) The result of using this mechanism is that
we will choose the object with the largest overall value of Si(cid:4)(cid:25)Tik to within a factor of (cid:7)(cid:28)(cid:11)(cid:4) even
in the worst case(cid:3)
(cid:3)
Theorem (cid:4)(cid:5)(cid:6) Let Z be the least value of Si (cid:4)(cid:25)Tik among all the objects i in the cache(cid:6) and let
(cid:5) (cid:5) (cid:3)
Z be the corresponding value for the PSS scheme(cid:7) Then Z (cid:3) (cid:14)(cid:7)(cid:2)(cid:11)(cid:16)Z (cid:7)
Proof(cid:2) Let us de(cid:17)ne the PSS group leaders as the set of least recently used objects in each of the
N groups(cid:3) Let j be the object among these group leaders chosen by the PSS scheme(cid:4) and let m be
the object with the optimal value of Si(cid:4)(cid:25)Tik when request k is received(cid:3) Let l be the least recently
used object(cid:14)group leader(cid:16) in the group towhich object m belongs(cid:3) Then(cid:4) we musthave Sl (cid:3) Sm(cid:2)(cid:11)
(cid:23)
and (cid:25)Tlk (cid:3) (cid:25)Tmk(cid:3) Consequently(cid:4) we must have Sl (cid:4) (cid:25)Tlk (cid:3) Sm (cid:4) (cid:25)Tmk(cid:2)(cid:11)(cid:3) But(cid:4) since j is the
optimal object within the set of group leaders and l is also a group leader(cid:4) it must be the case that
Sj (cid:4)(cid:25)Tjk (cid:3) Sl(cid:4)(cid:25)Tlk(cid:3) Combining the above two inequalities(cid:4) we get that Sj (cid:4)(cid:25)Tjk (cid:3) Sm(cid:4)(cid:25)Tmk(cid:2)(cid:11)(cid:3)
(cid:5) (cid:3)
In other words(cid:4) Z (cid:3) (cid:14)(cid:7)(cid:2)(cid:11)(cid:16)Z (cid:3)
(cid:5) (cid:3)
In reality(cid:4) the value of Z is typically so close to Z that the performance di(cid:5)erence between the
PSS scheme(cid:4) and a policy which uses a direct size(cid:15)time product is almost imperceptible(cid:3) We shall
illustrate a comparison of the hit ratios of these two policies in Section (cid:19)(cid:3)
Notice that the LOG(cid:11)(cid:15)SIZE discussed in (cid:6)(cid:11)(cid:19)(cid:12) bears at least some resemblance to the independently
derived PSS scheme(cid:22) The LOG(cid:11)(cid:15)SIZE scheme always chooses the least recently used items in the
non(cid:15)empty stacks corresponding to the largest size ranges(cid:3) In contrast(cid:4) the PSS scheme looks at
the least recently used objects of each stack(cid:4) and among these picks the objects which have the
least product of the Si(cid:4)(cid:25)Tik(cid:3)
(cid:5) Web(cid:6)Speci(cid:7)c Extensions
(cid:0)(cid:2)(cid:3) Handling General Access Costs
The scheme of the Section (cid:11) attempts to maximize the probability of a cache hit(cid:3) Although the
hit probability is certainly a reasonable measure to maximize via a cache replacement strategy(cid:4) it
could also be argued that not all objects on the web have the same access costs(cid:3) For example(cid:4) the
transfer time costs for larger objects are higher(cid:4) though this relationship is somewhat noisy and
far from linear(cid:3) Similarly(cid:4) the access cost of an object requested from a distant web server is likely
to be more than that of one requested from a nearby server(cid:3) In this subsection we notice that the
above replacement scheme may easily be extended to handle non(cid:15)uniform access costs(cid:4) assuming of
course that such costs are known(cid:3)
Let ci be the cost of accessing object i(cid:3) Then the generalized objective function can be written
as Pi(cid:0)C(cid:2)k(cid:3)ci (cid:4)yi(cid:2)(cid:25)Tik(cid:3) Similarly(cid:4) the generalized size adjusted LRU rule would place objects in
non(cid:15)decreasing order of (cid:14)Si (cid:4)(cid:25)Tik(cid:16)(cid:2)ci(cid:4) and greedily purge those objects with the highest indexes(cid:3)
Note that if all values of ci are uniform (cid:14)(cid:7)(cid:4) for example(cid:16)(cid:4) then the replacement policy reverts back
to our original one(cid:3)
(cid:0)(cid:2)(cid:4) Handling Object Expiration Times
Objects on the web are often assigned expiration times(cid:4) and our replacement algorithm should be
able to factor these in e(cid:5)ectively(cid:3) It is tempting here to employ the time to live for an object(cid:4)
namely the di(cid:5)erence between the expiration time of the objectand the current time(cid:3) Forexample(cid:4)
if an object has only a short time to live(cid:4) or perhaps is already stale(cid:4) it would seem to be a good
candidate for replacement by the incoming object(cid:3) Unfortunately(cid:4) this approach can lead to a very
unstable cache and also appears impractical to implement because of high maintenance costs(cid:3) We
shall usemore staticdatainstead(cid:4) datawhich is computed once foreachobjectatthe timeit enters
the cache(cid:3) In particular(cid:4) suppose an object i enters the cache at time t(cid:3) Let us de(cid:17)ne (cid:5)ti(cid:4) to be
the di(cid:5)erence between t and the time when it was last accessed(cid:3) Let (cid:5)ti(cid:0) be the di(cid:5)erence between
(cid:19)
the object expiration time and t(cid:3) (cid:14)Note that we are using (cid:5) to refer to time di(cid:5)erences(cid:29) we had
previously used (cid:25) to refer to di(cid:5)erences in terms the number of accesses(cid:3)(cid:16) We de(cid:17)ne the refresh
overhead factor for an incoming object i to be ri (cid:26) minf(cid:7)(cid:3)(cid:5)ti(cid:4)(cid:2)(cid:5)ti(cid:0)g(cid:3) This value is approximately
the reciprocal of the number of expected accesses before the object needs to be refreshed(cid:3) We
incorporate the refresh overhead factor into the replacement policy by ordering objects in terms
of non(cid:15)decreasing values of (cid:14)Si (cid:4)(cid:25)Tik(cid:16)(cid:2)(cid:14)ci(cid:4)(cid:14)(cid:7)(cid:2)ri(cid:16)(cid:16)(cid:4) and greedily purging those objects with the
highest indexes(cid:3)
If object i has no expiration date then (cid:5)ti(cid:0) is in(cid:17)nite(cid:4) so that the refresh overhead factor ri (cid:26) (cid:24)(cid:3)
Thus thereplacement policy revertsbacktoouroriginal one(cid:3) On the otherhand(cid:4) if objectihasnot
been accessed before(cid:4) then we can assume that (cid:5)ti(cid:4) is in(cid:17)nite(cid:4) so that the refresh overhead factor
ri (cid:26) (cid:7)(cid:3) (cid:14)In fact(cid:4) the statement ri (cid:26) (cid:7) holds whenever (cid:5)ti(cid:4) (cid:3) (cid:5)ti(cid:0)(cid:3)(cid:16) This would result in a quick
purge from the cache(cid:3) However(cid:4) the admission policy described in the next section will generally
not allow such an object toenter the cache in the (cid:17)rst place(cid:3)(cid:16) The refresh overhead factorwill have
only a marginal e(cid:5)ect on the replacement policy if objects are accessed much more frequently than
they expire(cid:3) Conversely(cid:4) when an object has an expiration rate nearly as big as the access rate(cid:4) the
refresh overhead factor will have a signi(cid:17)cant e(cid:5)ect(cid:3)
Similarly(cid:4) the more implementable PSS scheme can be adapted to the case when there are access
costs and(cid:28)or expiration times(cid:3) Instead of grouping together objects based upon the value of their
sizes(cid:4) we group togetherobjects based upon geometrically increasing ranges for the value of Si(cid:2)(cid:14)ci(cid:4)
(cid:14)(cid:7)(cid:2)ri(cid:16)(cid:16)(cid:3)
(cid:8) An Admission Control Policy
An admission control policy decides whether or not it is worthwhile caching an object in the (cid:17)rst
place(cid:3) Having a good admission control policy is especially important when caching non(cid:15)uniform
size objects(cid:4) because a considerable amount of disruption can be caused when an object is added
and others are purged from the cache(cid:3) Having too frequent replacements may lead to wasted space
and to storing objectswhich are never hit atall(cid:3) Admission control makesthe scheme less sensitive
to the transients in the workload(cid:3)
In order to do a good job with admission control(cid:4) we propose the construction of a small auxiliary
cache which maintains the identitiesof somenumber X of objects(cid:3) Foreach object in this auxiliary
cache we also maintain timestamps of the last access(cid:4) measured both in terms of the number of
object accesses and time(cid:4) together with access cost and expiration time data(cid:3) The access counter
is incremented each time an object is requested from the cache(cid:4) whether or not that request can be
(cid:17)lled(cid:3) Because the auxiliary cache contains identities ofobjects ratherthan the objects themselves(cid:4)
its size is negligible compared to thatof the main cache(cid:3) (cid:14)As a rule of thumb(cid:4) weset X tobe about
twice the averagenumber of objectsin the main cache(cid:3)(cid:16) The auxiliary cache is maintained in strict
LRU order(cid:3) Figure (cid:11) illustrates the auxiliary and main caches(cid:3)
We would like to have an admission control policy which ensures that at the kth iteration the
potential incoming object ik is popular enough to o(cid:5)set the loss of the objects it displaces(cid:3) So
we proceed as follows(cid:22) In the event that there is enough free space available for the object ik(cid:4) we
simply bring ik into the cache(cid:3) Otherwise(cid:4) we check if ik occurs in the auxiliary cache(cid:3) If it does
(cid:13)
Object 1
Auxiliary
cache
Object 2
contains Main cache
only URI contains the
and timestamps actual objects
of last few Object 3 cached
objects
accessed
Object 4
Figure (cid:11)(cid:22) Performing Admission Control
not(cid:4) then the object ik does not enter the main cache(cid:3) (cid:14)It is however(cid:4) added to the auxiliary cache
in accordance with LRU rules(cid:3)(cid:16) On the other hand(cid:4) if ik does occur in the auxiliary cache(cid:4) then we
determine if the decision which the replacement policy heuristic makes would be pro(cid:17)table(cid:3) That
is(cid:4) we compare the value (cid:14)cik (cid:4)(cid:14)(cid:7)(cid:2) rik(cid:16)(cid:16)(cid:2)(cid:25)Tikk with the sum Pi(cid:14)ci(cid:4)(cid:14)(cid:7)(cid:2)ri(cid:16)(cid:16)(cid:2)(cid:25)Tik of the set of
candidate outgoing objects determined using the replacement scheme(cid:3) We admit an object only if
it is pro(cid:17)table to do so(cid:3) Observe that the information needed can be obtained from the auxiliary
cache(cid:3) After this iteration(cid:4) the time stamp of the object ik is updated(cid:3)
The idea of admission control bears some resemblance in spirit to the (cid:11)Q approximation of LRU(cid:15)K
proposed in (cid:6)(cid:7)(cid:23)(cid:4) (cid:11)(cid:24)(cid:12) for making cache more robust to workload transients(cid:3) However(cid:4) the method of
doing so is di(cid:5)erent in this case(cid:3)
(cid:9) Empirical Results
Inthissectionweexperimentally comparetheperformancetheperformanceoffourdi(cid:5)erentcaching
schemes(cid:22) the naive LRU policy extension(cid:4) the LRUMIN policy of (cid:6)(cid:7)(cid:12)(cid:4) the SIZE policy discussed in
(cid:6)(cid:11)(cid:19)(cid:12)andthePSSpolicy proposedinthispaper(cid:3) Weemploybotheventandtracedrivensimulations(cid:3)
Note that the PSS scheme was obtained as a more easily implementable variant of the SLRU
algorithm described in Section (cid:11)(cid:3) Both of these schemes were implemented in conjunction with the
admission control of Section (cid:23)(cid:3) We compare SLRU to PSS in order to show that the two schemes
are virtually identical in terms of performance(cid:3)
Weareinterestedinexaminingtheperformanceofthealgorithmsundertheassumptionthatobjects
have varying sizes(cid:4) relative frequencies(cid:4) and combinations of these two factors(cid:3) The following were
the key performance metrics(cid:3)
(cid:8)
A comparison of PSS and SLRU Uncorrelated Case: Zipf Parameter 0.8
0.9 0.8
0.85
0.7
0.8
0.6
0.75 x PSS
Ratio 0.7 * SLRU Ratio0.5
Hit Hit
0.65
0.4
x PSS
0.6
o LRUMIN
0.3 * SIZE
0.55
+ LRU
0.5 0.2
0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45
Fraction of maximum required cache capacity Fraction of maximum cache capacity required
Figure (cid:20)(cid:22) Figure (cid:20)(cid:22) PSS and SLRU(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)(cid:3)Figure (cid:9)(cid:22) Hit Ratio(cid:4) No Correlation
(cid:14)(cid:7)(cid:16) The hit ratio is(cid:4) of course(cid:4) the primary measure
(cid:14)(cid:11)(cid:16) Cost savings(cid:4) as noted in Section (cid:9)(cid:4) can also be important(cid:3)
(cid:14)(cid:20)(cid:16) A (cid:17)nal measure is the robustness of the various schemes(cid:3) It is well known that the perfor(cid:15)
mance of caching policies for web objects often depends upon whether smaller objects have
higher frequency or vice versa(cid:3) The LRU scheme is very robust for uniform size objects and
varying distributions of relative frequencies(cid:3) All the schemes that we compare are in fact
generalizations of LRU in one fashion or another(cid:4) and consequently it is useful to see how the
correlation of size and frequency factors into the robustness of the proposed schemes(cid:3)
(cid:5)(cid:2)(cid:3) Event Driven Simulation
The primary motivation for performing event driven simulation is to understand the e(cid:5)ect of the
varying parameters on the performance of the schemes(cid:3) We test the caching schemes on objects
of sizes uniformly distributed between (cid:7) and (cid:23)(cid:24)(cid:24)(cid:3) That is(cid:4) we assume that there are (cid:23)(cid:24)(cid:24) di(cid:5)erent
objects(cid:4) with one object of each size i(cid:3) The objects were chosen to have Zipf(cid:15)like frequency distrib(cid:15)
(cid:2)
utions (cid:6)(cid:7)(cid:19)(cid:12)(cid:3) Thus thefrequency ofobjectiis proportionalto(cid:7)(cid:2)(cid:6)(cid:14)i(cid:16) (cid:4)where (cid:7) is theZipf parameter(cid:4)
and (cid:6) is a permutation vector(cid:3) By varying (cid:6) we can a(cid:5)ect the relationship between the size and
frequency(cid:3)
Since we used the PSS scheme asamore implementable surrogateforSLRU(cid:4) it is useful tocompare
the performance of the PSS and the SLRU schemes(cid:3) In Figure (cid:20) we consider the case where object
size and frequency are uncorrelated(cid:3) That is(cid:4) we choose (cid:6) randomly(cid:3) In this case(cid:4) we chose a Zipf
parameter of (cid:7)(cid:4) which yields a fully Zipf distribution(cid:3) As we see from the (cid:17)gure(cid:4) the hit ratios of
these two are so similar that it is almost impossible to distinguish between them(cid:3) In other words(cid:4)
the PSS scheme is in practice no worse than its theoretical ancestor(cid:3)
We next tested three di(cid:5)erent relationships between the frequency of an object and its size(cid:3) For
each of these cases(cid:4) we considered a Zipf parameter equal to (cid:24)(cid:3)(cid:8)(cid:4) which is somewhat less skewed(cid:3)
(cid:14)(cid:7)(cid:16) There is no correlation between the object size and the frequency(cid:3) In this case(cid:4) Figure (cid:9)
(cid:10)