Here we explain some of the parameters used in the workload
specification.
2.1 Working set size
For private tests, WebAxe-1 users can set the size of a working
set to arbitrary value, simulating the approximate size of a Web site a proxy
is expected to accelerate. To make public comparison possible, the working set
size should be set at either 100MB or 1GB. The former value
simulates a medium size Web site worth acceleration (perhaps with some room
for growth). The latter value corresponds to some of the most popular and
large sites.
2.2 Cache size
To calculate the cache size for the purpose of WebAxe-1, one must
add up the physical size of all disks that may hold cached objects and the
total RAM size.
For example, if your cache has 1GB of RAM and three 8GB disks
(two of which are used for caching and allocate 80% of disk capacity
for that purpose), then the cache size for the purpose of this test is
1+2*8GB or 17GB. Note that the 80% figure is irrelevant
here.
Please note that the current definition of the cache size penalizes the
caches that utilize only some portion of their disk space. On the other hand,
it is often impossible to guarantee that a proxy that is told to use
50% of disk capacity will use only one half of disk blocks. Depending
on the algorithm, a proxy may ``touch'' all disk blocks, keeping the usage
level at 50% at all times. Our primary goal is to touch every storage
block two times during the fill. It is not clear how to define the minimum
fill size to achieve that goal. The current definition may be an overkill.
Let us know if you have better ideas.
The test is divided into four major phases. Major phases are connected with
ramp-up/down segments to avoid sudden changes in request rate. We describe
major phases in detail below.
3.1 Fill phase
The first phase populates Web cache with server content. The ``goal'' of
this phase is specified in terms of ``fill size'' or the total volume of new
cachable objects piped through the cache. To approach steady state conditions,
the fill size is set to twice the cache size (see above for a ``cache
size'' definition).
The request rate during the fill is up to the tester to specify. Fill rate
must be between 10% and 100% of peak request rate.
Essentially, the fill rate is a parameter of the workload. However, we
suspect that the fill rate does not have a significant impact on the results
of the test (``best effort'' workload, however, may overwhelm large proxy
configurations as it resembles a denial-of-service attack).
Recurrence ratio is set to 5% (to speedup filling process). Objects
requested during the fill phase may be requested later.
3.2 Top1 phase
The first ``top'' phase measures the performance of the proxy under peak
load. Request rate is kept constant at peak level. Recurrence ratio
is set to 95% so very few new objects are added to the working set
(purging some old objects from the working set, of course).
The Top1 phase duration is 1hour.
3.3 Idle phase
The short idle phase is used to compare the performance of the proxy under
light load with the performance under peak load. Such comparisons help to
identify what performance measurements are load dependent for a given
proxy.
The request rate during idle phase is 10% of peak request rate.
The idle phase duration is 10min.
3.4 Top2 phase
Parameters of the second ``top'' phase are equal to those of the
Top1 phase. Top2 is used to check whether a proxy has
reached a steady state. Statistics collected during this phase is used for
most comparisons and conclusions.
3.5 Putting it all together
The table below summarizes phase configurations, including the ramp phases
not discussed above.
| Phase |
Load
(req/sec) |
Recurrence
(%) |
Goal |
| warm-up |
0 -> fill |
5% |
~5min |
| fill |
fill |
5% |
2*cache_size |
| link |
fill -> peak |
95% |
5min |
| top1 |
peak |
95% |
60min |
| dec |
peak -> idle |
95% |
5min |
| idle |
idle |
95% |
10min |
| inc |
idle -> peak |
95% |
5min |
| top2 |
peak |
95% |
60min |
| cool-off |
peak -> 0 |
95% |
1min |
|
The total duration of the test depends on the fill rate. The phases with
known duration add up to 2.5hours.