Workload Name: WebAxe-1 Polygraph Version: 2.2.9 or 2.5.5 Configuration: workloads/webaxe-1.pg workloads/webaxe-3.pg Workload Parameters: working set size,
peak request rate, and
cache filling parametersResults: TBA Synopsis: workload designed for testing Web Accelerators (a.k.a. reverse proxies)
1. Background
2. Parameters
2.1 Working set size
2.2 Cache size
3. Test phases
3.1 Fill phase
3.2 Top1 phase
3.3 Idle phase
3.4 Top2 phase
3.5 Putting it all together
4. Other features
PolyMix workloads are designed for testing ``client side'' Web proxies. Server side acceleration requires quite different workloads. Web server benchmarks can be used to test reverse proxies. However those benchmarks are often of low quality, not available without a fee, and/or require too much benchmarking resources to simulate high request rates. WebAxe-1 is our attempt to provide the caching community with a freely available quality high performance benchmark.
Here we explain some of the parameters used in the workload specification.
2.1 Working set size
For private tests, WebAxe-1 users can set the size of a working set to arbitrary value, simulating the approximate size of a Web site a proxy is expected to accelerate. To make public comparison possible, the working set size should be set at either 100MB or 1GB. The former value simulates a medium size Web site worth acceleration (perhaps with some room for growth). The latter value corresponds to some of the most popular and large sites.
2.2 Cache size
To calculate the cache size for the purpose of WebAxe-1, one must add up the physical size of all disks that may hold cached objects and the total RAM size.
For example, if your cache has 1GB of RAM and three 8GB disks (two of which are used for caching and allocate 80% of disk capacity for that purpose), then the cache size for the purpose of this test is 1+2*8GB or 17GB. Note that the 80% figure is irrelevant here.
Please note that the current definition of the cache size penalizes the caches that utilize only some portion of their disk space. On the other hand, it is often impossible to guarantee that a proxy that is told to use 50% of disk capacity will use only one half of disk blocks. Depending on the algorithm, a proxy may ``touch'' all disk blocks, keeping the usage level at 50% at all times. Our primary goal is to touch every storage block two times during the fill. It is not clear how to define the minimum fill size to achieve that goal. The current definition may be an overkill. Let us know if you have better ideas.
The test is divided into four major phases. Major phases are connected with ramp-up/down segments to avoid sudden changes in request rate. We describe major phases in detail below.
3.1 Fill phase
The first phase populates Web cache with server content. The ``goal'' of this phase is specified in terms of ``fill size'' or the total volume of new cachable objects piped through the cache. To approach steady state conditions, the fill size is set to twice the cache size (see above for a ``cache size'' definition).
The request rate during the fill is up to the tester to specify. Fill rate must be between 10% and 100% of peak request rate.
Essentially, the fill rate is a parameter of the workload. However, we suspect that the fill rate does not have a significant impact on the results of the test (``best effort'' workload, however, may overwhelm large proxy configurations as it resembles a denial-of-service attack).
Recurrence ratio is set to 5% (to speedup filling process). Objects requested during the fill phase may be requested later.
3.2 Top1 phase
The first ``top'' phase measures the performance of the proxy under peak load. Request rate is kept constant at peak level. Recurrence ratio is set to 95% so very few new objects are added to the working set (purging some old objects from the working set, of course).
The Top1 phase duration is 1hour.
3.3 Idle phase
The short idle phase is used to compare the performance of the proxy under light load with the performance under peak load. Such comparisons help to identify what performance measurements are load dependent for a given proxy.
The request rate during idle phase is 10% of peak request rate.
The idle phase duration is 10min.
3.4 Top2 phase
Parameters of the second ``top'' phase are equal to those of the Top1 phase. Top2 is used to check whether a proxy has reached a steady state. Statistics collected during this phase is used for most comparisons and conclusions.
3.5 Putting it all together
The table below summarizes phase configurations, including the ramp phases not discussed above.
Phase Load
(req/sec)Recurrence
(%)Goal warm-up 0 -> fill 5% ~5min fill fill 5% 2*cache_size link fill -> peak 95% 5min top1 peak 95% 60min dec peak -> idle 95% 5min idle idle 95% 10min inc idle -> peak 95% 5min top2 peak 95% 60min cool-off peak -> 0 95% 1min
The total duration of the test depends on the fill rate. The phases with known duration add up to 2.5hours.
- minimum number of simulated servers to support given peak request rate (we are not sure if this rule is good enough though),
- simulated servers with 0.3sec mean think time,
- Zipf(0.6) popularity distribution,
- number of robots determined using the same rules as in PolyMix-2,
- Web content model of the PolyMix-2 workload,
- 100msec client side packet delay and 0.1% client side packet loss.
The cache must be cleared (flushed) and restarted before each WebAxe-1 experiment.
This workload is under construction; please let us know if you have any comments or suggestions.