Web Polygraph 

 Documentation
    Reference Manual
        PGL
        Simulation models
           On Realism
 ·         Traffic generation
              Overview
              Request properties
              Request timing
              Reply properties
              Reply timing
              Example
           SSL layer
           Object life cycle
        Command options
        User-defined distr
        Loadable modules
        Output
     User Manual
     Workloads
 Downloads
    Copyright
     Changes
     Upgrade
 Support
    Wish List
     Sponsors
     Mailing List
     Users
 Miscellaneous
    Results
     Compliance
     Other Tools
     Search
  

Home · Search · Print · Help 


Traffic generation

This page explains the traffic model supported by Web Polygraph.

Table of Contents

1. Overview
2. Request properties
    2.1 Robot interest
    2.2 Visible server
    2.3 Response content type
    2.4 Object ID
    2.5 Request type
    2.6 Request method
3. Request timing
    3.1 The many faces of a constant request rate
    3.2 Dangers of the best effort mode
    3.3 Embedded objects
    3.4 Connection limits
    3.5 Request pipelining
4. Reply properties
5. Reply timing
6. Example

1. Overview

Traffic model is responsible for producing a stream of HTTP requests and replies. The properties of the stream depend on the model configuration and, to a certain degree, on behavior of the device under test.

The model has several independent components or algorithms described below.

2. Request properties

Here is an outline of the algorithm that is used to select the properties of the next request. The selection takes several steps.

select robot interest
if foreign interest
then
    select a URL from a trace
else
    select visible origin server
    select target origin server behind the visible server
    select response content type to expect
    select object to request
endif
select request type
select request method

2.1 Robot interest

The first step of the algorithm is robot interest selection. If a foreign interest is selected, the URL is selected from a trace specified in a foreign_trace field of the robot configuration (foreign URLs are repeated using the same recurrence algorithm used for Polygraph URLs). Otherwise, the visible server, the target server, and the object to request are selected to form a URL as described below.

2.2 Visible server

Visible origin server is one of the server addresses in the origins array of the robot configuration. Visible server address may be mapped into several ``target'' server addresses. Different target servers behind the same visible name may have different properties (e.g., images and CGI servers behind the same VIP of a L4 load balancing switch). Thus, a robot has to select one target server to know what objects the robot can ask for.

Note, however, that in some setups it is impossible for the robot to guarantee that the request will eventually reach the selected real server. If the mapping is request-independent (e.g., random or load-dependent), then the request may go to a different target server. On the other hand, a correctly configured network will not use request-independent mapping for servers that are not identical. Polygraph is designed to handle correctly configured setups, but may get confused if requests to different servers are mis-redirected.

2.3 Response content type

The response Content type is selected from the direct_access array of the target Server. Content properties may affect recurrence and, hence, object ID selection, so the response content type must be selected before we know what object we want to request. Before Polygraph v4.4.0, the content type was determined after the object ID was selected.

Configuring content-driven Robot behavior is discussed elsewhere.

2.4 Object ID

The following algorithm is used to select an object to request. All persistent object properties are defined by the object identifier (oid). Selecting an object boils down to selecting an oid.

if (we should and can repeat a request) {
    select ``old'' oid requested from the visible origin server before
} else {
    generate ``new'' oid for the visible origin server
}

The request repetition probability is determined by the recurrence ratio of the robot configuration. If a robot cannot repeat a request (e.g., no requests to the visible server were performed before), then a new oid is generated. Polygraph collects statistics on oid generation needs and oid generation deliverables so it is possible to check if the above algorithm was failing too often.

Object Popularity model configuration determines which old (i.e., already requested) oid is selected.

The actual oid generation algorithm is a little bit more complex because each robot has to maintain two ``worlds'' of URLs. The first, ``public'', world is shared among all the robots of all polyclt processes. The second, ``private'', world is not shared. For every URL not coming from a foreign trace, Polygraph decides which world should be hit based on robot's interest configuration field. The above algorithm is applied to the selected world if possible and to the alternate world if the selected world cannot accommodate the desired action. Again, collected statistics can be used to track the number of oid generation problems.

When the target server, the content type, and oid are selected, all other object properties such as URL extension or expiration time can be reproduced as needed.

Note that not all repeated requests offer a hit to a caching intermediary. Besides recurrence probability mentioned above, factors affecting hit ratio include object cachability status and request type.

2.5 Request type

The request type is selected from the robot req_types PGL field, regardless of the oid selection. Polygraph does not support some URL scheme and request type combinations. For the HTTP scheme, Basic, Ims200, Ims304, Reload, Range, and Upload request types are supported. The FTP URL scheme supports Basic and Upload request types only. If an unsupported request type is selected for an FTP URL, the Basic request type is used instead.

2.6 Request method

The request method is selected based on the request type. The method is selected at random only when there is a choice of methods for a given request type. The following table gives scheme-type-method mapping:

URL scheme Request type Request method Comment
HTTP Basic GET, HEAD, POST, PUT Ordinary HTTP request using one of the listed request methods
HTTP IMS GET, HEAD, POST, PUT IMS HTTP request using one of the listed request methods
HTTP Reload GET, HEAD, POST, PUT request with a "Pragma: no-cache and "Cache-Control: no-cache" header fields using one of the listed request methods
HTTP Range GET, HEAD, POST, PUT HTTP request with a Range header using one of the listed request methods
HTTP Upload PUT HTTP upload request
FTP Basic RETR FTP download request
FTP Upload STOR FTP upload request

3. Request timing

Request timing is modeled on a per-robot basis. No synchronization is implemented among robots. Each robot is a completely independent entity.

Two request timing options are supported: The default ``best effort'' request rate and the optional ``constant'' request rate. In best effort mode, a robot submits the next request right after the reply to the previous request was received. Constant request rate mode produces a stream of requests with a given mean throughput; the submission of the next request does not depend on the reply status.

Robots with configured req_rate field emit Poisson request stream with the specified mean request rate.

Robots with configured req_inter_arrival distribution emit request stream with inter-arrival times drawn from the distribution.

Best effort mode is enabled when neither req_rate nor req_inter_arrival field is set. This is the default.

3.1 The many faces of a constant request rate

When your robots do not use SSL or authentication, the configured request rate is what the DUT or an independent client observer should experience and report, on average. The Poisson inter-arrival gap and other factors introduce some randomness, but there should not be big surprises in a correctly configured tests. If you observe request rates that differ from the PGL-configured req_rate settings under these simple conditions, then you may be overloading Polygraph clients or overly restricting them with connection limits.

Things get a lot more complicated when you enable HTTPS and/or authentication. With SSL connections or authentication in play, Polygraph may have to send multiple HTTP requests to get a single resource, and you have to deal with multiple different request rates during the same test. The following table illustrates a few common cases. This information is valid for Polygraph v4.8.0 and later (earlier versions did not have a concept of a baseline transaction and their results were even more difficult to configure and interpret correctly).

Workload (PGL) i-line reported rate (console) DUT-observed rate Reporter rates (HTML) Notes
traffic mix req_rate baseline ssl connect auth‑ing baseline all requests
Simple GETs 100 100 - - - 100 100 100

No surprises here. All rates are the same.
Authenticated GETs
(no robot sessions)		 100 100 - - - 100 100 100

Same as simple, unauthenticated GETs because each robot remembers that the proxy wanted authentication information and sends that info with each request (except for the very first one). Thus, the proxy can authenticate each request (except for the very first one) without any intermediate HTTP 407 (Proxy Authentication Required) responses. In a steady state, the traffic rates are identical to those with simple, unauthenticated GETs.
Authenticated GETs
(short robot sessions)		 100 100 - - 100 200 100 200

After each GET transaction, the robot ends a "session", forgets that the proxy requires authentication, send the next GET request without any authentication information, and gets an HTTP 407 (Proxy Authentication Required) response. After a 407 response, the robot has to send an additional request with authentication info. This introduces an "auth-ing" stream of requests and doubles raw request rates (as seen by DUT).
CONNECT+GET
(no MitM)		 100 100 100 100 - 100 100 200

This test terminates the SSL connection after a single tunneled GET request. When the proxy does not perform a Man-in-the-Middle attack on the CONNECT tunnel, the proxy sees CONNECT requests but not GET requests. Only Polygraph report will tell you that the number of HTTP transactions per connection have doubled (and the baseline response time went up to account for the CONNECT transaction time!).
CONNECT+GET
(MitM bumping)		 100 100 100 100 - 200 100 200

A Man-in-the-Middle proxy "bumps" an SSL connection and "discovers" the GET request inside that connection. Thus, turning on bumping/MitM feature suddenly doubles the request rate from the proxy point of view, without any PGL or traffic on the wire changes!
CONNECT+10 GETs
(no MitM)		 100 100 100 10 - 10 100 110

This test terminates the SSL connection after 10 tunneled GET requests. When the proxy does not perform a Man-in-the-Middle attack on the CONNECT tunnel, the proxy sees CONNECT requests but not GET requests. Only Polygraph report will tell you that the number of HTTP transactions per connection increased by the order of magnitude (and the baseline response time went up a little to account for the CONNECT transaction time for the first out of those ten GETs!).
CONNECT+10 GETs
(MitM bumping)		 100 100 100 10 - 110 100 110

A Man-in-the-Middle proxy "bumps" an SSL connection and "discovers" all 10 GET requests inside that connection. Thus, turning on bumping/MitM feature suddenly increases request rate by more than an order of magnitude from the proxy point of view, without any PGL or traffic on the wire changes!

3.2 Dangers of the best effort mode

It is very important to understand that best effort workload introduces a tight dependency between throughput and response time. In general, those two metrics should be independent. Many test results, including at least one published by a trade magazine, were essentially bogus because they were using best effort workloads. With constant request rate, throughput is virtually independent from response time, given you have enough resources to support all pending transactions.

Unfortunately, users are tempted to use best effort robots because the latter require less configuration parameters. However, if the quality of the result is of any importance, best effort robots should be avoided. Any rule of thumb has exceptions, but people who can configure valid best effort workloads do not need this documentation :).

3.3 Embedded objects

Request stream is also affected by the presence of references to ``embedded'' objects in responses. When a robot finds a reference to an embedded object (e.g., an inlined image in HTML container), it requests that object just like a browser would. The requests to embedded objects do not follow the configured timing model. Instead, they are submitted immediately (if possible). The stream of requests for embedded objects changes the overall traffic pattern significantly.

The embed_recur field of a robot can be used to specify the probability that a request will be issued when a reference to an embedded object is found.

3.4 Connection limits

Using robot's open_conn_lmt field, one can prevent the robot from opening more than N connections for concurrent requests. Most real browsers have connection limits.

When the connection limit is reached, the robot will queue requests. The queue is drained in a first-in-first-out order, as connection slots become available. This queuing has a significant impact on the overall traffic pattern. In the extreme case, when a queue is never empty, a constant request rate robot will behave exactly like a best-effort one!

When using connection limits, make sure that robot request rate matches the number of available connections and expected response time. Always monitor queue lengths to make sure your constant request rate workload has not turned into best effort one. Report Generator can be used to compare the number of waiting requests with the number of active transactions. As a rule of thumb, the waiting queue length should not exceed 50% of the number of active transactions.

Another side effect of connection limits is that waiting requests consume memory. If the queue length continues to grow, Polygraph will eventually run out of memory and quit. On the bright side, the result of a test with growing waiting queues is almost always invalid (should be ignored) anyway.

Robot's waiting queue length can be limited using wait_xact_lmt configuration field.

3.5 Request pipelining

HTTP allows clients to send a request on the same connection as the previous request and before the end of the response to the previous request is received. Such requests are called pipelined requests. Many requests can be simultaneously pipelined on a connection, increasing pipeline "depth". Pipelining is meant to eliminate propagation delays for consecutive requests, decreasing overall response time. A browser typically pipelines requests when requesting images and other embedded objects, as it parses the HTML container page.

Polygraph robots can pipeline requests for embedded objects (and only for embedded objects). The pipeline depth is controlled by the pipeline_depth field of a PGL Robot specification. By default, requests are not pipelined. Pipelining requires enabling persistent connections. If a connection with pipelined requests is closed prematurely, all pipelined requests are retried using non-pipelined (but possibly persistent) connections. These retries are in agreement with HTTP spec recommendations:

Clients [...] SHOULD be prepared to retry their connection if the first pipelined attempt fails. If a client does such a retry, it MUST NOT pipeline before it knows the connection is persistent. Clients MUST also be prepared to resend their requests if the server closes the connection before sending all of the corresponding responses. (Section 8.1.2.2 "Pipelining", RFC 2616)

Note that pipelined requests are not a subject to artificial limits on the number of open connections (see the above Section). A pipelined request reuses an existing, active connection. Thus, if pipelining is enabled, the number of outstanding requests (and, hence, transactions) can exceed the specified limit on the number of open connections.

Polygraph servers support pipelined requests, regardless of robot settings. A polygraph server does not prefetch or read-ahead requests and, hence, cannot distinguish a pipelined request from a regular request on a persistent connection.

Request pipelining is supported in Polygraph starting with version 3.0.

4. Reply properties

The object identifier or oid determines static response properties such as MIME type, size, and URL extension. The possible values for these properties come from the Content PGL type. For example:

Content cntHTML = {
    kind = "HTML";
    mime = { type = "text/html"; extensions = [ ".html" : 60%, ".htm" ]; };
    obj_life_cycle = olcHTML;
    size = exp(8.5KB);
    cachable = 90%;
    checksum = 1%;

    may_contain = [ cntImage ];
    embedded_obj_cnt = zipf(13);
};

Polygraph robot selects a server for the request (see the Request properties section). The contents field of the selected server PGL configuration specifies the distribution of available content kinds. For example:

Server S = {
    ...    
    contents = [ cntImage: 65%, cntHTML: 15%, cntOther ];
    direct_access = [ cntHTML, cntOther ];
};

Only content kinds specified in the direct_access field can be requested in independent HTTP transactions (i.e., transactions not fetching an embedded object). For embedded objects such as images inside generated HTML pages, the object oid is taken from the URL embedded in the response content.

When a Polygraph server generates embedded URLs, it strives to preserve the overall content kind distribution specified in the server contents selector. Honoring that distribution may not be possible if the distribution of the number of embedded objects per container and the distribution of content kinds are out of balance. Polygraph reports expected distribution of content kinds and warns if it does not match the configured one close enough. Look for Server content distributions text in the verbose console log:

000.09| Server content distributions:
        Server loadSteps-1.1:
        content   planned%    likely%     error% mean_sz_bytes
        image       65.00      65.67       1.04    4612.66
        HTML        15.00      14.71      -1.92    8713.25
        download     0.50       0.49      -1.92  301938.89
        other       19.50      19.13      -1.92   25423.03
        expected average server-side cachability: 80.01%
        expected average server-side object size: 10653.99Bytes

5. Reply timing

Polygraph server replies as fast as it can unless a delay distribution was specified using the xact_think field. The delay is applied right after the connection is accepted (or after an idle persistent connection indicates request presence) and before the request headers are read by Polygraph process. Note that TCP stack buffers are likely to contain request headers, so an outside observer (e.e., a proxy) may think that the request has been read by Polygraph.

6. Example

Robot configuration below will produce a constant request rate Poisson stream of 0.4 requests per second. The robot will open no more than 4 connections and will request all embedded objects. Repeated requests will happen with 65% probability and will follow Uniform popularity model with 1% ``hot set'' size.

PopModel popModel = {
    pop_distr = popUnif();
    hot_set_frac =  1%;
    hot_set_prob = 10%;
};

Robot R = {
    recurrence      =  65%;
    embed_recur     =  100%;
    public_interest =  50%;
    req_types = [ "IMS" : 20%, "Reload" : 5%, "Basic" ];

    pop_model = popModel;

    req_rate = 0.4/sec;
    open_conn_lmt = 4;
    ...
};

Home · Search · Print · Help