This page explains the traffic model supported by Web Polygraph.
1. Overview
2. Request properties
2.1 Robot interest
2.2 Visible server
2.3 Response content type
2.4 Object ID
2.5 Request type
2.6 Request method
3. Request timing
3.1 The many faces of a constant request rate
3.2 Dangers of the best effort mode
3.3 Embedded objects
3.4 Connection limits
3.5 Request pipelining
4. Reply properties
5. Reply timing
6. Example
Traffic model is responsible for producing a stream of HTTP requests and replies. The properties of the stream depend on the model configuration and, to a certain degree, on behavior of the device under test.
The model has several independent components or algorithms described below.
Here is an outline of the algorithm that is used to select the properties of the next request. The selection takes several steps.
select robot interest if foreign interest then select a URL from a trace else select visible origin server select target origin server behind the visible server select response content type to expect select object to request endif select request type select request method2.1 Robot interest
The first step of the algorithm is robot interest selection. If a foreign interest is selected, the URL is selected from a trace specified in a foreign_trace field of the robot configuration (foreign URLs are repeated using the same recurrence algorithm used for Polygraph URLs). Otherwise, the visible server, the target server, and the object to request are selected to form a URL as described below.
2.2 Visible server
Visible origin server is one of the server addresses in the origins array of the robot configuration. Visible server address may be mapped into several ``target'' server addresses. Different target servers behind the same visible name may have different properties (e.g., images and CGI servers behind the same VIP of a L4 load balancing switch). Thus, a robot has to select one target server to know what objects the robot can ask for.
Note, however, that in some setups it is impossible for the robot to guarantee that the request will eventually reach the selected real server. If the mapping is request-independent (e.g., random or load-dependent), then the request may go to a different target server. On the other hand, a correctly configured network will not use request-independent mapping for servers that are not identical. Polygraph is designed to handle correctly configured setups, but may get confused if requests to different servers are mis-redirected.
2.3 Response content type
The response Content type is selected from the direct_access array of the target Server. Content properties may affect recurrence and, hence, object ID selection, so the response content type must be selected before we know what object we want to request. Before Polygraph v4.4.0, the content type was determined after the object ID was selected.
Configuring content-driven Robot behavior is discussed elsewhere.
2.4 Object ID
The following algorithm is used to select an object to request. All persistent object properties are defined by the object identifier (oid). Selecting an object boils down to selecting an oid.
if (we should and can repeat a request) { select ``old'' oid requested from the visible origin server before } else { generate ``new'' oid for the visible origin server }The request repetition probability is determined by the recurrence ratio of the robot configuration. If a robot cannot repeat a request (e.g., no requests to the visible server were performed before), then a new oid is generated. Polygraph collects statistics on oid generation needs and oid generation deliverables so it is possible to check if the above algorithm was failing too often.
Object Popularity model configuration determines which old (i.e., already requested) oid is selected.
The actual oid generation algorithm is a little bit more complex because each robot has to maintain two ``worlds'' of URLs. The first, ``public'', world is shared among all the robots of all polyclt processes. The second, ``private'', world is not shared. For every URL not coming from a foreign trace, Polygraph decides which world should be hit based on robot's interest configuration field. The above algorithm is applied to the selected world if possible and to the alternate world if the selected world cannot accommodate the desired action. Again, collected statistics can be used to track the number of oid generation problems.
When the target server, the content type, and oid are selected, all other object properties such as URL extension or expiration time can be reproduced as needed.
Note that not all repeated requests offer a hit to a caching intermediary. Besides recurrence probability mentioned above, factors affecting hit ratio include object cachability status and request type.
2.5 Request type
The request type is selected from the robot req_types PGL field, regardless of the oid selection. Polygraph does not support some URL scheme and request type combinations. For the HTTP scheme, Basic, Ims200, Ims304, Reload, Range, and Upload request types are supported. The FTP URL scheme supports Basic and Upload request types only. If an unsupported request type is selected for an FTP URL, the Basic request type is used instead.
2.6 Request method
The request method is selected based on the request type. The method is selected at random only when there is a choice of methods for a given request type. The following table gives scheme-type-method mapping:
URL scheme Request type Request method Comment HTTP Basic GET, HEAD, POST, PUT Ordinary HTTP request using one of the listed request methods HTTP IMS GET, HEAD, POST, PUT IMS HTTP request using one of the listed request methods HTTP Reload GET, HEAD, POST, PUT request with a "Pragma: no-cache and "Cache-Control: no-cache" header fields using one of the listed request methods HTTP Range GET, HEAD, POST, PUT HTTP request with a Range header using one of the listed request methods HTTP Upload PUT HTTP upload request FTP Basic RETR FTP download request FTP Upload STOR FTP upload request
Request timing is modeled on a per-robot basis. No synchronization is implemented among robots. Each robot is a completely independent entity.
Two request timing options are supported: The default ``best effort'' request rate and the optional ``constant'' request rate. In best effort mode, a robot submits the next request right after the reply to the previous request was received. Constant request rate mode produces a stream of requests with a given mean throughput; the submission of the next request does not depend on the reply status.
Robots with configured req_rate field emit Poisson request stream with the specified mean request rate.
Robots with configured req_inter_arrival distribution emit request stream with inter-arrival times drawn from the distribution.
Best effort mode is enabled when neither req_rate nor req_inter_arrival field is set. This is the default.
3.1 The many faces of a constant request rate
When your robots do not use SSL or authentication, the configured request rate is what the DUT or an independent client observer should experience and report, on average. The Poisson inter-arrival gap and other factors introduce some randomness, but there should not be big surprises in a correctly configured tests. If you observe request rates that differ from the PGL-configured req_rate settings under these simple conditions, then you may be overloading Polygraph clients or overly restricting them with connection limits.
Things get a lot more complicated when you enable HTTPS and/or authentication. With SSL connections or authentication in play, Polygraph may have to send multiple HTTP requests to get a single resource, and you have to deal with multiple different request rates during the same test. The following table illustrates a few common cases. This information is valid for Polygraph v4.8.0 and later (earlier versions did not have a concept of a baseline transaction and their results were even more difficult to configure and interpret correctly).
3.2 Dangers of the best effort mode
It is very important to understand that best effort workload introduces a tight dependency between throughput and response time. In general, those two metrics should be independent. Many test results, including at least one published by a trade magazine, were essentially bogus because they were using best effort workloads. With constant request rate, throughput is virtually independent from response time, given you have enough resources to support all pending transactions.
Unfortunately, users are tempted to use best effort robots because the latter require less configuration parameters. However, if the quality of the result is of any importance, best effort robots should be avoided. Any rule of thumb has exceptions, but people who can configure valid best effort workloads do not need this documentation :).
3.3 Embedded objects
Request stream is also affected by the presence of references to ``embedded'' objects in responses. When a robot finds a reference to an embedded object (e.g., an inlined image in HTML container), it requests that object just like a browser would. The requests to embedded objects do not follow the configured timing model. Instead, they are submitted immediately (if possible). The stream of requests for embedded objects changes the overall traffic pattern significantly.
The embed_recur field of a robot can be used to specify the probability that a request will be issued when a reference to an embedded object is found.
3.4 Connection limits
Using robot's open_conn_lmt field, one can prevent the robot from opening more than N connections for concurrent requests. Most real browsers have connection limits.
When the connection limit is reached, the robot will queue requests. The queue is drained in a first-in-first-out order, as connection slots become available. This queuing has a significant impact on the overall traffic pattern. In the extreme case, when a queue is never empty, a constant request rate robot will behave exactly like a best-effort one!
When using connection limits, make sure that robot request rate matches the number of available connections and expected response time. Always monitor queue lengths to make sure your constant request rate workload has not turned into best effort one. Report Generator can be used to compare the number of waiting requests with the number of active transactions. As a rule of thumb, the waiting queue length should not exceed 50% of the number of active transactions.
Another side effect of connection limits is that waiting requests consume memory. If the queue length continues to grow, Polygraph will eventually run out of memory and quit. On the bright side, the result of a test with growing waiting queues is almost always invalid (should be ignored) anyway.
Robot's waiting queue length can be limited using wait_xact_lmt configuration field.
3.5 Request pipelining
HTTP allows clients to send a request on the same connection as the previous request and before the end of the response to the previous request is received. Such requests are called pipelined requests. Many requests can be simultaneously pipelined on a connection, increasing pipeline "depth". Pipelining is meant to eliminate propagation delays for consecutive requests, decreasing overall response time. A browser typically pipelines requests when requesting images and other embedded objects, as it parses the HTML container page.
Polygraph robots can pipeline requests for embedded objects (and only for embedded objects). The pipeline depth is controlled by the pipeline_depth field of a PGL Robot specification. By default, requests are not pipelined. Pipelining requires enabling persistent connections. If a connection with pipelined requests is closed prematurely, all pipelined requests are retried using non-pipelined (but possibly persistent) connections. These retries are in agreement with HTTP spec recommendations:
Clients [...] SHOULD be prepared to retry their connection if the first pipelined attempt fails. If a client does such a retry, it MUST NOT pipeline before it knows the connection is persistent. Clients MUST also be prepared to resend their requests if the server closes the connection before sending all of the corresponding responses. (Section 8.1.2.2 "Pipelining", RFC 2616)Note that pipelined requests are not a subject to artificial limits on the number of open connections (see the above Section). A pipelined request reuses an existing, active connection. Thus, if pipelining is enabled, the number of outstanding requests (and, hence, transactions) can exceed the specified limit on the number of open connections.
Polygraph servers support pipelined requests, regardless of robot settings. A polygraph server does not prefetch or read-ahead requests and, hence, cannot distinguish a pipelined request from a regular request on a persistent connection.
Request pipelining is supported in Polygraph starting with version 3.0.
The object identifier or oid determines static response properties such as MIME type, size, and URL extension. The possible values for these properties come from the Content PGL type. For example:
Content cntHTML = { kind = "HTML"; mime = { type = "text/html"; extensions = [ ".html" : 60%, ".htm" ]; }; obj_life_cycle = olcHTML; size = exp(8.5KB); cachable = 90%; checksum = 1%; may_contain = [ cntImage ]; embedded_obj_cnt = zipf(13); };Polygraph robot selects a server for the request (see the Request properties section). The contents field of the selected server PGL configuration specifies the distribution of available content kinds. For example:
Server S = { ... contents = [ cntImage: 65%, cntHTML: 15%, cntOther ]; direct_access = [ cntHTML, cntOther ]; };Only content kinds specified in the direct_access field can be requested in independent HTTP transactions (i.e., transactions not fetching an embedded object). For embedded objects such as images inside generated HTML pages, the object oid is taken from the URL embedded in the response content.
When a Polygraph server generates embedded URLs, it strives to preserve the overall content kind distribution specified in the server contents selector. Honoring that distribution may not be possible if the distribution of the number of embedded objects per container and the distribution of content kinds are out of balance. Polygraph reports expected distribution of content kinds and warns if it does not match the configured one close enough. Look for Server content distributions text in the verbose console log:
000.09| Server content distributions: Server loadSteps-1.1: content planned% likely% error% mean_sz_bytes image 65.00 65.67 1.04 4612.66 HTML 15.00 14.71 -1.92 8713.25 download 0.50 0.49 -1.92 301938.89 other 19.50 19.13 -1.92 25423.03 expected average server-side cachability: 80.01% expected average server-side object size: 10653.99Bytes
Polygraph server replies as fast as it can unless a delay distribution was specified using the xact_think field. The delay is applied right after the connection is accepted (or after an idle persistent connection indicates request presence) and before the request headers are read by Polygraph process. Note that TCP stack buffers are likely to contain request headers, so an outside observer (e.e., a proxy) may think that the request has been read by Polygraph.
Robot configuration below will produce a constant request rate Poisson stream of 0.4 requests per second. The robot will open no more than 4 connections and will request all embedded objects. Repeated requests will happen with 65% probability and will follow Uniform popularity model with 1% ``hot set'' size.
PopModel popModel = { pop_distr = popUnif(); hot_set_frac = 1%; hot_set_prob = 10%; }; Robot R = { recurrence = 65%; embed_recur = 100%; public_interest = 50%; req_types = [ "IMS" : 20%, "Reload" : 5%, "Basic" ]; pop_model = popModel; req_rate = 0.4/sec; open_conn_lmt = 4; ... };