Reporting of the results

This page documents the Reporter interface, available in Web Polygraph starting with version 2.8.0. The deprecated Report Generator scripts are documented elsewhere.

Table of Contents

1. For the impatient
2. Introduction
3. Before you can use Reporter
4. How to generate reports
5. Sharing the results
6. Caution

1. For the impatient

# run the test, possibly on several clients and servers
clt1> polyclt --log clt1.log ...
clt2> polyclt --log clt2.log ...
...
srv1> polysrv --log srv1.log ...
srv2> polysrv --log srv2.log ...
...
# copy logs from each client and server to a single directory 
mon> scp \
    clt1:clt1.log clt2:clt2.log ... \
    srv1:srv1.log srv2:srv2.log ... \
    test123/logs/

# generate a report using both client and server logs
mon> reporter --label "Test-123" test123/logs/*.log

# view generated report at file:/tmp/polyrep/Test-123/index.html

2. Introduction

Web Polygraph includes an automated report generation tool called Reporter. Reporter uses client and server binary logs as well as optional user-supplied information to generate reports that can be then viewed with your favorite browser. The reports include ``executive summary'' page for the boss, and more detailed analysis for the rest of us. Analyzing reports is an essential step in any good testing methodology. What looks normal on the console or device monitor during a test, may look completely different once many factors are put together and long-term trends are plotted.

Last cache-off performance reports for individual entries were generated using a reporter tool.

3. Before you can use Reporter

Reporter uses gnuplot to generate all its graphs. If you do not have gnuplot installed, you must install it and rebuild Polygraph from scratch before you can use Reporter. Polygraph configure script disables reporter if it cannot find a gnuplot installation.

Gnuplot is one of the best plotting (graph generation) tools for Unix. We recommend version 3.6 or later.

example> gnuplot
gnuplot> show version
        Unix version 3.7              patchlevel 0
        last modified Thu Jan 14 19:34:53 BST 1999
        http://www.uni-karlsruhe.de/~ig25/gnuplot-faq/

When installing gnuplot, you must enable the PNG "terminal driver". See the "--with-png" option of gnuplot's configure script. The Factory web site contains more detailed instruction on how to install gnuplot. Reporter uses PNG images because recent gnuplot distributions do not support GIF due to patent problems with that format.

4. How to generate reports

Reporter is built as a reporter executable and is installed where all Polygraph executables are installed (e.g., /usr/local/polygraph/bin/). Running Reporter without any options will list available command-line options.

usage: reporter [--option ...] <log_file_name> ...
options:
  help                  list of options
  version               package version info
  label <str>           test label
  phases <list>         names of phases for executive summary
  report_dir <dir>      report's root directory
  tmp_dir <dir>         temporary dir
  plotter <exe_file>    gnuplot location

Running Reporter with just the filenames of binary logs will produce a complete report, for all test phases. In this case, Reporter will label the report using a common prefix of log file names. You can specify a custom label using the --label command line option. This label will be used for page titles and to name the directory that stores report pages (unless the --report_dir option explicitly says otherwise).

# generate a report using both client and server logs
mon> reporter --label "Test-123" test123/logs/*.log

The --phases option specifies which phase should be used for the baseline presentation in the report, including the executive summary. By default, Reporter either uses phases with a true primary flag or guesses most important phases based on request rate and duration of each logged phase. Reporter usually guesses correctly for complete PolyMix-4 and other standard tests. Reporter always includes details about all phases present in the logs.

5. Sharing the results

To share reports with others, one usually copies the entire report directory to a common web server. This allows for all the HTML pages and images within the report to be preserved. For example,

mon> scp -pr /tmp/polyrep/Test-123 \
    web:/httpd/htdocs/public-results/polyrep/

6. Caution

As with any automated report generator, caution should be taken when interpreting and sharing the results. Reporter may not be able to correctly interpret all the ``interesting'' things you feed it with. For example, if some of your logs are missing or partially corrupted, Reporter may produce strange results without warning you.