Home
Test Manager Guide
This guide describes how to keep track of the performance of the application by evaluating the test results. This guide is intended for test managers.
Evaluating results in web analytic reports
After the test run, evaluate the results in the web analytic reports. Web analytics collect data using new technologies thereby providing better user experience.
Customizing reports
You can customize reports to specifically investigate a performance problem in more detail than what is provided in the default reports.
Adding additional counters on a separate page
You can add additional counters on a separate page without editing an existing report to investigate performance problems in detail.

Test Manager Guide
This guide describes how to keep track of the performance of the application by evaluating the test results. This guide is intended for test managers.
- Evaluating results in web analytic reports
  After the test run, evaluate the results in the web analytic reports. Web analytics collect data using new technologies thereby providing better user experience.
  - Comparing results among runs
    To analyze the difference between two or more reports, you can compare them. For example, to analyze the performance of the application at different time slots or different milestone builds, you can compare two runs.
  - Comparing schedule stages
    When you are running a A schedule, in this context, is used to refer to both VU Schedule and Rate Schedule. that contains stages, time ranges are automatically created for each stage. You can view a report that compares these stages, and you also can set preferences to display the report automatically at the end of a staged run.
  - Comparing results from various regions or agent locations
    When you run a schedule that includes agents from different regions, use the Web Analytic reports to compare the performance data from these regions.
  - Generating functional test reports
    You can generate functional test reports of your tests, which summarize the pass or fail verdicts of elements in the test log. Functional reports are generated from the test run as HTML files that use predefined report designs.
  - Publishing test results to the server
    The test results indicates the quality of application under test. Different stakeholders to the application might want to check the quality of application but do not have the workbench or desktop client installed. As a desktop client user, you can publish the test result to IBM® Rational® Test Automation Server so that others can view it from the web browser.
  - Publishing specific results to the server
    If you have a single test result or multiple test results that are not yet published to IBM® Rational® Test Automation Server, you can publish all of them simultaneously.
  - Customizing reports
    You can customize reports to specifically investigate a performance problem in more detail than what is provided in the default reports.
    - Creating custom reports
      If the default reports do not address your needs, you can create your own reports.
    - Viewing trending reports
      To view the trend of response time for an application over a period of time, open the trend report for a run. In addition to the response time, you can view the trend for the loops, transactions, and performance requirements for the application.
    - Filtering results
      By filtering the results that are displayed in a report, you can remove unnecessary data and focus on the data that is significant to you. If you save the changes, the report will contain these updates the next time that you open it.
    - Adding additional counters on a separate page
      You can add additional counters on a separate page without editing an existing report to investigate performance problems in detail.
      - Displaying counter data in tables or as graphs
        You can display the statistics of counters in a table view or a graphical view so that you can analyze the counters information efficiently.
    - Customizing the appearance of graphs in a report
      To display the data in a table, bar chart, or line chart in a manner that caters to your test requirements, use the controls that are available in the View Options of a report.
    - Changing the report displayed during a run
      Use this page to select the default report that opens during a run. Typically, you select Determine default report based on protocols in test, which determines the protocols that you are testing and automatically opens the appropriate protocol-specific reports.
    - Modifying counters in a graph
      To gather additional information for diagnosing performance problems, you can modify the counters that are displayed in a graph.
    - Correcting time offset
      Response time breakdown and resource monitoring data is time stamped using the system clock of the host computer. If there are differences between the system clocks of the host computers that you include in a test, then response time breakdown and resource monitoring data are skewed in reports. The best practice is to synchronize the system clocks on all computers that you include in a test. When this is not possible, you can correct the time offset of each host computer after a test run. Typically, correct the time offset on all computers to match the system clock of the workbench computer.
  - Export test results
    You can export the test result in different formats to share it with different stakeholders.
  - Viewing response time breakdown
    You can do detailed analysis of the response time to find bottlenecks in the HTTP traffic of the application.
- Logs overview
  Rational® Performance Tester uses logs to store different types of information, which you can use to determine the reason for a test failure.
- Reports and counters
  See the description of the counters for all of the performance reports.

Adding additional counters on a separate page

You can add additional counters on a separate page without editing an existing report to investigate performance problems in detail.

Before you begin

You must have created and run the test or schedule.

About this task

Each report has its pre-defined counters that gather statistical information from the recorded test. If the counter information provided in the default reports do not address your needs, you can add additional counters on a separate page to diagnose the performance problems in detail.

You can click the Menu icon quick_view_menu and select Hide All Instances check box to hide the instance counters from the counter tree. Similarly, you can click the Menu icon and select the Hide Percentile check box to hide the percentile counters from the counter tree.

For example, when you want to add counters for the test report, you might want to hide percentile counters, which are specifically useful for the schedule runs.

Procedure

Double-click the report in the Test Navigator.
Click the Plus icon .
A new page is displayed along with the execution report with a counter tree on the left pane.
Click the Expand icon from the counter tree to view the available counters.
Alternatively, you can use the Search field to search the available counters by name.
Select the check box preceded with the counter name that you want to analyze.
For example, to analyze the standard deviation for the response time of all the pages:
1. Click Pages > Response Time > All pages.
2. Select the check box preceded with the counter name StdDev. The StdDev counter statistic is displayed as a graph.
The selected counters are displayed in a graph in the right pane.
Rename the new tab for the new page by completing the following steps:
1. Click Up arrow of the new tab.
2. Select Rename.
3. Enter a new name for the new tab, and then click OK.
  The new page that you created is saved with the name you entered.

Results

You have added additional counters to be displayed on a separate page. The default chart as a line chart is displayed.

What to do next

You can change the graph view for the selected counters. See Displaying counter data in tables or as graphs.