Setting the data that the test log collects

The test log shows the events that occurred during a run. By setting the level of information that is collected for a schedule run, you can control whether you receive individual response-time statistics for Page Percentile reports and information about verification points. You can set the level of detail for each type of event: errors, warnings, and other events.

About this task

The level of information collection directly affects log sizes. Depending on the setting that you select, the logs can become quite large. By limiting the log level and collecting the information from a representative sample of users, you can decrease your log size and still have sufficient information for analysis.

For example, if you are debugging a test, you might set all three What to Log fields to All or Action Details. These settings produce large test logs, especially if your tests are long or you are running a large number of users. Large test logs, in turn, increase the test log transfer time, and might even cause your computer to run out of disk space or the agent computer to run out of memory. To reduce transfer times and the likelihood of running out of disk space, sample information from a very small subset of users; smaller even than the default of 5 users per user group. A fixed sampling rate samples the same number of virtual users from each group. A percentage sampling rate samples a percentage of virtual users from each group, but guarantees that at least one user is sampled from a group.

Procedure

To set the amount of information collected in the test log and the rate of sampling:

  1. In the Test Navigator, browse to the schedule, and double-click it.
    The schedule opens.
  2. In the Schedule Contents area, click the name of the schedule.
  3. On the Test Log page, select the types of events that you want to collect under What to Log. You can collect errors only, errors and warnings, or all events. In other words, Also show warnings and And also show all other types are unavailable until you select Show errors and failures. Similarly, And also show all other types is unavailable until you select Also show warnings. If none of the What to Log check boxes are selected, no test log events are collected.
  4. For each type of event, set the Log Level to one of the following options:
    OptionDescription
    Schedule Actions Collects events that correspond to actions executed in the schedule:
    • The overall schedule verdict. The verdict can be one of these values:
      • Pass indicates that all verification points matched or received the expected response. For example, a response code verification point is set to PASS when the recorded response code is received during playback. If your test does not contain verification points, PASS means that all primary requests in the test were successful.
      • Fail indicates that at least one verification point did not match the expected response or that the expected response was not received.
      • Error indicates one of the following results: a primary request was not successfully sent to the server, no response was received from the server for a primary request, or the primary request response was incomplete or could not be parsed.
    • The start and stop time of the schedule, each user group, each virtual user, and each test invocation.
    • The start and stop time of each loop iteration, if loops are set in the schedule.
    • The start and stop time of each selector, if selectors are set.
    Primary Test Actions Typically, you set data collection at this level. Primary test actions include schedule actions, plus the following actions:
    • Test verdict, test start, and test stop events.
    • Loop iteration start and loop iteration stop events, if loops are present in the test.
    • Transaction start and stop events if transactions are present in the test.
    • For HTTP tests, Page title verification points. With this option you can see any page title verification points that you have set. The following events are collected:
      • The page verdict. You see a page verdict only if a connection problem occurs or if you have set verification points. Any failures or errors are rolled up to the test verdict level.
      • The start and stop time of each page.
      • The start and stop time of each loop, and the number of iterations of each loop, if you have set loops within a page.
      • The start and stop time of each transaction, and the duration of each transaction, if you have set page-level transactions in your test.
    • For SAP tests, SAP screen information, such as SAP screen title verification points.
    • For Citrix tests, connection elements, window events, and image synchronizations
    • For socket tests, connect, send, receive, and close elements.
    Secondary Test Actions Secondary test actions include primary test actions, plus this information:
    • For HTTP tests, request-level events. To collect information about response code or response size verification points that you have set, set data collection at this level of detail or greater.
      • The time that the first byte and last byte were sent.
      • The time that the first byte and last byte were received.
      • The character set of the response data.
      • Expected and actual values of page-level verification points that you have defined.
      • HTTP think events.
      • The start and stop time of each transaction, and the duration of each transaction, if you have set request-level transactions in your test.
    • For SAP tests, SAP element information (primarily Set Property or Call Method actions).
    • For Citrix tests, synchronization points, delays, text elements, and logoff elements.
    • For socket tests, this option does not apply.
    Action Details Action details include secondary test actions, plus this information:
    • For HTTP tests, request and response data; for example, HTTP headers and any request data.
    • For SAP tests, think time information.
    • For Citrix tests, think time information, mouse actions, and keyboard actions.
    • For socket tests, this option does not apply.
    All For HTTP, SAP, and Citrix tests, All and Action Details provide the same information.

    For socket send and receive actions, the exchanged data is also available in the test log, by means of attachments.

  5. To set a sampling rate, select Only sample information from a subset of users.
    The number or percentage that you select is applied to each user group. If you are running user groups at remote locations (that is, on agent computers), the number or percentage that you select is distributed evenly among each location.
    OptionDescription
    Fixed number of users The number is applied to each user group. Assume that a schedule contains two user groups. One group contains four users, and another group contains 1000 users. If you specify 2 for this option, two users are sampled from each group.
    Percentage of users The percentage is applied to each user group, but at least one user will be sampled from each group. Assume that a schedule contains two user groups. One group contains four users, and another group contains 1000 users. If sampling rate is set to 10%, one user is sampled from the first group, and 100 users are sampled from the second group. Similarly, if sampling rate is set to 25%, one user is sampled from the first group, and 250 users are sampled from the second group.
  6. If you want to log health failure events irrespective of the standard Test Log settings, enter a value in Number of health failure events to log (regardless of logging level). By default, this option is set to 50 and is useful to limit the number of events in the Test Log for playbacks that are expected to generate many failures.
    OptionDescription
    Fixed number of users The number is applied to each user group. Assume that a schedule contains two user groups. One group contains four users, and another group contains 1000 users. If you specify 2 for this option, two users are sampled from each group.
    Percentage of users The percentage is applied to each user group, but at least one user will be sampled from each group. Assume that a schedule contains two user groups. One group contains four users, and another group contains 1000 users. If sampling rate is set to 10%, one user is sampled from the first group, and 100 users are sampled from the second group. Similarly, if sampling rate is set to 25%, one user is sampled from the first group, and 250 users are sampled from the second group.

    With the default settings for the Test Log, where Show errors and failures option is set to All, the value specified in this field is relevant only if you are sampling for a subset of users. If you are not sampling for a subset of users, all the errors or failures will be logged regardless of the value in this field.

    If you clear the Show errors and failures or select the Only sample information from a subset of users, then you can use Number of health failure events to log (regardless of logging level) option to control how many failures appear in the test log. You can further refine which failure events are logged by setting the Affects page health to No in the Error Handling page.

    Perform the following sub-steps to minimize the logging of failure events:

    1. Clear all the What to Log options.
    2. Set the Number of health failure events to log (regardless of logging level) to 0.
    3. Disable the Execution Event Console by clearing the When schedules start check box in the Execution Event Console settings because failures are logged if they are reported in the Execution Event Console page.

Example

The default setting, to log all errors and warnings, as well as primary test actions, fits most purposes. However, you can log any type of information, from no information to all information from all users, although neither is a typical situation.
  • To see only errors and warnings, set the first two What to Log check boxes to All and clear the third check box, And also show all other types, to avoid logging successful events.
  • To check a schedule's structure, when you are not interested in the test execution results, set all three What to Log boxes to Schedule Actions.

    Both choices, as well as the default setting, will limit the size of the test log and reduce the total time to run the schedule by significantly shortening the test log transfer time at the end of a test.

If you are debugging a test, you might set all three What to Log fields to All or Action Details. These settings produce large test logs, especially if your tests are long or you are running a large number of users. Large test logs, in turn, increase the test log transfer time, and might even cause your computer to run out of disk space.