Defining requirements in tests

You can define requirements for elements in a test. These requirements specify acceptable thresholds of performance and validate service level agreements. Starting from version 9.2.0.1, you can define both performance and functional requirements in the tests. The verdict of the test is computed based on the requirements defined in the schedule. You can view the verdict in the Requirements report.

About this task

You can set requirements on protocol-specific test elements, on schedule elements, on data created by custom code, and on collected resource usage data. You define a requirement as standard or supplemental. A standard requirement determines that the requirement is significant enough to cause the entire run to be declared a failure if it fails. A supplemental requirement, although important, is not significant enough to cause the run to fail. For example, a supplemental requirement might be a request from development to validate a very specific data item provided by WebSphere® PMI monitoring.

Procedure

To define a requirement for the elements in a test:

  1. In the Test Navigator, browse to the test and double-click it.
    The test opens.
  2. In the Test Contents area, select the page or the request that will have the requirement.
    You can select multiple pages or multiple requests.
  3. In the Test Element Details area, click the Advanced tab, and select Enable Requirements.
    A table of requirements that apply to the page or to the request is displayed.
  4. Click the requirement to define, and add a definition, as follows:
    OptionDescription
    Name You can change the name of a requirement to improve readability. However, changing a requirement name causes a mismatch between the Requirements report, which uses the changed name, and the other reports, which use the default name. Therefore, when you change a requirement name, be sure to keep track of the original name.
    Operator Select an operator.
    Value Type a value.
    Standard Select to make the requirement standard. A standard requirement can cause a test to have a verdict of fail. Clear to make the requirement supplemental. In general, supplemental requirements are used for requirements that are tracked internally. A supplemental requirement cannot cause a run to fail, and supplemental results are restricted to two pages of the Performance Requirements report.
  5. Optionally, apply the defined requirement to other test elements:
    1. In the Test Contents area, select the test elements that will have the requirement.
      The elements must be of the same type, for example, all page elements.
    2. In the Requirements table, right-click the requirement row, and select Copy Requirements.
  6. Optionally, select Hide Undefined Requirements to hide the shaded rows, which indicate that a requirement is not defined, and improve readability.
  7. Select a requirement and click Clear to remove its definition. The requirement is still available and can be redefined.
  8. After you have defined a number of requirements on test elements, you might want to see all of the requirements defined for the test. To do so:
    1. In the Test Contents area, click the name (root) of the test.
    2. In the Test Element Details area, sekect the Requirements category.
      The Requirements page displays a summary of the performance and functional requirements defined in the test.
    3. To navigate to the original requirement definition, double-click the requirement row.

Example

You can define requirements in a test or in a schedule. When you define a requirement in a test, the requirement is defined individually for each test element—even if you select multiple test elements and apply the requirement to all of them at the same time. When you define a requirement in a schedule, the requirement is applied to the aggregate of test elements.

For example, assume that you select every page in a test and define this requirement: Average response time for page [ms] [For Run] must be less than 5 seconds. This means that if one page in the test has a response time of 6 seconds, the requirement on that page fails. The other pages, which have a response time of less than 5 seconds, pass.

Assume that you open a schedule and define this requirement: Average response time for all pages [ms] [For Run] must be less than 5 seconds. This measures the average response time for all of the pages. One page can have a response time of 30 seconds, but if enough pages have a response time low enough to counter the negative effect of that one page, the requirement passes.

For information on defining requirements in schedules, see Defining requirements in schedules.