Running test elements in random order

You can record multiple user scenarios in a test and then run each scenario in a random order. To do this, you put each scenario under a random selector and then select the proportion of time that the scenario should be run.

About this task

For example, you can record a test that includes logging on to a system, browsing through items in the system, buying various items, and then totaling the order. In this case, you could run the logging in and the totaling scenarios once, but put the browsing and buying scenarios under a random selector.

Procedure

  1. In the Test Navigator, browse to the test and double-click it.
    The test opens.
  2. Click the test element that will be controlled by the random selector, and then click Insert > Random Selector.
    Use Shift+Click to select multiple elements.
  3. You are asked whether you want to move the selected elements into a new random selector. Click Yes.
    Click No to insert an empty random selector into the test.
    Note: To set whether or not elements are moved automatically, or whether you are prompted, click Window > Preferences > Test > Test Editor, and click the General tab.
  4. Set the weight of the random selector. The weight determines the statistical probability that a specific element will be selected.
    1. If you have added a number of test elements, the Create weighted blocks window is displayed. You can select adjacent elements and group them. Each element—whether in a group or by itself—must be weighted.
    2. If you have added only one test element, the weighted block is displayed in the Test Element Details area with a default of 1.

Example

When a selector contains many different weights, you can mathematically determine the likelihood that a block will be executed. To do this, add the weights together and divide the weight for each block by that total.

For example, assume a selector contains six blocks set to the following weight:

  • two blocks set to a weight of 1
  • one block set to a weight of 2
  • two blocks set to a weight of 5
  • one block set to a weight of 9
The total of the weights is: 1 + 1 + 2 + 5 + 5 + 9 = 23. Therefore, the statistical likelihood of selection is:
Weight of block Likelihood of block being selected
1 (two blocks) 1/23. = 0.0435, or about 4.35% (for each block)
2 2/23 = 0.0870, or about 8.70%
5 (two blocks) 5/23 = 0.2174, or about 21.74% (for each block)
9 9/23 = 0.3913, or about 39.13%
Note that a higher weight increases the likelihood, but does not guarantee, that a block will be executed. Some variation might occur. For example, if you run a test 23 times, you cannot predict that the first and second blocks will execute exactly once, the third block exactly twice, the fourth and fifth blocks exactly five times, and the sixth block exactly nine times. However, the more times that the blocks are executed, the more accurate this prediction is.