Modifying a step target using an image as the main property

To fix recognition issues related to image correlation, you can change the candidate image for the test step, and perform a preview to see if the image is the appropriate one. You can modify the recognition threshold or the aspect ratio tolerance that is automatically applied for data correlation when the test is run to evaluate which images are the best match.

About this task

When a test is recorded, the graphic elements on which the actions are performed are not always easily identifiable; there might not be content description or label or text that would help identify or locate the objects on a mobile device screen. In such cases, the test generates steps that identify their target objects with images. Image correlation is applied when the test is played back to find the accurate images on which the actions are performed. But in some cases, the test report indicates that the generated image is not the right target element in a step, because the image is different when the test is played back on a mobile phone or on a tablet, for example (different width and height), or because a blurred image was generated.

If the element on which the action is performed is not recognized on a test step as expected, replace the generated element or modify the recognition threshold or tolerance ratio as needed. This task applies to tests recorded from Android and iOS Mobile native applications only.

Procedure

You can replace the generated image with an image captured during test recording and displayed in the SmartShot View.
  1. In the edited test, click the test step to be modified. In the Screen Capture view, select an image. The image is highlighted.
  2. If the 'image' property was generated automatically, drag the image in the Object identified by field. If the Object identified by field is not set to 'image', but to 'text', 'hint', or 'content description' , right-click the image property in the Properties table and select Identify step target using property image. Another method is to drag the image property on the Object identified by field, or use the copy and paste context menu in the property table. The image property is selected in the Object identified by field and the image becomes visible in the User Action Details area.
    Figure 1. Step target identified by an image propertyThe image is displayed in the User Action Details area
    Another way to add or modify an image on a test step, once the image identifier is selected, is to use the context menu items or the icons. You can choose an image on the clipboard, or in a file.
    Figure 2. Adding new images or replacing the existing image with another one in the clipboard or in a fileAdd an image or modify the image from the context menu or icons

    You can add multiple images on a test step so as to find the accurate one when the test is played back. You can remove the additional images using the Remove image icon Remove image.

You can replace the generated image with a candidate image in the test report, save the candidate image locally, and use it in the test step as a reference image.
  1. In the test report, click the image correlation details link on a test step. You can see if the candidate image and the reference image are matching.
  2. If the candidate image selected with the best threshold score is not appropriate, select another one. Right-click a candidate image and in the browser that opens, save the image as a .png file. In the test step, select the context menu Add new image from file, and select the .png file. It is added as a new reference image. Execute the test again.
    Figure 3. Image correlation details in the Test report The test report indicates that the candidate image and the reference image are matching on the target step
Image correlation is used to find the appropriate element on which the action is performed during play back, a recognition threshold and tolerance ratio are applied. You can modify the recognition threshold and tolerance ratio.
  1. Click the Set recognition threshold iconSet recognition threshold. The default recognition threshold is set to 80, the default tolerance ratio is set to 20. In the image matching preview view, you can see which images might be accurate to identify the target object. Green shows the best candidate, yellow is for images whose score is over the threshold and which are not the most appropriate, and red indicates that the score is under the threshold and the images are not matching. You can change the recognition threshold and the tolerance ratio and see the dynamic changes in the preview view. Move the cursors to increase or reduce the score and the ratio. You can preview the image selection in the Image matching preview view and see which image is a good candidate.
    Figure 4. Preview image matching and set the recognition threshold and aspect ratio tolerance to find the accurate imagePreview image matching, and set the recognition threshold and aspect ratio tolerance to find the accurate image.
  2. Save the test. After you complete these steps, execute the test again and verify that all steps are successful in the test report.