Verification point comparator

You can use comparators to compare verification point data after you play back a script with a verification point and to update the baseline file. If the verification point fails, the comparator shows both expected and actual values so that you can analyze the differences. You can then load the baseline file and edit or update it with the values from the actual file.

Opening the verification point comparator

Click View Results in the Rational® Functional Tester HTML log to open the comparator.
Notes:
  • If you encounter an error regarding the Java plugin when trying to launch the comparator from View Results in the HTML log, you must configure your plugin properly.
  • You must use Microsoft Internet Explorer to open View Results as browsers such as Mozilla Firefox and Google Chrome are not supported.
  • With automatically enabled test environments, you cannot open the verification point comparator by clicking View Results in the functional test HTML log. In such cases, open the corresponding project log file from the functional test project log in the Functional Test Projects view.

Using verification point comparator for functional test scripts played back from Rational® Quality Manager

If you play back the script from Rational® Quality Manager, and click View Results in the detailed log to open the comparator, you are prompted to log in to Rational® Quality Manager. This occurs when you use the Load baseline to edit or the Replace baseline with actual value functions.
Note: To open the comparator from the Rational® Quality Manager detailed playback log, ensure that Rational® Functional Tester is installed on the workstation where you are opening the log. Additionally, ensure that the Next-Gen plugin is disabled on the workstation.

When you open the comparator, the Log In to Rational Quality Manager dialog box is displayed and the Rational® Quality Manager server name and project area are displayed. You must specify a valid user name and password to log in to Rational® Quality Manager.

The Log In to Rational Quality Manager dialog box is displayed only the first time you use Load baseline to edit or the Replace baseline with actual value functions during an active Windows session. You are not prompted to log in a second time unless you have started a new Windows session and logged on to Windows as a different user.

Comparing verification points after playback

If you have one failed verification point, and you are using a log, select the log in the Functional Test Projects view. Right-click the log, and click Failed Verification Points. The verification point comparator is displayed.

If you have multiple failed verification points, and you are using a log, the Results for Verification Points wizard is displayed. Click a failed verification point in the list and click View Results or Finish. The comparator banner displays the name of your verification point.

You can specify color settings for several elements in the verification point comparator.

To edit verification point data, you must load the baseline by clicking the Load Baseline to Edit toolbar button Load Baseline to Edit button.

The verification point comparator window

The following sections describe the various components of the verification point comparator window and the toolbars.

Metadata

Metadata is displayed in the left pane of the verification point comparator window. It displays a set of properties that define how specific data is managed. You can edit this grid. For example, you could edit the ignore case or white space rule in a text verification point in this metadata grid. To edit, double-click the value in the Value column.

Main toolbar

The toolbar at the top of the verification point comparator has six buttons.

File: Save button File: Save: Saves any changes you have made.

File: Revert button File: Revert: Reverts to the state of the data at the last save you made. If you have not saved any changes since opening the comparator, it reverts to the state when it was opened. If you have edited and saved the changes, it reverts to the state at your last Save.

Load Baseline to edit button Load Baseline to Edit: Loads the baseline file so you can edit it. The baseline values are displayed instead of the expected values. These values can be edited individually or replaced with actual values.

Replace Baseline with actual value button Replace Baseline with actual value: Replaces the baseline values with all the values in the actual file. Then those values will become the baseline for future playbacks. If you want to replace only some of the values, edit them individually. This command replaces the entire file.

Hide/Show TestObject Info button Hide/Show TestObject Info: Toggles the display of the Test Objects and Recognition Data panes of the comparator window. When this information is hidden, the entire comparator window is used for the main data area. This is a sticky setting and is displayed as you last set it when you open the comparator later.
Note: If your test objects tree has multiple nodes, the verification point comparator displays these panes again the next time you open it, regardless of the Hide/Show TestObject Info setting.

Help button Help: Displays the help documentation for the verification point comparator. You can open the Rational® Functional Tester help any time from the Help menu in Rational® Functional Tester.

Menu bar

The menu bar contains the same commands that are represented with the toolbar buttons described in this topic.

File: Displays the commands Save, Revert, Baseline, and Replace.

Edit: Displays the commands Check All, Uncheck All, and Hide. This menu is grayed out until you load the baseline for editing (using the Load Baseline to edit toolbar button Load Baseline to edit button).

Difference: Displays the commands First, Previous, Next, and Last.

Test Object > Highlight: You can use this menu item if you need to verify an object in the application. If your test application is open, you can select an object in the test objects tree and then click this command to see the object highlighted in the application.

Preferences : Toolbars: You can use this menu item to control the display of the toolbars.
  • Test Object Appearance on the Tree: Displays the Edit Test Object Description dialog box, which allows you to customize the text displayed for each object in the Test Object Hierarchy.
  • Hide TestObject Info: You can use this command to toggle the display of the Test Objects and Recognition Data panes of the comparator window.

Help: Displays help documentation for verification point comparator.

Main data area

The right pane of the verification point comparator displays the verification point data. For example, in the case of a properties verification point, the Property and Value columns are displayed here. You can compare the verification point data here. If the verification point fails when you play back the script, the expected and actual values are displayed, irrespective of the type data display being used. In certain cases, the expected values are shown on the left pane and the actual values are shown on the right pane of the verification comparator window. In other cases, the values are displayed contiguously (such as nodes in a tree view), and the expected and actual values are shown in different colors if they are different. The expected value is red and the actual value is green in color. The actual values those that were recorded when you playback the script.

You can get seven types of displays from recording verification points, as described in the following sections, after the next section, Navigation Toolbar Buttons.

Navigation toolbar buttons

These four navigation buttons jump to the differences between the expected and actual files or the baseline and actual files. Differences are shown in red. The currently selected difference is highlighted.

Jump to First Difference button Jump to First Difference: Goes to the first difference in the expected/baseline and actual files.

Backward to Previous Difference button Backward to Previous Difference: Goes backward to the previous difference in the expected/baseline and actual files.

Forward to Next Difference button Forward to Next Difference: Goes forward to the next difference in the expected/baseline and actual files.

Jump to Last Difference button Jump to Last Difference: Goes to the last difference in the expected/baseline and actual files.

You can get the following types of displays after recording a verification point.

Properties verification point : grid display

When you create a properties verification point, the object properties are displayed in a grid format. The properties displayed on the grid belong to the object that is highlighted in the Test Objects tree. The properties appear are displayed in the left column and their values appear are displayed in the right column of the object properties grid.. You can edit which properties get tested in the Property column, and can edit the property values themselves in the Value column.

Properties with no check mark are not tested. You can select which properties you want to test by checking each of them. The checked properties are tested each time you play back a script with this verification point. You can check all properties in the list by clicking the Check All toolbar button Check All button . You can use the Uncheck All button Uncheck All button to clear all properties. Depending on how many properties you want to test, it is often easiest to either select or clear all of them using one of those buttons, and then individually select or clear exceptions.

The grid uses a nested tree hierarchy. If a folder shows up on the list, you can expand it by double-clicking it or selecting the expand icon. If you check or clear the folder icon itself, all the properties underneath are either tested or not tested.

To edit a value, you must double-click the grid cell. Click outside the cell to make the edit take effect. In most cases, double-clicking a value makes the cell an editable field, and you can just change the value. In some special cases, another dialog box is displayed which contains the information. For example, if the property is color, when you double-click the color value, the standard color dialog box is displayed. You must your edit there and close the color box. In other cases, a drop-down list might be displayed in the Value column when you double-click a value. For example, values that are either true or false are displayed in a drop-down list.

The grid has the following toolbar buttons for the properties verification point display. In the comparator, these buttons are displayed only when you are editing the baseline.

Check All button Check All: Includes a check mark in front of every property in the list. Checked properties are tested each time you play back the script with this verification point. Only checked properties are compared in the Comparator.

Uncheck All button Uncheck All: Clears the check mark in front of every property in the list. Do not test the cleared properties when you play back the script with this verification point.

Hide the Unchecked Properties/Show All Properties button Hide the Unchecked Properties/Show All Properties: Click Hide the Unchecked Properties to hide the cleared properties. Then you only view the properties that are tested. Click Show All Properties to display all properties, including any cleared ones.

The grid has the following pop-up menu commands for the properties verification point display. To access them, right-click a value in the Value column.

Open: Displays the value in a separate window if the value is a string or a complex value type which enables you to see long lines of text and makes it easier to edit.

Case Sensitive Regular Expression Case Sensitive Regular Expression: Toggles case-sensitive regular expression comparison on and off.

Evaluate Regular Expression Evaluate Regular Expression: Displays the regular expression evaluator, which enables you to test the regular expression before you use it in a verification point.

Convert Value to Regular Expression Convert Value to Regular Expression: Converts the property value to a regular expression.

Undo/Redo Regular Expression Undo/Redo Regular Expression: Cancels or redoes the regular expression conversion.

Convert Value to Numeric Range Convert Value to Numeric Range: Converts the property value to a numeric range.

Convert Value to Numeric Range Undo Numeric Range: Cancels the numeric range.

Convert Value to dataset Reference Convert Value to dataset Reference: Uses a dataset reference to use a dataset instead of a literal value in a verification point.

Undo dataset Reference Undo dataset Reference: Cancels the dataset reference in the verification point.

Replace Baseline On Current Selection Replace Baseline On Current Selection: Replaces the baseline value with the actual value for just the selected property. This is a per-property version of the Replace Baseline With Actual Value toolbar button Replace Baseline With Actual Value.

Compare object properties

To compare object properties, look at the expected or baseline values and actual values columns. The actual values are those that were captured when you played back the script. You can use the navigation buttons to navigate to all the differences, which are displayed in red. You can edit the baseline values or replace the baseline with the actual file.

Data verification point : menu hierarchy display

When you create a data verification point and choose the menu hierarchy or menu hierarchy with the properties test, the menus are displayed in a tree format in the main data area. Menu hierarchy and menu hierarchy with properties are two examples. The list of tests shown in the Data Value field is dependent on information provided by the object's proxy. Values other than these two may be displayed.

The tree displays the entire menu hierarchy of your application, or one top-level menu and its subitems, depending on how you recorded the verification point. If you chose the whole menu bar, each top-level menu is displayed in the tree, in the same order they are displayed in the menu bar. Each individual menu item is displayed under its top-level menu. You can use the plus and minus signs to open and close the list for each top level menu.

To edit a menu, double-click it in the tree. You must load the baseline first before doing this. The menu properties displayed in a grid, which you can then edit. You can edit the actual values by double-clicking a value in the Value column. You can also edit the list of properties that are tested during playback by using the checkbox beside each property. The checked items are tested. The toolbar buttons above the grid are the same ones that are found in the object properties grid, except for Hide/Show. The buttons work the same, except they apply to the selected menu property or value.

Compare menu hierarchy data

To compare menu hierarchy data, look at any differences shown in red and green. The expected values are displayed in red, and the actual values are shown underneath them in green. The actual values are what were captured when you played back the script. If the descriptions for the expected and baseline values are the same, but if there are some differences in their properties, the node is displayed as blue in color. You can use the navigation buttons to navigate to all the differences. You can edit the baseline values or replace the baseline with the actual file.

Data verification point : text display

When you create a data verification point and choose the Visible Text test, the text is displayed in a text box format in the main data area. For example, visible text. The list of tests shown in the Data Value field is dependent on information provided by the object's proxy. Values other than this one may be displayed.

The text is displayed in a text box area. You cannot edit directly in this area. To edit the verification point data, click Edit Text Edit Text button above the data display. You must load the baseline file before doing this. A small text editor containing the text is displayed. You can edit the text in this editor, and when you close it, the edited text is displayed in the baseline column of the comparator.

Compare text data

To compare text data, look at the expected and actual values columns. The actual values are those that were captured when you play back the script. You can use the navigation buttons to navigate to all the differences, which is displayed in red. You can edit the baseline values or replace the baseline with the actual file.

Data verification point : table display

When you create a data verification point and choose the table contents or selected table cells test, the table data is displayed in a table in the main data area. Table Contents and selected table cells are two examples. The list of tests is displayed in the Data Value field is dependent on information provided by the object's proxy. Values other than these may also appear.

The table displays the same information as the table in your application. To edit the verification point data, double-click any cell in the table to edit that cell. You must load the baseline file before doing this.

You can also edit which cells in the table get tested. Table cells that are within the comparison regions are shown with a grey background. If you are testing the entire table, all cells will be grey. You can use the drop-down list in the toolbar above the data region as a selection mechanism. (This doesn't show up until you load the baseline.) Choose Column, Row, or Cell Selection in the list, then make your selections in the table. For example, if you select Row Selection, when you click a cell in the second row, the whole second row will be selected. If you had chosen Cell Selection, only that cell would have been selected. After you select the data you want to compare, click the Update Comparison Region button Update Comparison Region to have your changes take effect.

The Cut, Copy, Paste, and Delete toolbar buttons above the table area apply to the selected row(s), and are only applicable within the Verification Point Comparator. (It does not use the system clipboard.)

You can right-click a table item to access a pop-up menu. The commands are the same as those listed above in the Properties Verification Point--Grid Display section.

There are features in the Metadata tab that you can also use to edit the table data. For example, you can edit the table's column headers or row headers by accessing them in the MetaData tab. To edit column headers, double-click the Value column of the columnHeaders property. A small editor opens that lets you edit the headers. The row headers work the same way if your table has them. Double-click the rowHeaders Value to edit them. In order for the column headers to be compared, you must change the compareColumnHeaders property to true in the MetaData tab. The compareRowHeaders value works the same way to indicate whether row headers will be compared.

If you double-click the Value of the compareRegions property in the Metadata tab, an editor will open showing the selected regions of your table. For selected sells, it shows the row index or key value pairs and the column header or index of each selected cell. For selected rows, it shows the row index or key value pairs. For selected columns, it shows the column header or index. Using this compare regions editor is another way you can select which regions get compared. If you click the Compare All Cells buttonCompare All Cells in this editor, all of the table cells will be tested.

If your table supports row keys or column keys, you can edit those and insert keys by double-clicking on the columnKeys and rowKeys values in the Metadata tab.

Compare table data

To compare table data, look at the expected and actual values columns. The actual values are those that were captured when you played back the script. You can use the navigation buttons to navigate to all the differences, which appear in red. You can edit the baseline values or replace the baseline with the actual file.

Data verification point -- tree hierarchy display

When you create a Data verification point and choose the Tree Hierarchy test, the data is displayed in a tree format in the main data area. For example, Tree Hierarchy. The list of tests shown in the Data Value field is dependent on information provided by the object's proxy. Values other than this might be displayed.

The tree displays the entire tree hierarchy in your application or the part of the tree selected when you create the verification point. Each item in the tree is displayed in the same order it is displayed in your application. Each individual item is displayed under its top-level item. You can use the plus and minus signs to open and close the list for each top-level item.

To edit an item in the hierarchy, double-click it in the tree. A small text box is displayed, which you can use to edit the item.

Compare tree hierarchy data

To compare tree hierarchy data, look at any differences displayed in red and green. The expected values are displayed in red, and the actual values are displayed in green. The actual values are those that were captured when you played back the script. You can use the navigation buttons to navigate to all the differences.

Data verification point : list display

When you create a data verification point and choose the List Elements test, the data is displayed in a list format in the main data area. List Elements is one example. The list of tests shown in the Data Value field is dependent on information provided by the object's proxy. Values other than this might be also displayed.

The list displays the same information as the list in your application, and in the same order. To edit a list item, double-click it in the list display. (If you have not done so, you must load the baseline first.) You can also edit the list of which items get tested during playback by using the checkbox beside each item. The checked items are tested.

The toolbar buttons preceding the list are the same ones that are found in the object properties grid described above in the Properties Verification Point : Grid Display section. The buttons work the same as described there, except they apply to the selected list item(s).

You can right-click a table item to access a pop-up menu. The commands are the same as those listed preceding the Properties Verification Point : Grid Display section.

Compare list data

To compare list data, look at the expected and actual values columns. The actual values are those that were captured when you played back the script. You can use the navigation buttons to navigate to all the differences, which are shown in red. You can edit the baseline values or replace the baseline with the actual file.

Data verification point : state display

When you create a data verification point and choose the checkbox Button State or Toggle Button State test, the data is displayed in a list format in the main data area. checkbox Button State or Toggle Button State are two examples. The list of tests displayed in the Data Value field is dependent on information provided by the object's proxy. Values other than this may be also displayed.

Compare state data

To compare state data, look at the expected and actual values columns. The actual values are those that were captured when you played back the script. You can edit the baseline values or replace the baseline with the actual file.

Test object data in the Verification point comparator window

While inserting the verification points, if you have not checked the Record Test Object relative Verification Points option available in the General Recorder page of the Windows > Preferences window, you can view the following test object data in the Verification Point comparator:
  • Test objects
  • Recognition and Administrative data

Test objects

This is the upper left pane of the Verification Point Comparator window. It is a partial version of the script's object map. This hierarchical display includes only the objects in your verification point. You cannot edit the Test Objects tree. You can choose an object within it and edit its properties or data in the right pane of the Verification Point Comparator window.

You can double-click folders in the tree to expand and collapse the objects beneath them. You must an individual object in the tree to see its properties or data in the right pane.

The checkboxes to the left of each node Verification Point Comparator window indicate whether that node is tested or not. Checked items get tested. After you load the baseline to edit, you can select or clear items.

Note: If your test application is open, you can select an object in the Test Objects tree and then click Test Object > Highlight from the Verification Point Comparator menu to see the object highlighted in the application. You must use this feature if you need to verify an object in the application.

Recognition and Administrative data

This is the lower left pane of the Verification Comparator window. The Recognition tab displays recognition data used by Rational® Functional Tester and is not editable. Some of these properties are the recognition properties that were listed in the Select an Object tab of the Verification Point and Action Wizard when you created the verification point. The Administrative tab displays internal administrative data of the object and is not editable. These properties are used to manage and describe the test object. Recognition and administrative data are the properties from the script's object map used to locate and manage this test object in the context of the associated script. You can use this information to determine what test object this is in the associated application under test.

The MetaData tab displays a set of properties that define how specific data is managed. This grid can be edited if you load the baseline. For example, you could edit the ignore case or white space rule in a text verification point in this metadata grid. To edit, double-click the value in the Value column.

The Recognition and Administrative properties are a snapshot of the object map properties for the test object at the time the verification point was created. They become historical information as the application evolves.

Note for ClearCase users

If you use the Rational® Functional Tester ClearCase® integration, you can check out your verification point files from the Comparator.

If the verification point baseline is not editable and is checked in, if you replace your baseline file (by clicking File > Replace or the Replace Baseline with Actual Value toolbar button Replace Baseline With Actual Value), Rational® Functional Tester will do an unreserved check-out of the scripts associated with the verification point.

If the verification point baseline is not editable and is checked in, if you load the baseline file (by clicking File > Baseline or the Load Baseline to Edit toolbar button Load Baseline to edit button), Rational® Functional Tester will open the ClearCase® check-out dialog box to allow you to check out the files, reserved or unreserved, if you want to. If you check out the files, when you click Finish the scripts will be checked out, and the baseline will be loaded and become editable. If you click Cancel, the baseline is loaded but is not editable.

If the verification point baseline is not editable and is not checked in, you cannot replace the baseline (the File > Replace menu and the Replace Baseline with Actual Value toolbar button Replace Baseline With Actual Value are disabled).

If the verification point baseline is not editable and is not checked in, if you load the baseline file (by clicking File > Baseline or the Load Baseline to Edit toolbar button Load Baseline to edit button), Rational® Functional Tester does not open the ClearCase® check-out dialog box. The baseline is loaded but is not editable.