Configuration of test runs from the command line

When you need to run Web UI, mobile, and windows tests using the command line without opening the desktop client to run these tests, you can use the command line of Rational® Functional Tester.

Overview

Rational® Functional Tester supports the usage of command line for certain tasks. To use the command line, you must go to the following directory:
  • <Install_Directory>\cmdline directory that contains cmdline.sh on Linux operating system.
  • <Install_Directory>\cmdline directory that contains cmdline.sh on Mac operating system.
  • <Install_Directory>\cmdline directory that contains cmdline.bat on Windows operating system.
After navigating to the directory, you can run the cmdline file. The command line supports a set of options. You must use the cmdline command with the supported options. You can enter cmdline -help in the command line to view all the supported options.

To run the tests from the command line, you must specify the path of the tests along with the command as follows:

cmdline -workspace <workspace_full_path> -project <project_relative_path> -suite <suite_relative_path>

For example:

cmdline -workspace “D:\My Workspace” -eclipsehome "C:\Program Files\IBM\SDP" -plugins "C:\Program Files\IBM\IBMIMShared\plugins" -project myProject -suite mytestsuite
Note:

The -workspace option is followed by a value that contains a space. If the value contains space, then you must enclose the value, D:\My Workspace within quotes. Otherwise, you can provide the value without quotes.

In addition to the supported options, you can create and use varfiles and config files as options. The files can contain specific paths and parameters for running tests on connected systems.

Using a configuration file

You can run tests from the command line by using a configuration file. Each command line option must be followed by an appropriate value.

The contents of a sample configuration file, config_file1 are as follows:
workspace=D:\My Workspace
eclipsehome=C:\Program Files\IBM\SDP
plugins=C:\Program Files\IBM\IBMIMShared\plugins
project=myProject
suite=mytestsuite

To create a config file using the desktop client, see Creating a command-line config file.

To run tests from the command line by using the sample config file config_file1, you must use the following command:

cmdline –configfile <config file path>

For example:

cmdline –configfile E:\Workspace1\Project1\Tests\config_file1.txt

Using a variable file

Variable file is an xml file with the .varinit extension that contains the variable names and values as pairs that are required to connect to the system for running tests.

This is a sample XML file as a variable file that specifies the browser to use on the computer on which you have installed Rational® Functional Tester:
<?xml version="1.0" encoding="UTF-8"?>
<inits>
<variable_init value="<browser>" name="RTW_WebUI_Browser_Selection"/>
</inits>

For example, if you connect Emulator:Pixel_2_API_28 as the mobile emulator and you want to use the Chrome browser to run the test, the variable file can be as follows:


<?xml version="1.0" encoding="UTF-8"?><inits>
<variable_init value="Chrome(Emulator:Pixel_2_API_28)" name="RTW_WebUI_Browser_Selection"/>
</inits>

To run tests from the command line by using a variable file, you must use the following command:

cmdline –varfile <variable file path>

For example:

cmdline –varfile E:\Workspace1\Project1\Tests\var1.varinit

Command line options

Each command line option is used in combination with the other options to accomplish a specific task. You must read and be familiar with the supported options and mandatory options that are required for each task. The following table lists all the supported options with description:

Option Description
Required
-project The path, including the file name of the project relative to the workspace.
-workspace The complete path to the Eclipse workspace.
-suite The path, including the file name of the test relative to the project. A test can be a Web UI test, compound test, or an Accelerated Functional Test.
In a command, it is mandatory to use one of the following options:
  • -suite
  • -aftsuite
Note: You must provide the file name along with the file extension if you are using an Accelerated Functional Test suite.

You must not use the -suite option along with the other options.

Starting from 9.2.1.1, you can execute multiple tests simultaneously.

For example, -suite test1:test2:test3.

Optional
-aftsuite The path, including the file name of the xml file to run an AFT test. The aftsuite option accepts aft XML as the parameter value. It supports only one aft XML as input.

For example, aftsuite="aftinput"

You must not use the -aftsuite option along with the other options.
-compare You can use this option along with -exportstatshtml and -execsummary to export the result in the compare mode. The value can be paths to the runs and are relative to the workspace. You must separate the paths by a comma.
-configfile You can use this option to specify the complete path to a file that contains the parameters for a test run. Each parameter must be on a single line. To create a configuration file, you must use an editor that does not wrap lines. Any parameter, whether required or optional, can be set in the configuration file. The command line parameters override the values in this file.
Notes:

  • If you are creating a config file manually, the file must be in the UTF-8 format. You must not use quotation marks in this file even for values that contain spaces.

  • You can create command line config file from the desktop client, which you can use while running tests from the command-line interface or Maven. See Creating a command-line config file. This option is available only for Web UI and compound tests.
-eclipsehome

You can use this option to provide the complete path of eclipse.exe.

For example, C:\Program Files\IBM\SDP
-execsummary You can use this option to export all of the reports for the test run in a printable format, also known as an executive summary, to the local computer. You must specify the path to store the executive summary.
-execsummaryreport You can use this option to export a specific report as an executive summary for the test run to the local computer. You must specify the ID of the report to export as -execsummaryreport <reportID>.

For example, to export an HTTP performance report, specify http.

Note: You must use this option along with -execsummary.

To copy the report IDs list into your command line, navigate to Window > Preferences > Test > Performance Test Reports > Export Reports. Under Select reports to export, select the required reports, and click Copy ID to clipboard. You can then paste the clipboard content on to your command line editor.
-exportlog You can use this option to specify the file path to store the exported test log.

Starting from 10.0.1, by using the -exportlog parameter, you can provide multiple parameter entries when running multiple tests. You must use a colon to separate the parameter entries.

For example: -exportlog c:/logexport.txt:c:/secondlogexport.txt

If there are multiple -suite option entries with a single -exportlog parameter entry, then the -exportlog option generates the appropriate number of test logs by appending 0, 1, 2, and so on to the -exportlog option entry name.

For example: -suite "sampletest1:sampletest2:sampletest3" -exportlog c:/logexport.txt

The command generates the following test logs:

  • logexport_0.txt
  • logexport_1.txt
  • logexport.txt

The last test log generated has the same name as that of the initial -exportlog entry.

Note: If there are multiple -suite and -exportlog parameter entries, the number of -suite entries must match with the number of -exportlog entries. Otherwise, the following error message is displayed:

Error, number of -suite and -exportlog entries do not match.
-exportReport You can use this option to export the unified report of UI tests to the file formats such as PDF, HTML, and XML.
Note: The exported XML file is a JUnit XML file. You can view this file in applications that support JUnit reporting formats.

The command syntax is as follows:

exportReport "type=<reporttype>;format=<file type1,file type2,file type3>;folder<destination folder path>;filename=<name of the exported file>"

For example, to export the report to only the pdf format, you can use exportReport "type=unified;format=pdf;folder=Exportedreport102;filename=testreport

If you want to export the report to multiple formats, you can specify the file formats as comma-separated values. The file type value can be in uppercase or lowercase.

For example, to export the report to all the supported formats, you can use

exportReport "type=unified;format=pdf,xml,html;folder=Exportedreport102;filename=testreport

The report in different file formats use the same file name that is specified in the command.
-exportstatreportlist You can use this option to specify a comma-separated list of report IDs along with -exportstats or -exportstatshtml to list the reports that you want to export in place of the default reports, or the reports selected under Preferences. To view this setting, navigate to Window > Preferences > Test > Performance Test Reports > Export Reports.

To copy the report IDs list into your command line, navigate to Window > Preferences > Test > Performance Test Reports > Export Reports. Under Select reports to export, select the required reports, and click Copy ID to clipboard. You can then paste the clipboard content on to your command line editor.
-exportstats You can use this option to export reports in comma-separated values (CSV) format, with the file name derived from the report name. This directory can be relative to the project or a directory on your file system. If the -exportstatreportlist option is not specified, the reports specified on the Export Reports page of the Performance Test Report preferences are exported.
-exportstatsformat You can use this option to specify a format for the result that you want to export along with the -exportstats option. You must use at least one of the following parameters with the -exportstatsformat option:

  • simple.csv

  • full.csv

  • simple.json

  • full.json

  • csv

  • json

For example, -exportstats <local_dir_path> -exportstatsformat simple.json

You can add multiple arguments separated by a comma.

For example, -exportstats <local_dir_path> -exportstatsformat simple.json, full.csv

When you want to export both simple and full type of test results in a json or csv format, you can specify json or csv as the arguments in the command. When the test run completes, the test result exports to simple.json and full.json files.

For example, -exportstats <local_dir_path> -exportstatsformat json

You can select the Command Line check box from the product preferences (Window > Preferences > Test > Performance Test Reports > Export Reports) when you want to export test results to one of the selected formats after the test run completes.

Remember: When you run the test from the command line, and if you use the -exportstats parameter, then the command line preferences take precedence over the preferences set in the product. Therefore, by default, the test result exports to a CSV format.

For example, when you select the Command Line option and Report format to json in the product preferences, and run the test from the command-line interface without using the -exportstats option. The result is exported to a json file after the test run is complete.
-exportstatshtml When you want to export web analytic results, you can use this option. The results are exported to the specified directory. You can then analyze the results on a web browser without using the test workbench.
-history Use this option when you want to view a record of all events that occurred during a test run. However, you must use the command suffixed with any of the following options:
  • jaeger: To send test logs to the Jaeger UI during the test run.
  • testlog: To send test logs as traditional test logs in Rational® Functional Tester during the test run.
  • null: To send no test logs either to the Jaeger UI or Rational® Functional Tester during the test run.
For example: 
-workspace workspace_full_path -project proj_rel_path 
-suite suite_rel_path -stdout -history comma delimited list of modes
-workspace C:/Users/IBM/rationalsdp/test_ws 
-project Project1 
-suite test1.testsuite -stdout -history jaeger
Note: You can add multiple options separated by a comma to send test logs during the test run to Rational® Functional Tester and the Jaeger UI.
For example:
-workspace C:/Users/IBM/rationalsdp/test_ws 
-project Project1 
-suite test1.testsuite -stdout -history jaeger, testlog
-import When you want to run Web UI tests that are in a source control system such as Git from a computer that runs the desktop product, you can clone the project resources in the remote repository to your computer. You can use an empty workspace folder on your computer to import the UI Test project resources and then run the tests from the command-line interface. The desktop product is enabled to run the Web UI tests without the need of the workspace in the cloned repository or your existing workspace. You must use the -workspace command argument to precede the -import command argument.
Note: You can use this command argument in the following scenarios:
  • You do not want to use your existing workspace.
  • You do not want to use the workspace cloned from a remote repository.
To run UI tests contained in UI Test projects that are in a remote repository, you must perform the following steps:
  1. Clone the remote repository that contains the UI Test project to your computer.
  2. Create an empty workspace on your computer.
  3. Run the following command from the command-line interface:

    cmdline -workspace <path_of_empty_workspace> -project <project_name> -import <path_to_cloned_project_folder> -suite <test_name>

    For example, cmdline -workspace C:\workspace -project UIProject1 -import d:\work\UIProject1 -suite Test1

    You can also import multiple projects by specifying the project paths separated by a comma.

    For example, cmdline -workspace C:\workspace -project UIProject1 -import d:\work\UIProject1,d:\work\UIProject2 -suite Test1
-importzip To import the project as test assets with dependencies into your workspace, use the -importzip option. This command is available from 9.2.1.1 and later.
-labels You can use the -labels option to add labels to test results when you run test assets from the command-line interface.

You can add multiple labels to a test result separated by a comma.

For example, -labels label1, label2

Note: If the name of the label contains a space character, then you must enclose it with quotes ().

For example, if the name of the label is test environment, then you must provide it as "test environment".

You can also use the -labels option along with the -publish option to add labels to a test result when you want to publish test results to Rational® Test Automation Server.

When you run test assets from the command-line interface by using the -labels option, then the same labels are displayed on the UI Test Statistical Report in Rational® Functional Tester.

Similarly, when you use the -labels option with the -publish option from the command-line interface, then the Results page of Rational® Test Automation Server displays the same label for the specific test asset.
-overwrite

Determines whether a result file with the same name is overwritten. The default value, false, indicates that the new result file is created. If the value is true, the file is overwritten and retains the same file name. You must use double quotes for values true or false.
-plugins The complete path to the folder that contains the plugins. Typically, on Windows operating systems, this folder is located at C:\Program Files\IBM\IBMIMShared\plugins.

Required. This option is required only if the folder is at a different location.
- protocolinput You can use this option with additional arguments as follows:
  • To run a Web UI test in parallel on different browsers

    -protocolinput "all.available.targets.in.parallel=all"

    -protocolinput "all.available.targets.in.parallel=chrome,ff,ie"

    Note: If you use the -protocolinput argument, you must not use the following equivalent -vmargs arguments: 
    -vmargs "-Dall.available.targets.in.parallel=all"
-vmargs "-Dall.available.targets.in.parallel=browser1,browswer2,browser3"

  • To specify the Web UI preferences such as highlighting the page element and capturing screens

    For example, -protocolinput "webui.highlight=<value>;webui.report.screenshots=<value>" where webui.highlight specifies whether the page element must be highlighted and webui.report specifies whether the screens must be captured while playing back the test in the browser.

  • To run only the failed tests from a previous playback in an Accelerated Functional Test suite

    cmdline -workspace workspacename -project projectname -aftsuite aftsuitname -exportlog exportlogpath -results autoresults -protocolinput "runfailedtests=true"

    In the preceding example, runfailedtests=true specifies whether the failed test from a previous playback must be rerun in Accelerated Functional Test suite.

  • To automatically resolve the browser and driver incompatibility, while you play back the Web UI tests

    -protocolinput "webui.browser.driver.autoupdate=true"

  • To apply guided-healing and self-healing features while you run Web UI tests

    cmdline -workspace workspacename -project projectname -suite test1 -exportlog exportlogpath -results autoresults -protocolinput "autoheal=true"
-publish You can use -publish parameter to publish test results to Rational® Test Automation Server.

You can use the following options along with the -publish parameter:

  • no

    You can use the no option if you do not want to publish test results after the run. This option is useful if the product preferences are set to publish the results, but you do not want to publish them.

  • You can use any of the following options to specify the project name:

    • serverURL #project.name=projectName&teamspace.name=name_of_the _teamspace

    • serverURL #project.name=projectName&teamspace.alias=name_of_the _teamspace_alias

    You must consider the following points while providing the project name:

    • If the project name is not specified, then the value of the -project parameter is used.

    • If you have a project with the same name in different team spaces, then you can append either the &teamspace.name=name_of_the _teamspace or &teamspace.alias==name_of_the_teamspace_alias options along with the -publish parameter.

      For example: -publish “https://localhost:5443/#project.name=test&teamspace.name=ts1”

      Where:

      • https://localhost:5443 is the URL of the server.

      • test is the name of the project.

      • ts1 is the name of the team space.

Note: If the name of the project or team space contains a special character, then you must replace it with %<Hex_value_of_special_character>.

For example, if the name of the team space is Initial Team Space, then you must provide it as Intial%20Team%20Space.

Where, %20 is the hexadecimal value of the space character.

Remember: If you provide the server and the project details under Window > Preferences > Test > Rational Test Automation Server in the product and if you use serverURL#project.name=projectName along with the -publish parameter, the server details in the command-line interface take precedence over the product preferences.
Important: You must provide the offline user token for the server by using the RTCP_OFFLINE_TOKEN environment variable before you use the -publish parameter in the command-line interface.
-publish_for You can use this option to publish the test results based on the completion status of the tests:
  • ALL - This is the default option. You can use this option to publish test results for any text execution verdict.
  • PASS - You can use this option to publish test results for the tests that have passed.
  • FAIL - You can use this option to publish test results for the tests that have failed.
  • ERROR - You can use this option to publish test results for the tests that included errors.
  • INCONCLUSIVE - You can use this option to publish test results for the tests that were inconclusive.
You can add multiple parameters separated by a comma.
-publishreports You can use this option to publish test results in Rational® Test Automation Server. You must use the -publishreports parameter along with the -publish parameter.
You can use the following values:
  • FT - This is an identifier for Functional Test Report. You can use this value to publish the unified report if it is available for the selected test. See Unified reports.
  • STATS - This is an identifier for Statistics Report. You can use this value to publish the web analytics report if it is available for the selected test. See UI Test Statistical report.
  • TESTLOG - This is an identifier for Test Log. You can use this value to publish the test log if it is available for the selected test. See Logs overview.
For example, -publishreports "STATS, TESTLOG"

The values specified here override the values selected in Window > Preferences > Test > Rational Test Automation Server > Results of Rational® Functional Tester.

You must prefix with ! to publish all the reports except the specified one.

For example, -publishreports "! TESTLOG"

All the reports except the TESTLOG report is published to Rational® Test Automation Server after executing the command.
-quiet Turns off any message output from the launcher and returns to the command shell when the run or the attempt is complete.
-results You can use this option to specify the name of the results file. The default result file name is the test name with a time stamp appended. You must specify a folder name that is relative to the project to store the test results.

For example, -results folder/resultname
-stdout You can use this option to display the information about the test on the command line.

After you run a test from the command line, the following outputs are displayed to give you the overall information of the test:

  • --VERDICT: The verdict of the test.
  • --REMOTE_RESULT: The URL of the result published to Rational® Test Automation Server.
  • --REMOTE_RESULT_UI: The URL of the result published to Rational® Test Automation Server and can be opened in a browser to analyze the result.
  • --LOCAL_RESULT: The path of the result saved locally.

For example, -workspace workspace_full_path -project proj_rel_path -publishpublish_url -stdout
-swapdatasets Use this option to replace dataset values during a test run. If a test is associated with a dataset, you can replace the dataset at run time while initiating the run from the command line.

You must ensure that both original and new datasets are in the same workspace and have the same column names. You must also include the path to the dataset when you run the -swapdatasets command.

For example, -swapdatsets /project_name/ds_path/ds_filename.csv:/project_name/ds_path/new_ds_filename.csv

You can swap multiple datasets that are saved in a different project by adding multiple paths to the dataset separated by a semicolon.

For example, 
-swapdatsets /project_name1/ds_path/ds_filename.csv:/project_name1/ds_path/new_ds_filename.csv;
/project_name2/ds_path/ds_filename.csv:/project_name2/ds_path/new_ds_filename.csv
-usercomments You can add text within double quotation mark () to display it in the User Comments row of the report.
Note: You can use the file CommandLine.exe to run the command to add comments in a language that might not support Unicode characters on Windows operating system.
-varfile You can use this option to specify the complete path to the XML file that contains the variable name and value pairs.

To run a Web UI test on a different browser than that was used for the recording, specify the predefined variable. For more information, see Defining a variable to run a test with a selected browser.
-vmargs To specify the Java™ maximum heap size for the Java™ process that controls the command line playback, use the -vmargs option with the -Xmx argument.

For example, when you use-vmargs -Xmx4096m, specify a maximum heap size of 4096m. This method is similar to specifying -Xmx4096m in the eclipse.ini file for the workbench when playing back the test from the user interface.

To collect the response time data for the app itself and for the server and network and display them in different bar charts, use -vmargs "-De2e.collect=true". For desktop-based web applications, the response time data is captured and displayed by default.

To execute tests in parallel on all mobile devices, which are in passive mode, connected to the workbench and ready for playback, use -vmargs "-Dall.available.targets.in.parallel=true".

To execute tests in parallel on all supported desktop browsers and connected mobile devices, use -vmargs "-Dall.available.targets.in.parallel=all".

To execute tests in parallel on selected desktop browsers and connected mobile devices, use -vmargs "-Dall.available.targets.in.parallel=browser1,browswer2,browser3". You must separate browser names with a comma. For example, firefox, ff, chrome, ie, ie64, safari, "-Dall.available.targets.in.parallel=browser1,browser2,browser3".