Running a test or schedule from a command line

You can run a test or schedule without opening the product by using the command-line interface.

Before you begin

You must have set the Web Reports preferences in the desktop client to view the status of the test run from the command line. See Accessing reports remotely.

Procedure

  1. Navigate to the directory that contains the cmdline.bat and cmdline.sh files.
    On Windows™ operating systems, this directory is located at productInstallationDirectory\cmdline. For example, C:\Program Files\IBM\SDP\cmdline.
  2. Issue the following command:

    cmdline -workspace workspace_full_path -project proj_rel_path -eclipsehome eclipse_full_path -plugins plugin_full_path -schedule sched_rel_path -suite suite_rel_path -importzip file_full_path.zip -varfile variable_file_full_path -servicename service -serviceargs service_args -configfile file_full_path -results result_file -overwrite {"true" | "false"} -quiet -users nn -vmargs JVM_args -rate RateRunnerGroupName=iterationNumber/duration, iterationNumber/duration-duration Stage1=durationOfStage; Stage2=durationOfStage -publish serverURL#project.name=projectName -publish_for {ALL,PASS,FAIL,ERROR, INCONCLUSIVE} -exportlog log_full_path -exportstats local_dir_path -exportstatsformat name of the file format -timerange "all, 5 Users, 10 Users" -exportstatshtml local_dir_path -compare "result_path1, result_path2" -exportstatreportlist stats_list -execsummary local_dir_path -execsummaryreport reportID -usercomments "any user comment" -publishreports "FUNCTIONAL, MOBILE_WEBUI, STATS, TESTLOG" -stdout -swapdatsets existing_dataset_file_path:new_dataset_file-path -history jaeger,testlog,null -overridermlabels "label name 1,label name 2"

    Notes:
    • The workspace is locked after you issue the command. To check the progress of the test or schedule during the run, invoke another workspace and open the project through that workspace.
    • On Linux operating system, the command must start with cmdline.sh.
    • The command line does not provide a way to specify the secure storage password for resource monitoring. You must provide the password in the workbench and ensure that it is stored and persisted in the schedule before you execute the schedule from the command line.
    If a value contains spaces, enclose the value in quotation marks. To see the online help for this command while you are in the directory that contains the .bat file, type cmdline -help.
    The following table explains each options:
    -workspace Required. The complete path to the Eclipse workspace.
    -project Required. The path, including the file name of the project relative to the workspace.
    -eclipsehome Optional.

    The complete path to the directory that contains eclipse.exe.

    For example, C:\Program Files\IBM\SDP
    -plugins Optional. The complete path to the folder that contains the plugins. Typically, on Windows operating systems, this folder is located at C:\Program Files\IBM\IBMIMShared\plugins.

    Required. This option is required only if the folder is at a different location.
    -schedule Optional. However, in a command, it is mandatory to use one of the following options:
    • -suite
    • -schedule
    • -servicename

    You must not use the -schedule option along with the other options. The path includes the file name of the schedule to run relative to the project.

    Starting from V9.2.1.1, you can execute multiple schedules simultaneously.

    For example, -schedule sch1:sch2:sch3
    -suite Optional. However, in a command, it is mandatory to use one of the following options:
    • -suite
    • -schedule
    • -servicename

    You must not use the -suite option along with the other options. The path includes the file name of the suite to run relative to the project.

    Starting from V9.2.1.1, you can execute multiple tests simultaneously.

    For example, -suite test1:test2:test3.

    -importzip Optional. To import the project as test assets with dependencies into your workspace, use the -importzip option. This command is available from V9.2.1.1 and later.

    You can execute test assets from the imported zip file, but you must specify the -importzip {complete path where the zip file is stored on your computer} option along with the -schedule or -suite options.

    For example, C:\User\Desktop\test1.zip

    -varfile Optional. You can use this option to specify the complete path to the XML file that contains the variable name and value pairs.
    -servicename Optional. However, in a command, it is mandatory to use one of the following options:
    • -suite
    • -schedule
    • -servicename

    You must not use the -servicename option along with the other options. The path includes the file name of the service to run relative to the project. Instead of running a performance test, the specified service is run when it is available.
    -serviceargs Optional. You can use this option to specify a series of arguments to pass to the service specified.

    For example, -serviceargs "-myserviceparm1 myserviceparm1value"

    The values are in quotation marks as they contain spaces.
    -configfile Optional. You can use this option to specify the complete path to a file that contains the parameters for a test or schedule run. Each parameter must be on a single line. To create a configuration file, you must use an editor that does not wrap lines. Any parameters, whether required or optional, can be set in the configuration file. The command line parameters override the values in this file.
    Notes:

    • If you are creating a config file manually, the file must be in the UTF-8 format. You must not use quotation marks in this file even for values that contain spaces.

    • You can create command line config file from the product, which you can use while running tests or schedules from the command-line interface or Maven. For more information about how to create a command line config file from the product, see related links.

    -results Optional. You can use this option to specify the name of the results file. The default result file name is the test or schedule name with a time stamp appended. You must specify a folder name that is relative to the project to store the test results.

    For example, -results folder/resultname
    -overwrite

    Optional. Determines whether a result file with the same name is overwritten. The default value, false, indicates that the new result file is created. If the value is true, the file is overwritten and retains the same file name. You must use double quotes for values true or false.
    -quiet Optional. Turns off any message output from the launcher and returns to the command shell when the run or the attempt is complete.
    -users Optional. Overrides the default number of virtual users in the run. For a schedule, the default is the number of users specified in the schedule editor. For a test, the default is one user. This option creates a new copy of the schedule that contains the specified number of users.
    -vmargs Optional. To specify the Java™ maximum heap size for the Java™ process that controls the command line playback, use the -vmargs option with the -Xmx argument.

    For example, when you use-vmargs -Xmx4096m, specify a maximum heap size of 4096m. This method is similar to specifying -Xmx4096m in the eclipse.ini file for the workbench when playing back the test from the user interface.
    -rate Optional. You can use this argument to specify a rate that you want to achieve for a workload in the Rate Runner group. For example, -rate "Rate Runner Group 1=1/s, 3/m; Rate Runner Group 2=5/s, 10/s".
    Here, Rate Runner Group 1 is the name of the Rate Runner group that has two stages. The desired rate for the first state is one iteration per second and the rate for the second stage is three iterations per minute.
    Note: The Rate Runner group name must match with the name in the Rate Schedule.
    -duration Optional. You can use this argument to specify the duration of the stages in the Rate Schedule.

    For example, -duration Stage1=10s; Stage2=3m

    Note: The stage number specified must exist in the Rate Schedule.
    -publish Optional. You can use this parameter to publish test results to Rational® Test Automation Server. You can use the following options along with the -publish parameter:

    • serverURL#project.name=projectName:

      You can use this option to specify the project name. If the project name is not specified, the name of the current project is used.
      Remember: If you provide the server and the project details under Window > Preferences > Test > Rational Test Automation Server in the product, and if you use serverURL#project.name=projectName along with the -publish parameter, the server details in the command-line interface takes precedence over the product preferences.

    • no:

      You can use this option if you do not want to publish test results after the run. This option is useful if the workbench preferences are set to publish the results, but you do not want to publish them.

    Important: You must provide the offline user token for the server by using the RTCP_OFFLINE_TOKEN environment variable before you use the -publish parameter in the command-line interface.
    -publish_for Optional. You can use this option to publish the test results based on the completion status of the tests:
    • ALL - This is the default option. You can use this option to publish test results for any text execution verdict.
    • PASS - You can use this option to publish test results for the tests that have passed.
    • FAIL - You can use this option to publish test results for the tests that have failed.
    • ERROR - You can use this option to publish test results for the tests that included errors.
    • INCONCLUSIVE - You can use this option to publish test results for the tests that were inconclusive.
    You can add multiple parameters separated by a comma.
    -exportlog Optional. You can use this parameter to specify the file directory path to store the exported HTTP test log.

    Starting from V10.0.1, by using the -exportlog parameter, you can provide multiple parameter entries when running multiple tests. You must use a colon to separate the parameter entries.

    For example: -exportlog c:/logexport.txt:c:/secondlogexport.txt

    If there are multiple -suite parameter entries with a single -exportlog parameter entry, then the -exportlog parameter generates the appropriate number of test logs by appending 0, 1, 2, and so on to the -exportlog parameter entry name.

    For example: -suite "sampletest1:sampletest2:sampletest3" -exportlog c:/logexport.txt The command generates the following test logs:

    • logexport_0.txt
    • logexport_1.txt
    • logexport.txt

    The last test log generated has the same name as that of the initial -exportlog entry.

    Note: If there are multiple -suite and -exportlog parameter entries, the number of -suite entries must match with the number of -exportlog entries. Otherwise, the following error message is displayed:

    Error, number of -suite and -exportlog entries do not match.
    -exportstats Optional. You can use this option to export reports in comma-separated values (CSV) format, with the file name derived from the report name. This directory can be relative to the project or a directory on your file system. If the -exportstatreportlist option is not specified, the reports specified on the Export Reports page of the Performance Test Report preferences are exported.
    -exportstatsformat Optional. You can use this option to specify a format for the result that you want to export along with the -exportstats option. You must use at least one of the following parameters with the -exportstatsformat option:

    • simple.csv

    • full.csv

    • simple.json

    • full.json

    • csv

    • json

    For example, -exportstats <local_dir_path> -exportstatsformat simple.json

    You can add multiple arguments separated by a comma.

    For example, -exportstats <local_dir_path> -exportstatsformat simple.json, full.csv

    When you want to export both simple and full type of test results in a json or csv format, you can specify json or csv as the arguments in the command. When the test run completes, the test result exports to simple.json and full.json files.

    For example, -exportstats <local_dir_path> -exportstatsformat json

    You can select the Command Line check box from the product preferences (Window > Preferences > Test > Performance Test Reports > Export Reports) when you want to export test results to one of the selected formats after the test run completes.

    Remember: When you run the test from the command line, and if you use the -exportstats parameter, then the command line preferences take precedence over the preferences set in the product. Therefore, by default, the test result exports to a CSV format.

    For example, when you select the Command Line option and Report format to json in the product preferences, and run the test from the command-line interface without using the -exportstats option. The result is exported to a json file after the test run is complete.
    -timerange Optional. You can use this option along with -exportstats, -exportstatshtml, and -execsummary to export test results within one or more time ranges. The value is the time range that you specify in the schedule.

    For example, all, 5 Users,10 Users. You must separate time ranges with a comma and use double quotation marks () when there is space in a time range.
    -exportstatshtml Optional. When you want to export web analytic results, you can use this option. The results are exported in the specified directory. You can then analyze the results on a web browser without using the test workbench.
    -compare You can use this argument along with -exportstatshtml and -execsummary to export the result in compare mode. The value can be paths to the runs and are relative to the workspace. You must separate the paths by a comma.
    -exportstatreportlist Optional. You can use this option to specify a comma-separated list of report IDs along with -exportstats or -exportstatshtml to list the reports that you want to export in place of the default reports, or the reports selected under Preferences. To view this setting, navigate to Window > Preferences > Test > Performance Test Reports > Export Reports.

    To copy the report IDs list into your command line, navigate to Window > Preferences > Test > Performance Test Reports > Export Reports. Under Select reports to export, select the required reports, and click Copy ID to clipboard. You can then paste the clipboard content on to your command line editor
    -execsummary Optional. You can use this option to export all of the reports for the test run in a printable format, also known as an executive summary, to the local computer. You must specify the path to store the executive summary.
    -execsummaryreport Optional. You can use this option to export a specific report as an executive summary for the test run to the local computer. You must specify the ID of the report to export.

    For example, to export an HTTP performance report, specify http. You must use this option along with -execsummary.

    To copy the report IDs list into your command line, navigate to Window > Preferences > Test > Performance Test Reports > Export Reports. Under Select reports to export, select the required reports, and click Copy ID to clipboard. You can then paste the clipboard content on to your command line editor
    -usercomments Optional. You can add text within double quotation mark () to display it in the User Comments row of the report.
    Note: You can use the file CommandLine.exe to run the command to add comments in a language that might not support Unicode characters on Windows operating system.
    -publishreports Optional. You can use this option to publish test results in Rational® Test Automation Server. The parameters that you can use with it are the following:
    • FUNCTIONAL
    • MOBILE_WEBUI
    • STATS
    • TESTLOG
    For example, -publishreports "STATS, TESTLOG"

    You must prefix with ! to publish all the reports except the specified one.

    For example, -publishreports "! TESTLOG"

    All the reports except the TESTLOG report is published to Rational® Test Automation Server after executing the command.
    -stdout Optional. You can use this option to display the information about the test or schedule on the command line.

    After you run a test or schedule from the command line, the following outputs are displayed to give you the overall information of the test or schedule:

    • --VERDICT: The verdict of the test or schedule.
    • --REMOTE_RESULT: The URL of the result published to Rational® Test Automation Server.
    • --REMOTE_RESULT_UI: The URL of the result published to Rational® Test Automation Server and can be opened in a browser to analyze the result.
    • --LOCAL_RESULT: The path of the result saved locally.

    For example, -workspace workspace_full_path -project proj_rel_path -schedule sched_rel_path -publishpublish_url -stdout
    -swapdatasets Optional. Use this option to replace dataset values during a test or schedule. If a test or schedule is associated with a dataset, you can replace the dataset at run time while initiating the run from the command line.

    You must ensure that both original and new datasets are in the same workspace and have the same column names. You must also include the path to the dataset when you run the -swapdatasets command.

    For example, -swapdatsets /project_name/ds_path/ds_filename.csv:/project_name/ds_path/new_ds_filename.csv

    You can swap multiple datasets that are saved in a different project by adding multiple paths to the dataset separated by a semicolon.

    For example, -swapdatsets /project_name1/ds_path/ds_filename.csv:/project_name1/ds_path/new_ds_filename.csv;/project_name2/ds_path/ds_filename.csv:/project_name2/ds_path/new_ds_filename.csv

    -history Use this command when you want to view a record of all events that occurred during a test or schedule run. However, you must use the command suffixed with any of the following options:
    • jaeger: To send test logs to the Jaeger UI during the test or schedule run.
    • testlog: To send test logs as traditional test logs in Rational® Performance Tester during the test or schedule run.
    • null: To send no test logs either to the Jaeger UI or Rational® Performance Tester during the test or schedule run.
    For example: 
    -workspace workspace_full_path -project proj_rel_path 
-suite suite_rel_path -stdout -history comma delimited list of modes
    -workspace C:/Users/IBM/rationalsdp/test_ws -project Project1 
-suite test1.testsuite -stdout -history jaeger
    Note: You can add multiple options separated by a comma to send test logs during the test or schedule run to Rational® Performance Tester and the Jaeger UI.
    For example:
    -workspace C:/Users/IBM/rationalsdp/test_ws -project Project1 
-suite test1.testsuite -stdout -history jaeger, testlog

    For more information about how to view test logs in the Jaeger UI and Rational® Performance Tester, see related links.
    - overridermlabels Optional. By using the -overridermlabels command, you can control the Resource Monitoring sources that are required to collect in a performance schedule during the schedule run.
    You can use this command if you want to perform any of the following actions:

    • To enable the Resource Monitoring from Service option for a performance schedule if the Resource Monitoring from Service option is not enabled from the schedule editor in Rational® Performance Tester.

    • To ignore Resource Monitoring sources that were set in the performance schedule and to change for a label matching mode.

    • To replace an existing set of Resource Monitoring labels that were set in the performance schedule and run the schedule with a new set of Resource Monitoring labels.

      Note: You must add the Resource Monitoring labels to the Resource Monitoring sources on the Resource Monitoring page in your Rational® Test Automation Server project. You can use these labels for adding the Resource Monitoring sources to run the performance schedule through the command line interface.

    The command accepts a comma-separated list of labels.

    For example, if you have added a label in Rational® Test Automation Server for a Resource Monitoring source as rm1, then run the following command to collect data from the source as follows:
    -workspace workspace_full_path -project proj_rel_path 
-suite suite_rel_path -overridermlabels "rm1"
    Note: You can add multiple labels to a performance schedule separated by a comma to collect data from the multiple sources during the schedule run. For example:
    -workspace workspace_full_path -project proj_rel_path
 -suite suite_rel_path -overridermlabels "rm1,rm2,rm3"

    If your label contains a comma (,), then when running the -overridermlabels command, you must replace the single comma with the double comma in the label.

    For example, if you have added a label to a Resource Monitoring source as (rm1,test), then you must run the following command to collect data from source as follows:
    -workspace workspace_full_path -project proj_rel_path -suite suite_rel_path -overridermlabels "rm1,,test"
    To stop the test run, you can open another command prompt window and use one of the following options with the cmdline option:
    Command Description
    -stoprun Optional. Stops the test run after the specified number of seconds. The block is executed, and the test log is transferred before stopping the run. You must use the -workspace command and specify the location of the workspace.
    -abandonrun Optional. Stops the test run immediately. You must use the -workspace command and specify the location of the workspace.
    Note: Messages are displayed to indicate when the test or schedule is launched and when it is completed unless you include the -quiet option.

Examples of the commands for running tests from the command line

You can run tests from the command line either by using a configuration file or by directly specifying the path of the test in the command. Each command-line option must be followed by an appropriate value.

The contents of a sample configuration file, config_file1 are as follows:

workspace=D:\My Workspace
eclipsehome=C:\Program Files\IBM\SDP
plugins=C:\Program Files\IBM\IBMIMShared\plugins
project=myProject
suite=mytestsuite

To run tests from the command line by using the sample config file config_file1 you must use the following command:

cmdline –configfile <config file path>

For example:

cmdline –configfile E:\Workspace1\Project1\Tests\config_file1.txt

To run the tests from the command line without using a configuration file, you must specify the path of the tests along with the command as follows:

cmdline <path of the test>

For example:

cmdline -workspace “D:\My Workspace” -eclipsehome "C:\Program Files\IBM\SDP" -plugins "C:\Program Files\IBM\IBMIMShared\plugins" -project myProject -suite mytestsuite

The -workspace command-line option is followed by a value that contains a space. If the value contains space, then you must enclose the value, D:\My Workspace within quotes. Otherwise, you can provide the value without quotes.

What to do next

After you run the test or schedule, you may want to export the results for further analysis. For more information, see Exporting report counters automatically.