Testing with Jenkins

Introduction

To automate testing with Jenkins, you must configure Jenkins primary server and Jenkins secondary server. This configuration allows a single Jenkins installation on the Jenkins primary server to host multiple Jenkins secondary server for building and running tests. You must install the latest version of the Rational® Performance Tester Jenkins plugin on the Jenkins primary server, and install the products themselves on the Jenkins secondary server, where you create the tests. For detailed information about the Jenkins primary and secondary server relationship, see the Distributed Builds section on the Jenkins site.

Before you begin

Installing the Rational® Performance Tester Jenkins plugin on the Jenkins master computer

1. Download the Rational® Performance Tester Jenkins plugin RPT-Jenkins-8.0 from the IBM WebSphere, Liberty & DevOps Community portal.
2. From the Jenkins dashboard, perform the following tasks:
   1. Click Manage Jenkins > Manage Plugins.
   2. Click Advanced.
   3. From the Upload Plugin section, click Choose File to locate and Open the Rational® Performance Tester Jenkins plugin.
   4. Click Upload.

   The Rational® Performance Tester Jenkins plugin is displayed in the Installed tab.

3. To allow Random TCP Ports for Java™ Network Launch Protocol (JNLP) agents, perform the following steps:

   1. Click Manage Jenkins > Configure Global Security.
   2. In the Agents section, select Random.
   3. Click Save to save and apply the changes.

Job Configuration

1. Create a new Jenkins free-style software project. For more information to create a free-style Jenkins project, see Building a software project in Jenkins.
2. Optionally, set an environment variable in Manage Jenkins > Configure System > Global properties from the Jenkins dashboard.

   Select the Environment variables check box if it is not selected. Add the variable and save the details.

3. From the Jenkins dashboard, perform the following tasks:
   1. Open the Jenkins free-style software project, and then click Configure.
   2. Click Build > Add build step.
   3. Click Run IBM® Rational® Performance Tester test.
4. Provide the details about the test run as shown. The following table explains each field.

   Field	Description
   Name	Required. The name of the test.
   Workspace	Required. The complete path to the Eclipse workspace.
   Project	Required. The path, including the file name of the project relative to the workspace.
   Test Suite Name	Required. The path, including the file name of the test to run related to the project.
   IMShared Location	Optional. The complete path to IBMIMShared location,if it is not the default location.
   Var File	Optional. The complete path to the XML file that contains the variable name and value pairs.
   Config File	Optional. The complete path to a file that contains the parameters for a test or schedule run.
   Results File	Optional. The name of the results file. The default result file is the test or schedule name with a time stamp appended. The results file is stored in the Results directory. If you are running multiple tests, do not provide a name for the results file.
   Overwrite Results File	Optional. Determines whether a result file with the same name is overwritten. The default value is true, which means the result file can be overwritten.
   Quiet	Optional. Turns off any message output from the launcher and returns to the command shell when the run or the attempt is complete.
   Number of Virtual Users	Optional. For a schedule, the default value is the number of users specified in the schedule editor. For a test, the default value is one user. Overrides the default number of users, if required.
   VM Args	Optional. Java™ virtual machine arguments to pass in.
   Dataset Override	Optional. For a test or schedule, the default value is the dataset specified in the test editor or schedule editor. Overrides the default dataset value to run if required. Note: You must use the Dataset Override option to replace the dataset values during a test or schedule run. You must ensure that both original and new datasets are in the same workspace and have the same column names. You must also include the path to the dataset. For example, /project_name/ds_path/ds_filename.csv:/project_name/ds_path/new_ds_filename.csv. You can swap multiple datasets that are saved in a different project by adding multiple paths to the dataset separated by a semicolon (;). For example, /project_name1/ds_path/ds_filename.csv:/project_name1/ds_path/new_ds_filename.csv;/project_name2/ds_path/ds_filename.csv:/project_name2/ds_path/new_ds_filename.csv
   Exported Statistical Report Data File	Optional. The complete path to a directory to store exported statistical report data.
   Exported Statistical Report Format	Optional. You can use this field to specify a format for the report that you want to export. You must use at least one of the following formats for this field: simple.csv full.csv simple.json full.json csv json For example, Exported Statistical Report Data File <local_dir_path> Exported Statistical Report Format json. You can add multiple formats for the report separated by a comma. If you want to export both the simple and full reports in a json or csv format, you can specify json or csv as the format in the field. The reports are saved to the location specified in the Exported Statistical Report Data File field. You must use the Exported Statistical Report Format field with the Exported Statistical Report Data File field.
   Custom Report Format Files	Optional. A comma-separated list of absolute paths to custom report format files (.view files) to use when exporting statistical report data with the Export Statistical Report Data File option.
   Exported Statistical Report in html	Optional. The complete path to a directory to export web analytic results. Analyze the results on a web browser without using the test workbench. If you run multiple tests, do not provide a value in this field. The web analytic results will be exported to Jenkins workspace.
   Resource Monitoring Labels Override	Optional. For a schedule (Rate schedule or VU schedule), use Resource Monitoring Labels Override to perform any of the following actions: To enable the Resource Monitoring from Service option for a performance schedule if the Resource Monitoring from Service option is not enabled from the schedule editor in Rational® Performance Tester. To ignore Resource Monitoring sources that were set in the performance schedule and to change for a label matching mode. To replace an existing set of Resource Monitoring labels that were set in the performance schedule and run the schedule with a new set of Resource Monitoring labels. Note: You can add multiple Resource Monitoring labels separated by a comma. Important: You must add the Resource Monitoring labels to the Resource Monitoring sources on the Resource Monitoring page in your IBM® Rational® Test Automation Server project.
   User Comments	Optional. Add text within the double quotation mark to display it in the User Comments row of the report.

   If you do not supply a value for Exported Statistical Report Data File, these logs will be saved in Jenkins workspace/temp directory.

5. Click Save.
6. Optionally, to run multiple tests under the same job, click Add build step again, and provide details for the next test.

Environment variable configuration

You can then run the Jenkins build by using environment variables in the Run IBM® Rational® Performance Tester test step. The environment variable that you added to your test is substituted with the actual value associated with the variable that you set in Manage Jenkins > Configure System > Global properties for the job. The Rational® Performance Tester Jenkins plugin uses the actual value while running the job.

For example,

If you set the environment variable named workspace with the value C:\Users\IBM\workspace1 in Manage Jenkins > Configure System > Global properties, then you can use $workspace or ${workspace} as input to the corresponding text field of your choice for the Run IBM® Rational® Performance Tester test step in the Jenkins build step. When you run the build, $workspace or ${workspace} is substituted with its corresponding value C:\Users\IBM\workspace1.

Jenkins primary server and secondary server configuration

The Jenkins primary and secondary server configurations are supported by Rational® Performance Tester Jenkins plugin. Refer to Distributed builds for more information.

While creating the job configuration, in addition to the preceding steps, you must select the Restrict where this project can be run check box and provide the name of the slave node in the Label Expression field. This is the location where the products are installed and where tests can be run.

Running tests

After you save the project, you must open the project, and then click Build Now. This action starts the test run on the slave computer. You must specify the relative path from the project to the test including the file name of the test. To run multiple tests from different projects, you must add new build step for each project.

Building result and logs

1. After the build completes, click the build number and open the console for the project. Locate the Test Result to check the test execution status.
   Note: If you add multiple build steps to run multiple tests, multiple Test Result instances are displayed.

   In Rational® Performance Tester, if the IBM® Rational® Test Automation Server URL is configured in Window > Preferences > Test > and Publish result after execution is set as Always in Window > Preferences > Test > > Results, then the Reports information section on the Console page displays the names of the report along with its corresponding URLs. The report URLs are the Rational® Test Automation Server URLs where the reports are stored. You can access the report URLs to view the test execution information at any point of time.

2. For product logs, log in to the slave computer and search in the Jenkins slave workspace/temp directory.