Best practices / lessons learned

This section is organized in the following main parts, which are discussed one by one in more detail within the remainder of this article:

1. Tips for an optimized SoapUI configuration when using a CM (configuration management) system

2. Tips for creating SoapUI TestCases

3. Tips for setting up the SoapUI Testrunner script

4. Tips for configuring SoapUI within Hudson (CI environment)

In order to give more context to the lessons learned and best practices described here, an overview of the used terms and definitions are described below. Furthermore, an overview of the relations between the artifacts, configuration items and tooling used to enable soapui automatic test execution is given.

Terms and definitions

SoapUI specific (description)                                                                                       

- soapui application: application used to configure and execute webservice tests

- soapui tool configuration: global soapui configuration, like proxy settings, time-outs, etc.

- soapui project configuration: the functional soapui projects in which all the soapui tests are defined

- soapui workspace configuration: set of soapui projects that belong together

- soapui commandline runner: commandline version of soapui which is used for test automation

- soapui workspace configuration: set of soapui projects that belong together

- soapui commandline runner:         commandline version of soapui which is used for test automation

Test automation specific (description)                                                                          

- soapui testrunner script: ant build script used for testscript automation

- ant properties file: contains mandatory properties for enabling automatic test runs

- Hudson job configuration: using jobs in Hudson automatic tests can be specified

SoapUI automatic test execution overview

Tips for an optimized SoapUI configuration when using a CM system like SVN or Subversion

Make a new project filesystem independent by using the relative path features of SoapUI

Using this configuration no absolute paths are specified anymore within the SoapUI project configuration.

For this the following settings are needed:

- Make use of workspace functionality within SoapUI in combination with setting projectroot property to ${workspacedir}

- Make use of Resource Root property, which configures using relative paths within your SoapUI project.

Note that this will be automatically done if you enable the ‘Relative paths’ checkbox when creating a new SoapUI project; but it can be enabled afterwards as you see by specifying the value of ‘${projectDir}’ for the Resource Root property.

Optimal CM configuration

When using a configuration management system it is a best practice to check-in the following files to promote reuse without the need to adapt files after checking out, because someone else has another checkout directory configured.

The following files need to be within the CM system:

SoapUI workspace file (e.g. default-soapui-workspace.xml); within this file all the imported SoapUI projects are specified including their relative paths based on the workspace file.

SoapUI project files (e.g. P4MeterReading.xml); contains all the SoapUI tests for a specific project. So, you can have many of those.

SoapUI settings file in which the tool specific features are configured, varying from proxy settings to socket timeouts (all preferences within the ‘soapui preferences’ overview are available within this file)

Tips for creating SoapUI TestCases

- DO NOT DEVELOP TESTS FOR PRODUCTION SYSTEMS.

- Try to use one soapUI project per webservice endpoint. You can use more, but then you need to make use of additional commandline arguments (this

will be discussed within the remainder of this article)

- Add the properties which are needed to execute your soapui projects including environment specific properties to an environment specific property file

(e.g. dev.properties), which is a list of key/value pairs.

#soapuitestrunner.dir = the directory which contains the SoapUI executable ‘testrunner.bat’.

soapuitestrunner.dir=C:/Program Files/eviware/soapUI-2.5/bin

- Add test specific properties (e.g. inputvalues of your webservice) into a soapui specific properties file.

SoapUI has a bad habbit to adapt your test specific properties files on access; therefore it is a best practice to leave the ‘Save to’ field empty and adapt the properties file outside soapui to prevent updates to those property files.

When there is a need to use different inputvalues within your test for your different environments (dev/tst) you could add a global property (e.g. Example_EnvironmentName).

When using the testrunner.bat to trigger your soapui project you will have to add this property to the testrunnercommandlineadditions’ commandline argument. Further, you will have to refer to this global property within you

‘Load from’ location inside your properties file.

Note that when using this approach you will have to make sure that on your file system you must have the following

directory structure defined in order to make this work:

- dev/

- tst/

Use a global property (e.g. ${Example_Endpoint}) in each SOAP request in your testcases. Overriding this variable allows the same testcases to be executed in different environments (dev/tst/acc).

Make sure the test can be executed using the commandline testrunner.bat script (see below for more details).

Tips for setting up the SoapUI Testrunner script

The SoapUI testrunner script is a small ant-based script that allows you to automatically execute SoapUI testcases

and outputs the results in a JUnit style html report.

Prerequisites

Download apache ant. Extract the zip-file and make sure the location to the \bin directory is in your windows PATH. (If you want to check whether all is OK, run ‘ant -version’ on the command line).

Download ant-contrib package:

http://downloads.sourceforge.net/ant-contrib/ant-contrib-0.6-bin.zip?use_mirror=puzzle.

Extract the zip and put the jar file into your \lib directory of the extracted ant version.

Download and install SoapUI.

http://downloads.sourceforge.net/soapui/soapUI-2.5.1-installer.exe?use_mirror=surfnet

Getting the script

Ok. Now we’re all set and we can start using the ant script. If you read the article on JavaLobby you can see the usage of the testrunner.bat within this file, with the difference that we added some extra arguments so that we can execute all the soapui projects using a simple for loop construction.

Setting up the script

In order to run OK, the script expects a .properties file. An example of the script is shown below (e.g. ‘dev.properties’)

#soapuitestrunner.dir = the directory which contains the SoapUI executable ‘testrunner.bat’.

soapuitestrunner.dir=C:/Program Files/eviware/soapUI-2.5/bin

#some additional items for the commandline, e.g. the required global properties.

testrunnercommandlineadditions=-GExample_Endpoint=http://testmachine.local:5000 –Gexample_EnvironmentName=dev

#soapuiprojects.dir = the base directory for the SoapUI project files (.xml)

soapuiprojects.dir=C:/bea_workspaces/workSpaceStudioNew2/

#soapuiprojects.relativepaths = The relative paths (from soapuiprojects.dir) of all projects

#for which to run the testsuites. Use ‘,’ to separate.

soapuiprojects.relativepaths=ServicesTest/MeterTest/soapui/MeterTest-soapui-project.xml,ServicesTest/DataTest/soapui/DataTest-soapui-project.xml

#output.dir = the directory where the testresults and html results will go.

output.dir=C:/.hudson/jobs/UnitTest-SoapUI/workspace/Test/Tooling/soapui/junit

—————————————————————————————————————————————————–

To select which .properties file is used, an environmentvariable must be set to specify which environment is used (dev/test/acc). This is done by using ’set username dev’ via the commandline.

 

The property file can contain the following properties.

- soapuitestrunner.dir should be the path to the SoapUI testrunner.bat on your system (usually something like ‘C:/Program Files/eviware/soapUI-2.5/bin’)

- testrunnercommandlineadditions can contain additional items that are part of the soapui testrunner commandline. It can be used to set global properties (as in this example: testrunnercommandlineadditions=- GExample_Endpoint=http://testmachine.local:5000 -GOIP_EnvironmentName=dev). For more info on valid commandline options, see: http://www.soapui.org/userguide/commandline/testcaserunner.html

- webservice.host should contain the host for the execution of the webservices. This allows executing the same testprojects on different machines (for example local, DEV, TEST, UAT, etc.). An example could be: testmachine:5300 or localhost:7001. Note that I didn’t use this argument, because I am using different endpoints within a single soapui project, and then this property doesn’t work so well. Therefore, the ‘testrunnercommandlineadditions’ property is introduced to have the possibility to post more than 1 endpoint to a specific soapui project. Note that using this property is recommended, because when you use this property you have the safety that no one can by accident use an endpoint of a production system; this can occur if someone takes a wsdl in which a production endpoint of a webservice is specified. Instead, the configured webservice.host value is used in such case.

- soapuiprojects.dir should contain the base directory of the SoapUI projects for which to run the testcases.

- output.dir is the output dir for the SoapUI results and the html based output.

Ok. We’re set. Let’s run a script!

Running the script

- Open a ‘cmd’-window and navigate to the location where you stored the ant build script.

- Select the properties file, we choose dev.properties, so execute ’set USERNAME=dev’

- Execute command ‘ant’.

Now the script should run, and if all properties in dev.properties are ok, it will run and produce results in the selected ‘output.dir’. The html results can be found in ‘output.dir/reports/html/index.html’.

Note that this tutorial is not an executable version, which means that the ant build script can not be executed as is due to usage of dummy endpoints throughout this tutorial.

Tips for configuring SoapUI within Hudson (CI environment)

The general how to of how soapui can be configured within Hudson can be found within the JavaLobby article (discussed in the beginning of this article). Below I have described some best practices in relation to this.

- One best practice is to create an environment variable HUDSON_HOME on the server running Hudson. This variable name will be used by Hudson to store all configuration and job data. Without this variable Hudson will create a folder under $USERHOME/.hudson (On Windows it will be c:\Documents and Settings[user].hudson]. One advantage of using HUDSON_HOME variable is that it will allow you to start Hudson with any user, what is somehow common when using a remote Windows server to host.

- When you’re using a CM (Configuration Management) environment (e.g. SVN/Subversion) it is recommended to check in all hudson configuration files + Hudson jobs configuration (config.xml files within jobs section). All those files can be found in the HUDSON_HOME folder (or if you didn’t define that your user directory)

- Install Hudson as a Windows Service. See http://java.dzone.com/articles/installing-hudson-windows

- Try using parameterized variables when using property file; in this way you can decide at execution time of a test against which environment you want to run.

- Store your testresults within a subdirectory of your HUDSON_HOME folder, in order to retrieve the results easily

As a result of this the results of the tests are nicely shown after executing your tests.

Met vriendelijke groet,
Michel Christianen