Excluding testcases from testsuite properties in Soapui - groovy

I want to disable certain test cases based on the value of some test suite properties (i.e if the property IsActive1 = false, then testcase1 will be disabled).
I used this code at test suite setup script but got an error:
def testSuite = context.testCase.testSuite;
def totalTestCases = testSuite.getTestCases().size();
for(n in (0..totalTestCases-1)) {
if(testSuite.getTestCaseAt(n).name == "${#TestSuite#IsActive}"){
testSuite.getTestCaseAt(n).setDisabled(true)
}
}
How can I do that?

Here is how I would go to achieve this.
Assuming that there is a suite with test cases, namely TestCase1, TestCase2, TestCase3, TestCase4, TestCase5. Using this approach, it would be a simple change of the property value to include or exclude the test case name to be disable. I believe that this would be better to change things from one place(single property) instead of changing IsActive property for each test case. If you have more cases to handle this would be a burden (going to each case and modify true or false for IsActive)
Create a custom property at test suite level, say DISABLE_TESTS with value (comma separated list to disable) as TestCase1, TestCase5
Here is the Setup Script at Suite level which would achieve to selectively execute the test cases.
/**
* Test Suite Setup Script
* Enable or disable selected list of test cases
* Based on the test suite property DISABLE_TESTS value
**/
//Disables a Test case
def disableTestCase(testCaze) {
log.info "Disabling test case : ${testCaze.name}"
testCaze.disabled = true
}
//Enables a Test case
def enableTestCase(testCaze) {
log.info "Enabling test case : ${testCaze.name}"
testCaze.disabled = false
}
//Read the test case names to enable and convert it to list
def disableList = testSuite.getPropertyValue('DISABLE_TESTS')?.split(',')?.collect { it.trim()} ?: []
log.info "List of test for disable: ${disableList}"
//Loop thru the test cases and enable or disable
testSuite.testCaseList.each { kase ->
//If the above list contains test case name disable, enable otherwise
if (disableList && disableList.contains(kase.name)) {
disableTestCase(kase)
} else {
enableTestCase(kase)
}
}
Since some of the cases are disabled, it would be good to enable all the cases as part of Test Suite Teardown Script. It would look mostly same, but there wont be any condition.
/**
* Test Suite Teardown Script
* Which enables all the test cases after selective execution
**/
//Enables a Test case
def enableTestCase(testCaze) {
log.info "Enabling test case : ${testCaze.name}"
testCaze.disabled = false
}
//Loop thru the test cases and enable
testSuite.testCaseList.each { kase ->
enableTestCase(kase)
}

You can do that with a Groovy script. You could place this script in the Setup script tab on testSuite Window, in order that the script is executed before your testSuite, disabling/enabling testCases depending on the properties.
Based on your requeriments this script could be something like:
// get property names getting only
// the one which start with "isactive"
def isActiveList = testSuite.getPropertyNames().findAll {
it.toLowerCase().startsWith('isactive')
}
log.info isActiveList
// get testCaseList in order
def testCaseList = testSuite.getTestCaseList()
// for each "isactive" property
// enable or disable the corresponding
// testCase
isActiveList.each{ name ->
// get the value
def isActive = testSuite.getPropertyValue(name).toBoolean()
// get the order from the property name
def testCaseIndex = name.toLowerCase() - 'isactive'
log.info "testCase at index $testCaseIndex will be disabled ${!isActive}"
// get the testCase and enable/disable depend
// on isactive property
// NOTE: IsActive1 corresponds to array pos 0
// so a -1 is necessary
log.info testCaseList[testCaseIndex.toInteger()-1]?.disabled = !isActive
}
Maybe it's better to rename your properties based on the testCase name's to enable/disable instead of using a number; to avoid undesired behaviors if you change i.e the testCase order or add a new one...
Hope it helps,

Related

How to fail the test case - Cucumber with Groovy

When I check the particular cucumber step in the step definition file, if the condition is false, I wanted to make that particular test case should be stopped and fail and continue with the next test case.
If I have to use the FailureHandling.STOP_ON_FAILURE, how to code the line?
#When("User enters the (.*) in the Login")
def user_enter_userid_in_the_Login(String uid) {
if (uid=='')
/** FAIL the TEST CASE **/
else
WebUI.setText(findTestObject('Object Repository/ORTC01/Page_/input_userid'), uid,
FailureHandling.STOP_ON_FAILURE)
WebUI.delay(5)
}
def user_enter_userid_in_the_Login(String uid) {
assert uid!='' //this will fail test if uid==''
...continue success code
}
also possible to provide the error message
assert uid!='' : "the parameter 'uid' could not be empty"
it approximately corresponds to:
if(uid=='')throw new Exception("the parameter 'uid' could not be empty")

Invoking closure/method from inside setup method in Groovy

For some reason, setup method doesn't recognize the closure that came before it, because when this code gets put into a test suite and ran, it throws
MissingMethodException about the closure I'm trying to call:
int initialNumberOfRows = 0
def findRowCount = {
initialNumberOfRows = 5
}
/**
* Some methods below are samples for using SetUp/TearDown in a test suite.
*/
/**
* Sets up test environment.
*/
#SetUp(skipped = false) // Please change skipped to be false to activate this method.
def setUp() {
// login and select the test company first
WebUI.callTestCase(findTestCase('Test Cases/TestCompanySelectGoesToDashboard'), [('shouldLogout') : false, ('shouldCloseBrowser') : false])
// go to the "Discounts" page
WebUI.click(findTestObject('PageMenuOptions/a_Discounts'))
// determine the row count
WebUI.delay(5)
findRowCount()
}
I run that code and get greeted with the following exception message:
groovy.lang.MissingMethodException: No signature of method: DiscountsSuite.findRowCount() is applicable for argument types: () values: []
The closure, as of right now, is a stub, but will be used to get the number of rows, after which we will delete as cleanup for the test suite (I make the rows via the test suite, I gotta clean em up!)

How to Enable/Disable Particular Assertion in all Test Case using Groovy?

I have suite to run the regression Test Case in Soap UI. It has a Assertion Capture Response which measures the time of each request. This is needed on demand.
If metrics needed then I need to Enable the Capture Response Time assertion and if it is not needed then I don't need the capture response time.
I have written the below code to check that whether it is disabled or not. If it is disabled then OK else i need to disable it.
The following code returns
java.lang.NullPointerException: Cannot get property 'disabled' on null object.
Can any one help on this?
def project = testRunner.getTestCase().getTestSuite().getProject().getWorkspace().getProjectByName("Regression");
//Loop through each Test Suite in the project
for(suite in project.getTestSuiteList())
{
//log.info(suite.name)
//Loop through each Test Case
if(suite.name == "ReusableComponent")
{
for(tcase in suite.getTestCaseList())
{
log.info(tcase.name)
for(tstep in tcase.getTestStepList())
{
stepName = tstep.name
suiteName=suite.name
caseName=tcase.name
def testStep = testRunner.testCase.testSuite.project.testSuites["$suiteName"].testCases["$caseName"].getTestStepByName("$stepName")
log.info(testStep.getAssertionByName("CaptureResponseTime").disabled)
}
}
}
}
Below statement is causing NullPointerException:
log.info(testStep.getAssertionByName("CaptureResponseTime").disabled)
In order to avoid NPE, then change it to:
log.info(testStep.getAssertionByName("CaptureResponseTime").isDisabled)
If you need to disable the assertion, then use below statement:
testStep.getAssertionByName("CaptureResponseTime")?.disabled = true
Another input:
In order to get the project, do not use workspace.
Instead use:
def project = context.testCase.testSuite.project

How to Report Results to Sauce Labs using Geb/Spock?

I want to use the Sauce Labs Java REST API to send Pass/Fail status back to the Sauce Labs dashboard. I am using Geb+Spock, and my Gradle build creates a test results directory where results are output in XML. My problem is that the results XML file doesn't seem to be generated until after the Spock specification's cleanupSpec() exits. This causes my code to report the results of the previous test run, rather than the current one. Clearly not what I want!
Is there some way to get to the results from within cleanupSpec() without relying on the XML? Or a way to get the results to file earlier? Or some alternative that will be much better than either of those?
Some code:
In build.gradle, I specify the testResultsDir. This is where the XML file is written after the Spock specifications exit:
drivers.each { driver ->
task "${driver}Test"(type: Test) {
cleanTest
systemProperty "geb.env", driver
testResultsDir = file("$buildDir/test-results/${driver}")
systemProperty "proj.test.resultsDir", testResultsDir
}
}
Here is the setupSpec() and cleanupSpec() in my LoginSpec class:
class LoginSpec extends GebSpec {
#Shared def SauceREST client = new SauceREST("redactedName", "redactedKey")
#Shared def sauceJobID
#Shared def allSpecsPass = true
def setupSpec() {
sauceJobID = driver.getSessionId().toString()
}
def cleanupSpec() {
def String specResultsDir = System.getProperty("proj.test.resultsDir") ?: "./build/test-results"
def String specResultsFile = this.getClass().getName()
def String specResultsXML = "${specResultsDir}/TEST-${specResultsFile}.xml"
def testsuiteResults = new XmlSlurper().parse( new File( specResultsXML ))
// read error and failure counts from the XML
def errors = testsuiteResults.#errors.text()?.toInteger()
def failures = testsuiteResults.#failures.text()?.toInteger()
if ( (errors + failures) > 0 ) { allSpecsPass = false }
if ( allSpecsPass ) {
client.jobPassed(sauceJobID)
} else {
client.jobFailed(sauceJobID)
}
}
}
The rest of this class contains login specifications that do not interact with SauceLabs. When I read the XML, it turns out that it was written at the end of the previous LoginSpec run. I need a way to get to the values of the current run.
Thanks!
Test reports are generated after a Specification has finished execution and the generation is performed by the build system, so in your case by Gradle. Spock has no knowledge of that so you are unable to get that information from within the test.
You can on the other hand quite easily get that information from Gradle. Test task has two methods that might be of interest to you here: addTestListener() and afterSuite(). It seems that the cleaner solution here would be to use the first method, implement a test listener and put your logic in afterSuite() of the listener (and not the task configuration). You would probably need to put that listener implementation in buildSrc as it looks like you have a dependency on SauceREST and you would need to build and compile your listener class before being able to use it as an argument to addTestListener() in build.gradle of your project.
Following on from erdi's suggestion, I've created a Sauce Gradle helper library, which provides a Test Listener that parses the test XML output and invokes the Sauce REST API to set the pass/fail status.
The library can be included by adding the following to your build.gradle file:
import com.saucelabs.gradle.SauceListener
buildscript {
repositories {
mavenCentral()
maven {
url "https://repository-saucelabs.forge.cloudbees.com/release"
}
}
dependencies {
classpath group: 'com.saucelabs', name: 'saucerest', version: '1.0.2'
classpath group: 'com.saucelabs', name: 'sauce_java_common', version: '1.0.14'
classpath group: 'com.saucelabs.gradle', name: 'sauce-gradle-plugin', version: '0.0.1'
}
}
gradle.addListener(new SauceListener("YOUR_SAUCE_USERNAME", "YOUR_SAUCE_ACCESS_KEY"))
You will also need to output the Selenium session id for each test, so that the SauceListener can associate the Sauce Job with the pass/fail status. To do this, include the following output in the stdout:
SauceOnDemandSessionID=SELENIUM_SESSION_ID

How to retrieve Jenkins build parameters using the Groovy API?

I have a parameterized job that uses the Perforce plugin and would like to retrieve the build parameters/properties as well as the p4.change property that's set by the Perforce plugin.
How do I retrieve these properties with the Jenkins Groovy API?
Update: Jenkins 2.x solution:
With Jenkins 2 pipeline dsl, you can directly access any parameter with the trivial syntax based on the params (Map) built-in:
echo " FOOBAR value: ${params.'FOOBAR'}"
The returned value will be a String or a boolean depending on the Parameter type itself. The syntax is the same for scripted or declarative syntax. More info at: https://jenkins.io/doc/book/pipeline/jenkinsfile/#handling-parameters
If your parameter name is itself in a variable:
def paramName = "FOOBAR"
def paramValue = params.get(paramName) // or: params."${paramName}"
echo """ FOOBAR value: ${paramValue}"
Original Answer for Jenkins 1.x:
For Jenkins 1.x, the syntax is based on the build.buildVariableResolver built-ins:
// ... or if you want the parameter by name ...
def hardcoded_param = "FOOBAR"
def resolver = build.buildVariableResolver
def hardcoded_param_value = resolver.resolve(hardcoded_param)
Please note the official Jenkins Wiki page covers this in more details as well, especially how to iterate upon the build parameters:
https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+System+Groovy+script
The salient part is reproduced below:
// get parameters
def parameters = build?.actions.find{ it instanceof ParametersAction }?.parameters
parameters.each {
println "parameter ${it.name}:"
println it.dump()
}
For resolving a single parameter (I guess what's most commonly needed), this is the simplest I found:
build.buildVariableResolver.resolve("myparameter")
in your Groovy System script build step.
Regarding parameters:
See this answer first. To get a list of all the builds for a project (obtained as per that answer):
project.builds
When you find your particular build, you need to get all actions of type ParametersAction with build.getActions(hudson.model.ParametersAction). You then query the returned object for your specific parameters.
Regarding p4.change: I suspect that it is also stored as an action. In Jenkins Groovy console get all actions for a build that contains p4.change and examine them - it will give you an idea what to look for in your code.
I've just got this working, so specifically, using the Groovy Postbuild plugin, you can do the following:
def paramText
def actionList = manager.build.getActions(hudson.model.ParametersAction)
if (actionList.size() != 0)
{
def pA = actionList.get(0)
paramText = pA.createVariableResolver(manager.build).resolve("MY_PARAM_NAME")
}
In cases when a parameter name cannot be hardcoded I found this would be the simplest and best way to access parameters:
def myParam = env.getProperty(dynamicParamName)
In cases, when a parameter name is known and can be hardcoded the following 3 lines are equivalent:
def myParam = env.getProperty("myParamName")
def myParam = env.myParamName
def myParam = myParamName
To get the parameterized build params from the current build from your GroovyScript (using Pipeline), all you need to do is:
Say you had a variable called VARNAME.
def myVariable = env.VARNAME
Get all of the parameters:
System.getenv().each{
println it
}
Or more sophisticated:
def myvariables = getBinding().getVariables()
for (v in myvariables) {
echo "${v} " + myvariables.get(v)
}
You will need to disable "Use Groovy Sandbox" for both.
If you are trying to get all parameters passed to Jenkins job you can use the global variable params in your groovy pipeline to fetch it.
http://jenkins_host:8080/pipeline-syntax/globals
params
Use something like below.
def dumpParameter()
{
params.each {
println it.key + " = " + it.value
}
}
thanks patrice-n! this code worked to get both queued and running jobs and their parameters:
import hudson.model.Job
import hudson.model.ParametersAction
import hudson.model.Queue
import jenkins.model.Jenkins
println("================================================")
for (Job job : Jenkins.instanceOrNull.getAllItems(Job.class)) {
if (job.isInQueue()) {
println("------------------------------------------------")
println("InQueue " + job.name)
Queue.Item queue = job.getQueueItem()
if (queue != null) {
println(queue.params)
}
}
if (job.isBuilding()) {
println("------------------------------------------------")
println("Building " + job.name)
def build = job.getBuilds().getLastBuild()
def parameters = build?.getAllActions().find{ it instanceof ParametersAction }?.parameters
parameters.each {
def dump = it.dump()
println "parameter ${it.name}: ${dump}"
}
}
}
println("================================================")
The following can be used to retreive an environment parameter:
println System.getenv("MY_PARAM")
The following snippet worked for me to get a parameter value in a parameterized project:
String myParameter = this.getProperty('binding').getVariable('MY_PARAMETER')
The goal was to dynamically lock a resource based on the selected project parameter.
In "[✓] This build requires lockable resources" I have the following "[✓] Groovy Expression":
if (resourceName == 'resource_lock_name') {
Binding binding = this.getProperty('binding')
String profile = binding.getVariable('BUILD_PROFILE')
return profile == '-Poradb' // acquire lock if "oradb" profile is selected
}
return false
In "[✓] This project is parameterized" section I have a "Choice Parameter" named e.g. BUILD_PROFILE
Example of Choices are:
-Poradb
-Ph2db
-DskipTests -T4
The lock on "resource_lock_name" will be acquired only if "-Poradb" is selected when building project with parameters
[-] Use Groovy Sandbox shall be unchecked for this syntax to work

Resources