How to identify that the step is RunTestCase in SoapUI? - groovy

I have following Structure:
Each single functionality is broken down in Reusable script and reusing all in the Main Suite.
TestCase 1:
1. Login as Normal Customer (This is calling login test case from Reusable script)
2. Extract Session from STEP 1
3. Add diner card (This is calling add card test case from Reusable script)
4. View Added Card (This is calling view test case from Reusable script)
5. etc..
Now Each test case in Reusable script returns a property that r_result (Passed or Failed)
Now I wanted to check each run test case and see the property r_result is Passed or Failed. If It is failed, I need to check where the First Failed occurs (in RunTestCase) and report that error.
Is It possible to isolate ONLY RunTestCase steps in each Test case and use it in closure to get the results of each RunTestCase results?

Here is the script which can fetch the list of matching test steps across the soapui project.
Please follow the in-line comments.
result variable has all the list of test steps of required type. The you can leverage and do the needful using this data.
Script:
import com.eviware.soapui.impl.wsdl.teststeps.WsdlRunTestCaseTestStep
//To identify lookup test step is not this step
def currentStepMap = [ suite : context.testCase.testSuite.name, case : context.testCase.name, step : context.currentStep.name ]
//Type of step to look for
def stepTypes = [WsdlRunTestCaseTestStep]
//To hold the final result
def result = []
//Find the test step details of matching step
def getMatchingMap = { suite, kase, step ->
def tempMap = [suite : suite.name, case : kase.name, step: step.name]
def isNotMatching = currentStepMap != tempMap ? true : false
if (isNotMatching &&(stepTypes.any{step in it}) ) {
tempMap
} else { [:] }
}
def project = context.testCase.testSuite.project
//Loop thru the project and find the matching maps and list them
project.testSuiteList.each { suite ->
suite.testCaseList.each { kase ->
kase.testStepList.each { step ->
def tempResult = getMatchingMap(suite, kase, step)
if (tempResult) {
result << tempResult
}
}
}
}
if (result) {
log.info "Matching details: ${result} "
} else {
log.info "No matching steps"
}

Related

Get all test cases and test steps in SoapUI-project

I'm looking for a way to get all the test cases with belonging test steps in SoapUI Pro.
I've managed to get all the test steps under one test case (Groovy in Setup Script)
def projectName= testRunner.testCase.testSuite.project.getName()
def testcaseName = testRunner.getTestCase().getName()
File myTestFile = new File("C:/temp/" + projectName + ".txt")
myTestFile .withWriterAppend{ out ->
out.println("TestCase: " + testcaseName)
testRunner.testCase.getTestStepList().each(){
out.println("TestStep: " + it.getName())
}
}
How can I get all test cases and test steps?
This will help you:
for ( testSuite in testRunner.testCase.testSuite.project.getTestSuiteList() ) {
for ( testCase in testRunner.testCase.testSuite.getTestCaseList() ) {
for ( testStep in testCase.getTestStepList() ) {
log.info "${testSuite.getName()} : ${testCase.getName()} : ${testStep.getName()} is a test step of type: ${testStep.getClass().toString().tokenize('.')[-1]}"
}
}
}
I can't comment (<50 reputation) or Edit (edit queue is full) on ou_ryperd's answer, but I put here a little change to anyone else that will use his code.
If the code is executed within a case context, the 2nd for called always the current TestSuite. Call the TestSuite from the 1st for instead, this way:
for ( testSuite in testRunner.testCase.testSuite.project.getTestSuiteList() ) {
for ( testCase in testSuite.getTestCaseList() ) {
for ( testStep in testCase.getTestStepList() ) {
log.info "${testSuite.getName()} : ${testCase.getName()} : ${testStep.getName()} is a test step of type: ${testStep.getClass().toString().tokenize('.')[-1]}"
}
}
}
All credits to ou_ryperd.

Jenkinsfile Pipeline DSL: How to Show Multi-Columns in Jobs dashboard GUI - For all Dynamically created stages - When within PIPELINE section

Jenkins 2.89.4 rolling
I saw almost all stackoverflow posts which show how we can successfully run parallel steps/stages (using list/maps etc) --OR hardcoding them directly --OR even create dynamic stages for Jenkinsfile (as seen in this post: Scripted jenkinsfile parallel stage)
My requirements are:
A pipeline which builds N. no of projects under "BUILD" steps i.e. parallel builds on each of those projects. i.e. it runs Gradle on all N projects. Here I have a Jenkinsfile which was created by a declarative JOB DSL Groovy. Here my Gradle projects are not set as multi-projects so I can't call the top level gradle and say, Gradle please do your parallel magic (within Gradle).
I want to run build of these N projects in their own separate parallel dynamically created stages (GUI Columns) as seen in jenkins job's dashboard.
I want to see the output of (Gradle build/console) each project's build separately i.e. I don't want to mix the console output of each projects build which are running in parallel in just ONE COLUMN (i.e. column named BUILD).
In this URL https://jenkins.io/blog/2017/09/25/declarative-1/ I see, how you can run parallel stages/steps but in doing so, either it's mixing those parallel step's output in just one column (I mean under BUILD column) --OR if you want it under separate stages/columns (i.e. the post says Test on Linux or Windows separately then you are still hard-coding all the stages/steps in Jenkinsfile early on (rather than using just a list or array hash which I'd prefer to update for adding more or less stages / parallel steps as in my case, they all follow the same standard). What I want is to just update in one place How many steps and what are all of the stages in just one place (list/array).
I'm not using Jenkins Blue Ocean for now.
Usually if you have parallel steps within a stage, their console std output for all steps gets mixed into one console output / stage/column when you click to see the console output for that given parallel step/stage; When you hover over the BUILD column (assuming there were parallel steps in BUILD stage) in job's dashboard (std output for all those steps is mixed up and very hard to see individual project step's console output just for a given step/stage).
If we want to create separate stages (dynamically) then Jenkins should be able to show console output of a given step/dynamic stage within the parallel section (i.e. each column should show their own project's build console output).
Using the above URL, I'm able to do the following after trying this script:
// main script block
// could use eg. params.parallel build parameter to choose parallel/serial
def runParallel = true
def buildStages
node('master') {
stage('Initializing Parallel Dynamic Stages') {
// Set up List<Map<String,Closure>> describing the builds
buildStages = prepareBuildStages()
println("Initialised pipeline.")
}
for (builds in buildStages) {
if (runParallel) {
parallel(builds)
} else {
// run serially (nb. Map is unordered! )
for (build in builds.values()) {
build.call()
}
}
}
stage('Done') {
println('The whole SHENZI is complete.')
}
}
// Create List of build stages to suit
def prepareBuildStages() {
def buildList = []
for (i=1; i<4; i++) {
def buildStages = [:]
for (name in [ 'Alpha', 'Tango', 'Chyarli' ] ) {
def n = "${name} ${i}"
buildStages.put(n, prepareOneBuildStage(n))
}
buildList.add(buildStages)
}
return buildList
}
def prepareOneBuildStage(String name) {
def proj_name = name.split(' ')[0]
def proj_parallel_sub_step = name.split(' ')[1]
//Return the whole chunkoni
return {
stage("Build\nProject-${proj_name}\nStep ${proj_parallel_sub_step}") {
println("Building ${proj_name} - ${proj_parallel_sub_step}")
sh(script:'sleep 15', returnStatus:true)
}
}
}
When I'm putting the above Groovy Script (which is creating DYNAMIC Stages) inside Pipeline Script or Pipeline Script from SCM (i.e. the same code available in a .groovy file) -- it runs successfully and creates dynamic stages under BUILD step for each of the 3 projects and runs 3 steps (Nth) for all 3 projects in parallel and then starts the next Nth step for all 3 projects and so on.
If you see below, we also got individual columns in Jenkins job dashboard for them.
Now, When I put the above script in Jenkinsfile (Pipeline DSL) where I have pipeline { .... } section, it's not working and giving me the following error.
Using my JOB DSL, I created a new Jenkins Pipeline job where Pipeline Script from SCM calls a groovy file (which now contains):
//----------------------------------------------------
// Both - Parallel Run and GUI View in JF Jenkins job.
//----------------------------------------------------
def runParallel = true
def buildStages
def wkspace = /var/lib/jenkins/workspaces/ignore_this_variale_or_its_value_for_now
// Create List of build stages to suit
def prepareBuildStages() {
def buildList = []
for (i=1; i<3; i++) {
def buildStages = [:]
for (name in [ 'Alpha', 'Tango', 'Chyarli' ] ) {
def n = "${name} ${i}"
buildStages.put(n, prepareOneBuildStage(n))
}
buildList.add(buildStages)
}
return buildList
}
//---
def prepareOneBuildStage(String name) {
def proj_name = name.split(' ')[0]
def proj_parallel_sub_step = name.split(' ')[1]
// return the whole chunkoni (i.e. for a given stage) - will be named dynamically.
return {
stage("Build\nProject-${proj_name}\nStep ${proj_parallel_sub_step}") {
println("Building ${proj_name} - ${proj_parallel_sub_step}")
sh(script:'sleep 15', returnStatus:true)
}
}
}
// Set up List<Map<String,Closure>> describing the builds
buildStages = prepareBuildStages()
//---------------------
String jenkinsBaselines
// SEE NOW --- we have this section called 'pipeline'
pipeline {
agent {
node {
label 'rhat6'
customWorkspace wkspace
}
}
options {
ansiColor('xterm')
timeout(time: 8, unit: 'HOURS')
skipDefaultCheckout()
timestamps()
}
environment {
someEnvVar = 'aValue'
}
//------------- Stages
stages {
stage('Initializing Parallel Dynamic Stages') {
// Set up List<Map<String,Closure>> describing the builds
println("Initialised pipeline.")
}
for (builds in buildStages) {
if (runParallel) {
parallel(builds)
} else {
// run serially (nb. Map is unordered! )
for (build in builds.values()) {
build.call()
}
}
}
stage('Done') {
println('The whole SHENZI is complete.')
}
}
//---------------------
}
Running the Jenkinsfile Jenkins job now gives me this error:
[BFA] Scanning build for known causes...
[BFA] No failure causes found
[BFA] Done. 1s
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 69: Not a valid stage section definition: "buildStages = prepareBuildStages()". Some extra configuration is required. # line 69, column 5.
stage('Initializing Parallel Dynamic Stages') {
^
WorkflowScript: 69: Unknown stage section "println". Starting with version 0.5, steps in a stage must be in a steps block. # line 69, column 5.
stage('Initializing Parallel Dynamic Stages') {
^
WorkflowScript: 75: Expected a stage # line 75, column 5.
for (builds in buildStages) {
^
WorkflowScript: 86: Unknown stage section "println". Starting with version 0.5, steps in a stage must be in a steps block. # line 86, column 5.
stage('Done') {
^
WorkflowScript: 69: No "steps" or "parallel" to execute within stage "Initializing Parallel Dynamic Stages" # line 69, column 5.
stage('Initializing Parallel Dynamic Stages') {
^
WorkflowScript: 86: No "steps" or "parallel" to execute within stage "Done" # line 86, column 5.
stage('Done') {
^
6 errors
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:133)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:557)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:518)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:290)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
How can I get this working in Jenkinsfile pipeline section and still able to get individual columns per dynamically created stage for a given project N and Step M?
Tried the following way, still errors the say way.
//------------- Stages
stages {
stage('Initializing Parallel Dynamic Stages') {
// Set up List<Map<String,Closure>> describing the builds
buildStages = prepareBuildStages()
println("Initialised pipeline.")
// tried this way too. within a stage
buildStages.each { bld -->
parallel(bld)
}
}
stage('Done') {
println('The whole SHENZI is complete.')
}
}
//---------------------
So I did some poking into this and got it working.
The new code within Jenkinsfile (for STAGES) is now:
//------------- Stages
stages {
stage('Start Pipeline') {
steps {
script {
sh "echo HELLO moto razr!"
}
}
}
stage('Initializing Parallel Dynamic Stages'){
steps {
script {
// Run all Nth step for all Projects in Parallel.
buildStages.each { bs -> parallel(bs) }
// OR uncomment the following code (if conditional on boolean variable).
/*
for (builds in buildStages) {
if (runParallel) {
parallel(builds)
} else {
// run serially (nb. Map is unordered! )
for (build in builds.values()) {
build.call()
}
}
}
*/
}
}
}
stage('Done') {
println('The whole SHENZI is complete.')
}
}
//---------------------
That's all it took to work.
For clear messages / stage names, I tweaked the function as well and We won't set this variable buildStages within pipeline { ... }
//---
def prepareOneBuildStage(String name) {
def proj_name = name.split(' ')[0]
def proj_parallel_sub_step = name.split(' ')[1]
// return the whole chunkoni (i.e. for a given stage) - will be named dynamically.
return {
stage("BUILD Project-${proj_name} Parallel_Step_${proj_parallel_sub_step}") {
println("Parallel_Step # ${proj_parallel_sub_step} of Project => ${proj_name}")
sh(script:"echo \"Parallel_Step # ${proj_parallel_sub_step} of Project => ${proj_name}\" && sleep 20", returnStatus:true)
// -- OR -- you can call Gradle task i.e. rpm / any other / any other command here.
}
}
}
// Set up List<Map<String,Closure>> describing the builds. section now.
buildStages = prepareBuildStages()
//---------------------
This implementation is now creating N no. of parallel stages i.e. a separate column per project for a given Nth step (when looking at Jenkinsfile job's dashboard) for P no. of projects.
It will run All P projects for a given Nth step in parallel.
It will wait for Nth step for all Projects to complete first and then jump to the next Nth step.
What this means is, if Project ALPHA Step #1 is complete, it'll still wait for all Step #1 of other 2 projects and then launch Step #2 of all projects in parallel.
Challenge: How can we make ALPHA Project's Step #2 to start as soon as ALPHA project's Step #1 is complete i.e. it won't wait for Step 1 of other 2 projects to complete and could possibly run Step #2 of ALPHA project 1 in parallel with Step N(=1) or N+1 of other projects.
This assumes all projects are independent of each other and projects don't share contents generated by a given project/their stage/steps in any other project/stage/step.
Depending upon your own requirements, you may want to wait (i.e. don't run Step 2 of all projects until Step 1 of all projects are fully complete) --OR-- you may want to run Step 2 of ALPHA project with let's say - Step 2 of TANGO project while project CHYARLI's step 1 is still in progress.
As the main scope of this post was to get separate dynamically created columns/stages per project (running in parallel within pipeline { ... } section), I think, I got what I was looking for.
NOTE: Go easy with parallel if you want to run concurrent builds of a pipeline. For more info about issues related to running parallel build actions concurrently, see here: Jenkins - java.lang.IllegalArgumentException: Last unit does not have enough valid bits & Gradle error: Task 'null' not found in root project
You can also run the stages parallel as follow
pipeline {
agent { node { label 'master' } }
stages {
stage('Add regression tests') {
steps {
script {
def testList = ["a", "b", "c", "d"]
def branches = [:]
for (int i = 0; i < 4 ; i++) {
int index=i, branch = i+1
stage ("branch_${branch}"){
branches["branch_${branch}"] = {
node ('master'){
sh "echo 'node: ${NODE_NAME}, index: ${index}, i: ${i}, testListVal: " + testList[index] + "'"
}
}
}
}
parallel branches
}
}
}
}
}

runner.results do not provide results in sequence of test run

I want to get status of testcases run in a testsuite using teardown scripts.
I am able to get the status but not in the sequence of the run of testcases.
I am getting results in random order. The names are in random order everytime.
for ( testCaseResult in runner.results )
{
log.info "$testCaseName"
}
Whenever I do that, I do get them in the correct order...
You may want to try something like this then:
// In this manner, I would expect you to get the testcases in correct order
for (def tc in runner.testSuite.testCaseList) {
// Now loop through the results in order to get the result for the current tc
for (def tcRunner in runner.results) {
def matchFound = false
if (tcRunner.testCase.name.equals(tc.name)) {
matchFound = true
// do your thing
}
if (!matchFound) {
// Do whatever you want to do, if the specific testresult was not found.
}
}
}

How to print all assertion failure messages into HTML report of SoapUI

I am running test case and asserting data using groovy. I want to print each and every failed message to html junit generate report.
Example Code
import groovy.json.JsonSlurper
def ResponseMessage = messageExchange.response.responseContent
def jsonString = new JsonSlurper().parseText(ResponseMessage)
assert !(jsonString.isEmpty())
assert jsonString.code == 200
assert jsonString.status == "success"
def accountInfo = jsonString.data
assert !(accountInfo.isEmpty())
def inc=0
//CHECKING LANGUAGES IN RESPONSE
if(accountInfo.languages.id!=null)
{
log.info("Language added successfully")
}
else
{
log.info("Language NOT added.") //want to display this in html report
inc++
}
if(accountInfo.educations!=null)
{
log.info("Educations added successfully")
}
else
{
log.info("Educations NOT added.") //want to display this in html report
inc++
}
assert inc<=0,"API unable to return all parameters, Please check logs"
Scenario
What I am doing here is , IF test condition does not match and go to ELSE, I do increment of variable inc by 1. So at the end if fail my test if inc>0.
Report
In junit style html generated report, if test failed it display only one message called API unable to return all parameters, Please check logs
But what I want is to display each IF condition message into HTML report if for any condition goes into ELSE section.
Couple of pointers:
assert stops at first failure and only this failure message is part of junit report.
having said that, user will not know if there are any further verification failures for the current response.
the messages that are part of if..else are not part of junit report
in order to achieve that, need to collect all those messages and finally show the collected error messages.
below solution uses a variable, messages and appends each failure which allows to show them at the end. this way all failures can be shown in the report if that is desirable which OP requested for.
user can also show the messages in the report using below statement apart from assert statement
if (messages) throw new Error(messages.toString())
Script Assertion
import groovy.json.JsonSlurper
//check if the response is empty
assert context.response, 'Response is empty or null'
def jsonString = new JsonSlurper().parseText(context.response)
def messages = new StringBuffer()
jsonString.code == 200 ?: messages.append("Code does not match: actual[${jsonString.code}], expected[200]\n")
jsonString.status == "success" ?: messages.append("Status does not match: actual[${jsonString.status}], expected[success]\n")
def accountInfo = jsonString.data
accountInfo ?: messages.append('AccountInfo is empty or null\n')
def inc=0
//CHECKING LANGUAGES IN RESPONSE
if(accountInfo.languages.id) {
log.info('Language added successfully')
} else {
log.error('Language NOT added.') //want to display this in html report
messages.append('Language not added.\n')
inc++
}
if(accountInfo.educations) {
log.info('Educations added successfully')
} else {
log.error('Educations NOT added.') //want to display this in html report
messages.append('Educations NOT added.\n')
inc++
}
//inc<=0 ?: messages.append('API unable to return all parameters, Please check logs.')
//if(messages.toString()) throw new Error(messages.toString())
assert inc<=0, messages.append('API unable to return all parameters, Please check logs.').toString()

How can I retrieve the build parameters from a queued job?

I would like to write a system groovy script which inspects the queued jobs in Jenkins, and extracts the build parameters (and build cause as a bonus) supplied as the job was scheduled. Ideas?
Specifically:
def q = Jenkins.instance.queue
q.items.each { println it.task.name }
retrieves the queued items. I can't for the life of me figure out where the build parameters live.
The closest I am getting is this:
def q = Jenkins.instance.queue
q.items.each {
println("${it.task.name}:")
it.task.properties.each { key, val ->
println(" ${key}=${val}")
}
}
This gets me this:
4.1.next-build-launcher:
com.sonyericsson.jenkins.plugins.bfa.model.ScannerJobProperty$ScannerJobPropertyDescriptor#b299407=com.sonyericsson.jenkins.plugins.bfa.model.ScannerJobProperty#5e04bfd7
com.chikli.hudson.plugin.naginator.NaginatorOptOutProperty$DescriptorImpl#40d04eaa=com.chikli.hudson.plugin.naginator.NaginatorOptOutProperty#16b308db
hudson.model.ParametersDefinitionProperty$DescriptorImpl#b744c43=hudson.mod el.ParametersDefinitionProperty#440a6d81
...
The params property of the queue element itself contains a string with the parameters in a property file format -- key=value with multiple parameters separated by newlines.
def q = Jenkins.instance.queue
q.items.each {
println("${it.task.name}:")
println("Parameters: ${it.params}")
}
yields:
dbacher params:
Parameters:
MyParameter=Hello world
BoolParameter=true
I'm no Groovy expert, but when exploring the Jenkins scripting interface, I've found the following functions to be very helpful:
def showProps(inst, prefix="Properties:") {
println prefix
for (prop in inst.properties) {
def pc = ""
if (prop.value != null) {
pc = prop.value.class
}
println(" $prop.key : $prop.value ($pc)")
}
}
def showMethods(inst, prefix="Methods:") {
println prefix
inst.metaClass.methods.name.unique().each {
println " $it"
}
}
The showProps function reveals that the queue element has another property named causes that you'll need to do some more decoding on:
causes : [hudson.model.Cause$UserIdCause#56af8f1c] (class java.util.Collections$UnmodifiableRandomAccessList)

Resources