I am running on an CI machine my soapUI automation solution, invoked by testRunner.sh.
is is invoked as following:
/Projects/SoapUI-5.2.1/bin/testrunner.sh ~/sautomation_work/Automation_Project.xml
I would like to stop the whole process in case a certain API http status code is not 200.
Any ideas ?
currently, the only way I can do this is by invoking the last test suite "FinalReport" and disable rest of the test scripts currently available in the running test suite.
The code is as following:
public testSuiteStop() {
def properties = new com.eviware.soapui.support.types.StringToObjectMap();
def reportTestCase = testRunner.testCase.testSuite.project.getTestSuiteByName("Report").getTestCaseByName("FinalReport");
reportTestCase.run(properties, true);
def testSuite = context.testCase.testSuite;
def totalTestCases = testSuite.getTestCases().size();
for(testCaseItem in (0..totalTestCases-1)) {
testSuite.getTestCaseAt(testCaseItem).setDisabled(true)
}
}
I have found a way to exit the current soapUI execution in case one of the critical test has already failed.
The idea is to disable the remaining test suites and test cases, and execute a "final" test suite that reports the failed test execution.
my code looks like this:
import com.eviware.soapui.impl.wsdl.WsdlProject;
import com.eviware.soapui.impl.wsdl.WsdlTestSuite;
import com.eviware.soapui.impl.wsdl.testcase.WsdlTestCase;
WsdlTestSuite testSuite = context.testCase.testSuite;
WsdlProject project = context.testCase.getProject();
// Disable remaining test cases in the current test suite.
for (WsdlTestCase testCase in testSuite.testCaseList) {
testCase.setDisabled(true);
}
// Disable rest of the test suites
for (WsdlTestSuite testSuiteName in project.testSuiteList) {
testSuiteName.setDisabled(testSuiteName.name != "LastTestSuite");
}
Related
Is it possible/is there a plugin to dynamically create jenkins pipeline stages from pytest?
(run pytest in a pipeline, and then every test that pytest runs will create a new stage in the pipeline, so the blue ocean view of the jenkins job will have created stages for all the tests)
I probably could list all the tests that will run, and then run them manually in parallel, something like this:
stage("Run Tests"){
steps {
script {
def tests = sh(script: "pytest tests/ --collect-only", returnStdout: true).trim().split().toList()
// manipulate tests so that I have a dictionary of filename - test name
run_tests = [:]
tests.each { test -> //test[0] is file name and test[1] is test name
run_tests[test[1]] = {
stage(test[1]) {
sh "pytest ${test[0]} -k ${test[1]}"
}
}
}
parallel run_tests
}
}
}
but it would be easier if there was a native way to do this within pytest or within Jenkins
Thank you!
I am using Jest for testing. How do I access the filename or file path of the current test being run?
I need a conditional statement that runs different lines of code based on whether it is a unit test file or integration test file.
Here is an example of what I am trying to achieve:
beforeAll(() => {
if (integration_test_file){
// run this this code
} else if (unit_test_file){
// run this code instead
}
})
This information is available in Jest environment. This is the case for custom environment:
const Environment = require('jest-environment-node'); // or jest-environment-jsdom
module.exports = class MyEnvironment extends Environment {
constructor(config, context) {
super(config, context);
this.testPath = context.testPath;
}
async setup() {
await super.setup();
this.global.IS_INTEGRATION = /match integration/.test(this.testPath);
}
}
The environment is instantiated for each test suite, testPath contains full path to current test file.
IS_INTEGRATION global variable will be available in setupFilesAfterEnv and tests themselves. In case the code needs to be evaluated for all tests, it may belong to environment setup and teardown methods.
You can access the name of the file path of the test being ran in jest via its global variables.
the file path can be found under the global variable
jasmine.testPath
or
global.jasmine.testPath
this answer only applies if you're using jest with its default test runner "jasmine" or "jasmine2". results will differ based on the test runner you use. see
https://jestjs.io/docs/en/configuration#testrunner-string
__dirname works a treat (ref: https://www.geeksforgeeks.org/how-to-get-the-path-of-current-script-using-node-js/)
In my project, I want to keep all groovy utilities test step under one test case and to call them again and again where ever is needed. Like reading the test data file etc. I would be able to achieve that if the below problem is resolved. I tried a lot of ways but couldn't make it. Any help is welcome!!
For Example
script 1: test1Script
def sayHellow(){
log.info "Hello!!"
}
Script 2 : test2Script
import groovy.lang.Binding
import groovy.util.GroovyScriptEngine
def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
def projectPath = groovyUtils.projectPath
def scriptPath = projectPath + "\\GroovyScripts\\"
//GroovyShell shell = new GroovyShell()
//Util = shell.parse(new File(directoryName + "groovyUtilities.groovy"))
//groovyUtilities gu = new groovyUtilities(Util)
// Create Groovy Script Engine to run the script.
GroovyScriptEngine gse = new GroovyScriptEngine(scriptPath)
// Load the Groovy Script file
externalScript = gse.loadScriptByName("sayHello.groovy")
// Create a runtime instance of script
instance = externalScript.newInstance()
// Sanity check
assert instance!= null
// run the foo method in the external script
instance.sayhellowTest()
When I'm calling that method from another script, I'm getting below exception
groovy.lang.MissingPropertyException: No such log for class test1Script
The error shows that groovy runtime calls your method but it can't find the log property. I assume that this log variable is declared in the test1Script body, e.g. def log = ... In this case the variable becomes local to its declaration scope and it's not visible to the script functions. To make it visible, it can be annotated by #Field or it should be undeclared (doesn’t have type declaration, just log = ...). The latter, however, requires you to pass the variable value via so-called bindings when running the script as you run it. So given my assumptions above, you can annotate your log variable as a field and it should work:
//just for the sake of example it prints to stdout whatever it receives
#groovy.transform.Field
def log = [info: {
println it
}]
def sayHellow() {
log.info "Hello!!"
}
Now calling sayHellow from another script prints "Hello" to stdout:
...
instance.sayHellow()
It is very important to declare, context, testRunner, and Log variables in the called script.
script 1: sayHello.groovy
public class groovyUtilities {
def context
def testRunner
def log
def sayhellowTest() {
log.info "Hi i'm arpan"
}
}
script 2: RunAnotherGroovyScript.groovy
def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
def projectPath = groovyUtils.projectPath
def scriptPath = projectPath + "\\GroovyScripts\\"
// Create Groovy Script Engine to run the script and pass the location of the directory of your script.
GroovyScriptEngine gse = new GroovyScriptEngine(scriptPath)
// Load the Groovy Script file
externalScript = gse.loadScriptByName("sayHello.groovy")
// Create a runtime instance of script
instance = externalScript.newInstance(context: context, log: log, testRunner: testRunner)
// Sanity check if the instance is null or not
assert instance != null
// run the foo method in the external script
instance.sayhellowTest()
Standoutput:
Hi i'm arpan
"I want to keep all groovy utilities test step under one test case and
to call them again and again where ever is needed. Like reading the
test data file etc."
OK, so to me this simply sounds like you have a library of reusable functions and want to be able to call them from any test you might be running.
I suppose you could store them with another test and then call them from the test you're currently running, but SoapUI comes with the neat feature in that you can store your common functions/libraries 'outside' of the SoapUI project.
I have lots of such Groovy libraries and I store mine under the bin/scripts folder of SoapUI. I typically call common functions from a Script assertion test step in the test I'm running. For example, I have a getUserDetails type test step. I can do all the usual assertions against the step like valid response code, SLA etc. I can then use a Script assertion test step. This type of step allows you to run a chunk of Groovy script. This is OK for specific cases, but you wouldn't want to paste in something common as you need to update every Script assertion if something changes. But you can call an 'external' groovy script. Also, the Script Assertion step is just a method that has log, context and message exchange passed into it, so no need to instantiate your own. Just pass them into you external groovy script...
So, as an illustration...
ValidateUser.groovy (stored in bin/scripts/groovyScripts/yourOrg/common)
package groovyScripts.yourOrg.common; // Package aligns with folder it's stored in.
Class ValidateUser {
def log = null;
def context = null;
def messageExchange = null;
// Constructor for the class
ValidateUser(logFromTestStep, contextFromTestStep, messageExchangeFromTestStep) {
// Assign log, context and messageExchange to the groovy class.
this.log = logFromTestStep;
this.context = contextFromTestStep;
this.messageExhange = messageExchangeFromTestStep;
}
def runNameCheck() {
// Performs some validation. You have access to the API response via
// this.messageExchange
log.info("Running the Name Check");
}
}
In the test step of interest, go to the assertions and create a 'Script Assertion' From here you can instantiate your external class and call some method. E.g.
def validateUserObject = new groovyScripts.yourOrg.common.ValidateUser(log, context, messageExchange);
validateUserObject.runNameCheck();
What I like about these external type scripts is that I can use any text editor I like. Also, when I make a change and press Save, SoapUI is monitoring the scripts folder for changes and reloads the script so no need to restart SoapUI.
I am working on a SoapUI project where I need to run my test suite using test runner. I am using external groovy scripting for environment variable. The problem I am facing here is whenever I am running test case from test runner its return Workspace as null, which is used in External groovy. So in external groovy I am getting workspace as null causing error [getProjectByname() cannot be invoked on null]. Below is the
constructor of global script where workspace is used
AvengerAPITestManager(String TestProject, String TestSuite, String TestCase,String TestStep)
{
TestName = "AvengerAPITests";
testProject = SoapUI.getWorkspace().getProjectByName(TestProject);
tSuite = testProject.getTestSuiteByName(TestSuite);
tCase = testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase);
tStepName = TestStep.toString();
tStep=testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase).getTestStepByName (TestStep);
}
Above we have user SoapUI.getWorkspace() which is working fine when trying to run from soapUI but whever I m trying to run from testrunner SoapUI.getWorkspace comes out to be null. I even tried passing workspace like I am passing testproject name still it didnt worked.
I tried something like this also
AvengerAPITestManager(Object workspace,String TestProject, String TestSuite, String TestCase, String TestStep)
{
TestName = "AvengerAPITests";
testProject = workspace.getProjectByName(TestProject);
tSuite = testProject.getTestSuiteByName(TestSuite);
tCase = testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase);
tStepName = TestStep.toString();
tStep = testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase).getTestStepByName(TestStep);
}
In the above code I tries passing Workspace object from the test case as I passed Testcase name and all but still I m getting null for workspace. Please tell me how do I deal with the problem.
Here is usefull working example https://github.com/stokito/soapui-junit
You should place your sample-soapui-project.xml to /src/test/resources folder that will expose it to classpath
If you want to use soap ui in external code, try to directly create new test runner with specific project file:
SoapUITestCaseRunner runner = new SoapUITestCaseRunner();
runner.setProjectFile( "src/dist/sample-soapui-project.xml" );
runner.run();
Or if you want to define test execution more precisely, you can use something like this:
WsdlProject project = new WsdlProject( "src/dist/sample-soapui-project.xml" );
TestSuite testSuite = project.getTestSuiteByName( "Test Suite" );
TestCase testCase = testSuite.getTestCaseByName( "Test Conversions" );
// create empty properties and run synchronously
TestRunner runner = testCase.run( new PropertiesMap(), false );
PS: don't forget to import soap ui classes, that you use in your code and put them to classpath.
PPS: If you need just run test cases outside the soap ui and/or automate this process, why not just use testrunner.sh/.bat for the same thing? (here is description of this way: http://www.soapui.org/Test-Automation/functional-tests.html)
I am not sure if this is going to help anyone out there but here is what I did to fix the problem I was having with workspace as null causing error[getProjectByname() cannot be invoked on null] When i run from cmd
try this:
import com.eviware.soapui.model.project.ProjectFactoryRegistry
import com.eviware.soapui.impl.wsdl.WsdlProjectFactory
import com.eviware.soapui.impl.wsdl.WsdlProject
//get the Util project
def project = null
def workspace = testRunner.testCase.testSuite.project.getWorkspace();
//if running Soapui
if (workspace != null) {
project = workspace.getProjectByName("Your Project")
}
//if running in Jenkins/Hudson
else{
project = new WsdlProject("C:\\...\\....\\....\\-soapui-project.xml");
}
if (project.open && project.name == "Your Project") {
def properties = new com.eviware.soapui.support.types.StringToObjectMap()
def testCase = project.getTestSuiteByName("TestSuite 1").getTestCaseByName("TestCase");
if(testCase == null)
{
throw new RuntimeException("Could not locate testcase 'TestCase'! ");
} else {
// This will run everything in the selected project
runner = testCase.run(new com.eviware.soapui.support.types.StringToObjectMap(), false)
}
}
else {
throw new RuntimeException("Could not find project ' Order Id....' !")
}
The above code will run everything in the selected project.
My app server uses node.js cluster API cluster.fork() to fork multiple child processes. This works fine.
However, when I try to use vows for unit test, the test also got run multiple times because of the call to cluster.fork() inside my app server; as I instantiate the server inside my test code, as follows:
basic-test.js
var vows = require('vows');
var MyAppServer = require('../my_app');
// start the server
var app = MyAppServer.start();
var suite = vows.describe('My Tests');
suite.discuss('connections API')
.addBatch({ ... })
.export(module);
How do I prevent test code to run multiple times in this case ? This test is included in npm test, so I need a way to instantiate my app server inside test itself.
At the top you can do
var cluster = require('cluster');
Then wrap the suite in an if:
if (cluster.isMaster) {
var suite = ...
...
}
For more info on isMaster, check the documentation