Is it possible/is there a plugin to dynamically create jenkins pipeline stages from pytest?
(run pytest in a pipeline, and then every test that pytest runs will create a new stage in the pipeline, so the blue ocean view of the jenkins job will have created stages for all the tests)
I probably could list all the tests that will run, and then run them manually in parallel, something like this:
stage("Run Tests"){
steps {
script {
def tests = sh(script: "pytest tests/ --collect-only", returnStdout: true).trim().split().toList()
// manipulate tests so that I have a dictionary of filename - test name
run_tests = [:]
tests.each { test -> //test[0] is file name and test[1] is test name
run_tests[test[1]] = {
stage(test[1]) {
sh "pytest ${test[0]} -k ${test[1]}"
}
}
}
parallel run_tests
}
}
}
but it would be easier if there was a native way to do this within pytest or within Jenkins
Thank you!
Related
def Greet(name) {
echo "Hello ${name}"
}
pipeline {
agent any
stages {
stage('Test') {
steps {
sh """
Greet('Build Team')
"""
}
}
}
}
I am trying to call 'Greet' function from the stage 'Test' inside the shell . But getting error as
/var/lib/jenkins/workspace/Check1#tmp/durable-ce807c1d/script.sh: line 2: syntax error near unexpected token `'Bob''
Any one let me know how we can call 'Greet' function from the stage 'Test' inside the shell .
I am using Jest for testing. How do I access the filename or file path of the current test being run?
I need a conditional statement that runs different lines of code based on whether it is a unit test file or integration test file.
Here is an example of what I am trying to achieve:
beforeAll(() => {
if (integration_test_file){
// run this this code
} else if (unit_test_file){
// run this code instead
}
})
This information is available in Jest environment. This is the case for custom environment:
const Environment = require('jest-environment-node'); // or jest-environment-jsdom
module.exports = class MyEnvironment extends Environment {
constructor(config, context) {
super(config, context);
this.testPath = context.testPath;
}
async setup() {
await super.setup();
this.global.IS_INTEGRATION = /match integration/.test(this.testPath);
}
}
The environment is instantiated for each test suite, testPath contains full path to current test file.
IS_INTEGRATION global variable will be available in setupFilesAfterEnv and tests themselves. In case the code needs to be evaluated for all tests, it may belong to environment setup and teardown methods.
You can access the name of the file path of the test being ran in jest via its global variables.
the file path can be found under the global variable
jasmine.testPath
or
global.jasmine.testPath
this answer only applies if you're using jest with its default test runner "jasmine" or "jasmine2". results will differ based on the test runner you use. see
https://jestjs.io/docs/en/configuration#testrunner-string
__dirname works a treat (ref: https://www.geeksforgeeks.org/how-to-get-the-path-of-current-script-using-node-js/)
Not found any reference to this particular question.
I am looking to find a way to achieve something like this in a Jenkins pipeline which runs our acceptance tests using Protractor and Cucumber.js:
steps {
container('selenium') {
script {
try {
{
//run tests
}
}
catch (err) {
if (env.testFailed == 'true') {
println "A test failure exists - build status updated to failure"
currentBuild.result = 'FAILURE'
error "Test(s) have failed"
}
else {
println "No test failures exist - build status updated to success"
currentBuild.result = 'SUCCESS'
}
}
}
}
}
This would fail the build if the env var of testFailed is 'true'. The reason for this is we are encountering bugs with Protractor-Cucumber framework where if a failed test retries and passes the exit code of the stage is still 1.
So in the After hook of each test I am setting the env var using node.js to true if the Scenario status is failed:
if (scenario.result.status === Status.FAILED) {
process.env.testFailed = 'true';
}
if (scenario.result.status === Status.PASSED) {
process.env.testFailed = 'false';
}
The problem I have found is that the Jenkins pipeline fails to read the env var value in the code block of the catch section. It is always null.
Any ideas?
1) change the After hook to write the true/false flag to a file in sync.
2) read the file in catch block
catch(err) {
testFailed = sh(script:'cat result.flag.txt', returnStdout: true).trim()
if(testFailed == 'true') {
...
}
}
Another option if there is total/passed/failed case number in output of npm test
lines = []
try {
lines = sh(script:'npm test', returnStdout: true).readLines();
}
catch(err) {
size = lines.size()
// parse the last 20 lines to extract fail/pass/total number
for(int i=size-20;i<size;i++) {
line[i]
}
}
WHY IT DOESN'T WORK NOW?
I see that you're running your tests in a container. When you set an environment variable, it's reflected on the scope of your container not the Jenkins master server
WHAT YOU COULD TRY TO DO
This actually depends on how you run the tests, but this should be an option
// run tests here
// you should have a variable for your container
def exit_code = sh(script: "sudo docker inspect ${container.id} --format='{{.State.ExitCode}}'", returnStdout: true)
sh "exit ${exit_code}"
This actually also depends how you start the tests inside the container,
So if you update your answer with this information I could help you
I am running on an CI machine my soapUI automation solution, invoked by testRunner.sh.
is is invoked as following:
/Projects/SoapUI-5.2.1/bin/testrunner.sh ~/sautomation_work/Automation_Project.xml
I would like to stop the whole process in case a certain API http status code is not 200.
Any ideas ?
currently, the only way I can do this is by invoking the last test suite "FinalReport" and disable rest of the test scripts currently available in the running test suite.
The code is as following:
public testSuiteStop() {
def properties = new com.eviware.soapui.support.types.StringToObjectMap();
def reportTestCase = testRunner.testCase.testSuite.project.getTestSuiteByName("Report").getTestCaseByName("FinalReport");
reportTestCase.run(properties, true);
def testSuite = context.testCase.testSuite;
def totalTestCases = testSuite.getTestCases().size();
for(testCaseItem in (0..totalTestCases-1)) {
testSuite.getTestCaseAt(testCaseItem).setDisabled(true)
}
}
I have found a way to exit the current soapUI execution in case one of the critical test has already failed.
The idea is to disable the remaining test suites and test cases, and execute a "final" test suite that reports the failed test execution.
my code looks like this:
import com.eviware.soapui.impl.wsdl.WsdlProject;
import com.eviware.soapui.impl.wsdl.WsdlTestSuite;
import com.eviware.soapui.impl.wsdl.testcase.WsdlTestCase;
WsdlTestSuite testSuite = context.testCase.testSuite;
WsdlProject project = context.testCase.getProject();
// Disable remaining test cases in the current test suite.
for (WsdlTestCase testCase in testSuite.testCaseList) {
testCase.setDisabled(true);
}
// Disable rest of the test suites
for (WsdlTestSuite testSuiteName in project.testSuiteList) {
testSuiteName.setDisabled(testSuiteName.name != "LastTestSuite");
}
I am working on a SoapUI project where I need to run my test suite using test runner. I am using external groovy scripting for environment variable. The problem I am facing here is whenever I am running test case from test runner its return Workspace as null, which is used in External groovy. So in external groovy I am getting workspace as null causing error [getProjectByname() cannot be invoked on null]. Below is the
constructor of global script where workspace is used
AvengerAPITestManager(String TestProject, String TestSuite, String TestCase,String TestStep)
{
TestName = "AvengerAPITests";
testProject = SoapUI.getWorkspace().getProjectByName(TestProject);
tSuite = testProject.getTestSuiteByName(TestSuite);
tCase = testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase);
tStepName = TestStep.toString();
tStep=testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase).getTestStepByName (TestStep);
}
Above we have user SoapUI.getWorkspace() which is working fine when trying to run from soapUI but whever I m trying to run from testrunner SoapUI.getWorkspace comes out to be null. I even tried passing workspace like I am passing testproject name still it didnt worked.
I tried something like this also
AvengerAPITestManager(Object workspace,String TestProject, String TestSuite, String TestCase, String TestStep)
{
TestName = "AvengerAPITests";
testProject = workspace.getProjectByName(TestProject);
tSuite = testProject.getTestSuiteByName(TestSuite);
tCase = testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase);
tStepName = TestStep.toString();
tStep = testProject.getTestSuiteByName(TestSuite).getTestCaseByName(TestCase).getTestStepByName(TestStep);
}
In the above code I tries passing Workspace object from the test case as I passed Testcase name and all but still I m getting null for workspace. Please tell me how do I deal with the problem.
Here is usefull working example https://github.com/stokito/soapui-junit
You should place your sample-soapui-project.xml to /src/test/resources folder that will expose it to classpath
If you want to use soap ui in external code, try to directly create new test runner with specific project file:
SoapUITestCaseRunner runner = new SoapUITestCaseRunner();
runner.setProjectFile( "src/dist/sample-soapui-project.xml" );
runner.run();
Or if you want to define test execution more precisely, you can use something like this:
WsdlProject project = new WsdlProject( "src/dist/sample-soapui-project.xml" );
TestSuite testSuite = project.getTestSuiteByName( "Test Suite" );
TestCase testCase = testSuite.getTestCaseByName( "Test Conversions" );
// create empty properties and run synchronously
TestRunner runner = testCase.run( new PropertiesMap(), false );
PS: don't forget to import soap ui classes, that you use in your code and put them to classpath.
PPS: If you need just run test cases outside the soap ui and/or automate this process, why not just use testrunner.sh/.bat for the same thing? (here is description of this way: http://www.soapui.org/Test-Automation/functional-tests.html)
I am not sure if this is going to help anyone out there but here is what I did to fix the problem I was having with workspace as null causing error[getProjectByname() cannot be invoked on null] When i run from cmd
try this:
import com.eviware.soapui.model.project.ProjectFactoryRegistry
import com.eviware.soapui.impl.wsdl.WsdlProjectFactory
import com.eviware.soapui.impl.wsdl.WsdlProject
//get the Util project
def project = null
def workspace = testRunner.testCase.testSuite.project.getWorkspace();
//if running Soapui
if (workspace != null) {
project = workspace.getProjectByName("Your Project")
}
//if running in Jenkins/Hudson
else{
project = new WsdlProject("C:\\...\\....\\....\\-soapui-project.xml");
}
if (project.open && project.name == "Your Project") {
def properties = new com.eviware.soapui.support.types.StringToObjectMap()
def testCase = project.getTestSuiteByName("TestSuite 1").getTestCaseByName("TestCase");
if(testCase == null)
{
throw new RuntimeException("Could not locate testcase 'TestCase'! ");
} else {
// This will run everything in the selected project
runner = testCase.run(new com.eviware.soapui.support.types.StringToObjectMap(), false)
}
}
else {
throw new RuntimeException("Could not find project ' Order Id....' !")
}
The above code will run everything in the selected project.