Not found any reference to this particular question.
I am looking to find a way to achieve something like this in a Jenkins pipeline which runs our acceptance tests using Protractor and Cucumber.js:
steps {
container('selenium') {
script {
try {
{
//run tests
}
}
catch (err) {
if (env.testFailed == 'true') {
println "A test failure exists - build status updated to failure"
currentBuild.result = 'FAILURE'
error "Test(s) have failed"
}
else {
println "No test failures exist - build status updated to success"
currentBuild.result = 'SUCCESS'
}
}
}
}
}
This would fail the build if the env var of testFailed is 'true'. The reason for this is we are encountering bugs with Protractor-Cucumber framework where if a failed test retries and passes the exit code of the stage is still 1.
So in the After hook of each test I am setting the env var using node.js to true if the Scenario status is failed:
if (scenario.result.status === Status.FAILED) {
process.env.testFailed = 'true';
}
if (scenario.result.status === Status.PASSED) {
process.env.testFailed = 'false';
}
The problem I have found is that the Jenkins pipeline fails to read the env var value in the code block of the catch section. It is always null.
Any ideas?
1) change the After hook to write the true/false flag to a file in sync.
2) read the file in catch block
catch(err) {
testFailed = sh(script:'cat result.flag.txt', returnStdout: true).trim()
if(testFailed == 'true') {
...
}
}
Another option if there is total/passed/failed case number in output of npm test
lines = []
try {
lines = sh(script:'npm test', returnStdout: true).readLines();
}
catch(err) {
size = lines.size()
// parse the last 20 lines to extract fail/pass/total number
for(int i=size-20;i<size;i++) {
line[i]
}
}
WHY IT DOESN'T WORK NOW?
I see that you're running your tests in a container. When you set an environment variable, it's reflected on the scope of your container not the Jenkins master server
WHAT YOU COULD TRY TO DO
This actually depends on how you run the tests, but this should be an option
// run tests here
// you should have a variable for your container
def exit_code = sh(script: "sudo docker inspect ${container.id} --format='{{.State.ExitCode}}'", returnStdout: true)
sh "exit ${exit_code}"
This actually also depends how you start the tests inside the container,
So if you update your answer with this information I could help you
Related
While in development we occasionally use skip or only to debug a particular test or test suit. Accidentally, we might forget to revert the cases and push the code for PR. I am looking for a way to detect or automatically run all tests even for skip and only tests in our CI pipeline(using Github action). It can be in either case as follow.
Fail the test when there are skip or only tests.
Run all tests even for skip and only.
Very much appreciate any help.
I came up with a solution for the second part of the question about running all tests even for skip and only. I don't think it's elegant solution, but it works and it's easy to implement.
First of all you need to change test runner to jest-circus if you work with jest bellow 27.x version. We need it so our custom test environment will use handleTestEvent function to watch for setup events. To do so, install jest-circus with npm i jest-circus and then in your jest.config.js set testRunner property:
//jest.config.js
module.exports = {
testRunner: 'jest-circus/runner',
...
}
From Jest 27.0 they changed default test runner to jest-circus so you can skip this step if you have this or higher version.
Then you have to write custom test environment. I suggest to write it based on jsdom so for example we also have access to window object in tests and etc. To do so run in terminal npm i jest-environment-jsdom and then create custom environment like so:
//custom-jsdom-environment.js
const JsDomEnvironment = require('jest-environment-jsdom')
class CustomJsDomEnvironment extends JsDomEnvironment {
async handleTestEvent(event, state) {
if(process.env.IS_CI === 'true' && event.name === 'setup') {
this.global.describe.only = this.global.describe
this.global.describe.skip = this.global.describe
this.global.fdescribe = this.global.describe
this.global.xdescribe = this.global.describe
this.global.it.only = this.global.it
this.global.it.skip = this.global.it
this.global.fit = this.global.it
this.global.xit = this.global.it
this.global.test.only = this.global.test
this.global.test.skip = this.global.test
this.global.ftest = this.global.test
this.global.xtest = this.global.test
}
}
}
module.exports = CustomJsDomEnvironment
And inform jest to properly use it:
//jest.config.js
module.exports = {
testRunner: 'jest-circus/runner',
testEnvironment: 'path/to/custom/jsdom/environment.js',
...
}
Then you just have to setup custom environment value IS_CI in your CI pipeline and from now on all your skipped tests will run.
Also in custom test environment you could watch for skipped test and throw an error when your runner find skip/only. Unfortunately throwing an error in this place won't fail a test. You would need to find a way to fail a test outside of a test.
//custom-jsdom-environment.js
const JsDomEnvironment = require('jest-environment-jsdom')
const path = require('path')
class CustomJsDomEnvironment extends JsDomEnvironment {
constructor(config, context) {
super(config, context)
const testPath = context.testPath
this.testFile = path.basename(testPath)
}
async handleTestEvent(event, state) {
if(process.env.IS_CI === 'true' && event.name === 'add_test') {
if(event.mode === 'skip' || event.mode === 'only') {
const msg = `Run ${event.mode} test: '${event.testName}' in ${this.testFile}`
throw new Error(msg)
}
}
}
}
module.exports = CustomJsDomEnvironment
I build VS Code extension
I have wrapper in a class like this
public exec(cmd: string): string {
try {
return execSync(cmd, { cwd: this.workspaceRoot }).toString();
}
catch (e) {
return '' + e;
}
}
If in the code I run
let tags = this.exec('git tag --sort=-v:refname')
I do get a list of tags. Actually, all other commands run also correctly like git status, git config and other. But as soon as I run this.
let res = this.exec(`git push origin ${name}`);
It hangs forever. If I pass wrong tag it stops with error, but if I put correct tag, it suspends. I try to console.log('git push origin ${name}') and then copy result and run that command in a terminal, it runs correctly.
What can be the reason of a such behavior?
My current config file looks something like this:
function() {
var env = karate.env;
karate.log('karate.env system property was:', env);
karate.configure('ssl', true);
if (!env) {
env = 'dev';
}
var config = {
env: env,
internalGateway: 'https://gateway.com.au',
externalGateway: 'https://gateway.com.au',
GatewayManagerURL: 'https://manager.com.au'
}
if (env == 'dev') {
}
else if (env == 'e2e') {
}
return config;
}
This is the only file I have for environments. I am unsure as to how can I run my tests in multiple environments.
Do I need to create a new karate.config.<env>.js file (as per the docs) to run my tests in a new environment?
You need only this one file. Now the config JSON returned has some default values set. What you can do now is have different values for e2e, for example:
else if (env == 'e2e') {
config.internalGateway = 'https://gateway-e2e.com.au';
}
And when you run your tests, you switch environments on the command-line. This is just setting a Java System Property. There are many other ways to do this:
mvn test -DargLine="-Dkarate.env=e2e"
All this is explained here: https://github.com/intuit/karate#switching-the-environment
I am trying to get all the files matching a filter. Here is my code:
import static groovy.io.FileType.FILES
node {
try{
stage('get files')
new File('.').eachFileRecurse(FILES) {
if(it.name.endsWith('.nuspec')) {
echo it.Name
}
}
}catch(err) {
echo 'Err: Incremental Build failed with Error: ' + err.toString()
}finally {
}
I get the following error:
Err: Incremental Build failed with Error: org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new java.io.File java.lang.String
I've checked there are No pending script approvals.
I can't find any option to disable sandboxing.
The script runs fine when I try it in the script console.
How do I allow the script to run on my builds?
Thanks!
https://jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#findfiles-find-files-in-the-workspace
def files = findFiles(glob: '**/*.nuspec')
echo "${files[0].name} ${files[0].path} ${files[0].directory} ${files[0].length} ${files[0].lastModified}"
I am writing a groovy script where I am extracting text from a response and writing it to an output file on my system. The problem I encounter is when I run the groovy script that calls the test. The script assertions runs and logs the text twice to the file.
It seems to write to the file just before it leaves the assertion.
I have tried the following:
...
...
if (context.alreadyWritten == null || !context.alreadyWritten) {
inputFile.append (testString+ "\n")
log.info testString
} else {
log.info ('Already written!')
}
I have set the flag (context.alreadyWritten) to false in groovy before I execute the test step.
SOAPUI version : 5.3.0
I see there was an issue previous with Smartbear when appending to file in an assertion script. However the workaround was advised to resolve this:
if (context.alreadyWritten == null || !context.alreadyWritten) {
}
Which does not resolve my issue
When I log the result using log.info I see only one instance of the message been logged.
Any ideas.
Thanks
If you are using Script Assertion, try below:
//Define the file name, change as needed
def fileName = '/path/to/file.xml'
//check if you got the response
if (context.response) {
log.info 'Response available and not empty'
def file = new File(fileName)
if (!context?.alreadySaved) {
file.write(context.response)
context.alreadySaved = true
log.info 'response written to file'
} else {
log.info 'Response already written'
}
} else {
log.info 'there is no response to save'
}