I have a Jenkinsfile I would like to trigger certain steps based on env.JOB_NAME. As a test I have done this;
#!/usr/bin/env groovy
pipeline {
agent any
stages {
stage('Get ID') {
when {
"${env.JOB_NAME}" == 'Notification Sender (dev)'
}
steps {
echo "${env.JOB_NAME}"
}
}
}
}
However I get the error;
WorkflowScript: 6: Expected a when condition # line 6, column 11.
when {
^
WorkflowScript: 6: Empty when closure, remove the property or add some content. # line 6, column 11.
when {
^
Can I make the stage run based on the env.JOB_NAME using the when condition?
Yes, you can.
Try the following
when {
expression {
env.JOB_NAME == 'Notification Sender (dev)'
}
}
There is full documentation on the Pipeline syntax page, but the relevant part is
expression
Execute the stage when the specified Groovy expression evaluates to true, for example: when { expression { return params.DEBUG_BUILD } }
Related
Is it possible to hide a build step in jenkins pipeline?
I use shared library I would like to hide (hide==it will not present on the GUI) some steps when these is not needed
Thank you,
First off, I presume you are trying to hide a Stage, not a Step?
If so, it is not possible to completely hide it, simply because some builds might have the stage turned on, and some might not - therefore the stage should always be there, but empty(disabled) when excluded from the build.
You need to put when { expression { return ... } } at the beginning of your stage and return any expression that evaluates to true or false depending on if you want the stage to run or not.
Example code:
stage('My Test Stage') {
when {
expression {return <TRUE OR FALSE HERE>}
}
steps {
...
}
}
Solution (workaround):
stage('Steps for hide') {
steps {
script {
if ( conditions == "true" ) {
echo "steps 1"
}
else if ( conditions == "false" ){
echo "steps 2"
}
else if ( conditions == "null" ) {
echo "No conditions step required."
}
}
}
}
Trying to make SEND_SLACK_NOTIF, SLACK_CHANNEL and SLACK_MESSAGE variables and set them as environment variables globally in Jenkins file but I'm not sure which method is superior in a multi-stage pipeline. Don't mind the indenting!
Method 1:
#!/usr/bin/groovy
node('large') {
withEnv(['SEND_SLACK_NOTIF=true',
'SLACK_CHANNEL=UT24K22K1',
"SLACK_MESSAGE=FAILURE: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'\nBetter fix it! (${env.BUILD_URL})"]){
stage('Test') {
if (env.SEND_SLACK_NOTIF) {
slackSend channel: env.SLACK_CHANNEL, color: 'danger', message: env.SLACK_MESSAGE, tokenCredentialId: 'slack-integration-token'
}
}
}
}
Method 2:
#!/usr/bin/groovy
env.SEND_SLACK_NOTIF = true
env.SLACK_CHANNEL = 'UT24K22K1'
env.SLACK_MESSAGE = "FAILURE: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]'\nBetter fix it! (${env.BUILD_URL})"
node('large') {
stage('Test') {
if (env.SEND_SLACK_NOTIF) {
slackSend channel: env.SLACK_CHANNEL, color: 'danger', message: env.SLACK_MESSAGE, tokenCredentialId: 'slack-integration-token'
}
}
}
If you just consider your two options, I'd go with option 1 because it's more declarative than option 2. However the superior way is the one described in the Jenkins documentation:
pipeline {
agent {
...
}
environment {
SEND_SLACK_NOTIF = 'true'
SLACK_CHANNEL = 'UT24K22K1'
}
stages {
...
}
}
You might also want to reconsider which variables you really need. If you only use them once, don't use a variable, but use the value directly.
Jenkins 2.89.4 rolling
I saw almost all stackoverflow posts which show how we can successfully run parallel steps/stages (using list/maps etc) --OR hardcoding them directly --OR even create dynamic stages for Jenkinsfile (as seen in this post: Scripted jenkinsfile parallel stage)
My requirements are:
A pipeline which builds N. no of projects under "BUILD" steps i.e. parallel builds on each of those projects. i.e. it runs Gradle on all N projects. Here I have a Jenkinsfile which was created by a declarative JOB DSL Groovy. Here my Gradle projects are not set as multi-projects so I can't call the top level gradle and say, Gradle please do your parallel magic (within Gradle).
I want to run build of these N projects in their own separate parallel dynamically created stages (GUI Columns) as seen in jenkins job's dashboard.
I want to see the output of (Gradle build/console) each project's build separately i.e. I don't want to mix the console output of each projects build which are running in parallel in just ONE COLUMN (i.e. column named BUILD).
In this URL https://jenkins.io/blog/2017/09/25/declarative-1/ I see, how you can run parallel stages/steps but in doing so, either it's mixing those parallel step's output in just one column (I mean under BUILD column) --OR if you want it under separate stages/columns (i.e. the post says Test on Linux or Windows separately then you are still hard-coding all the stages/steps in Jenkinsfile early on (rather than using just a list or array hash which I'd prefer to update for adding more or less stages / parallel steps as in my case, they all follow the same standard). What I want is to just update in one place How many steps and what are all of the stages in just one place (list/array).
I'm not using Jenkins Blue Ocean for now.
Usually if you have parallel steps within a stage, their console std output for all steps gets mixed into one console output / stage/column when you click to see the console output for that given parallel step/stage; When you hover over the BUILD column (assuming there were parallel steps in BUILD stage) in job's dashboard (std output for all those steps is mixed up and very hard to see individual project step's console output just for a given step/stage).
If we want to create separate stages (dynamically) then Jenkins should be able to show console output of a given step/dynamic stage within the parallel section (i.e. each column should show their own project's build console output).
Using the above URL, I'm able to do the following after trying this script:
// main script block
// could use eg. params.parallel build parameter to choose parallel/serial
def runParallel = true
def buildStages
node('master') {
stage('Initializing Parallel Dynamic Stages') {
// Set up List<Map<String,Closure>> describing the builds
buildStages = prepareBuildStages()
println("Initialised pipeline.")
}
for (builds in buildStages) {
if (runParallel) {
parallel(builds)
} else {
// run serially (nb. Map is unordered! )
for (build in builds.values()) {
build.call()
}
}
}
stage('Done') {
println('The whole SHENZI is complete.')
}
}
// Create List of build stages to suit
def prepareBuildStages() {
def buildList = []
for (i=1; i<4; i++) {
def buildStages = [:]
for (name in [ 'Alpha', 'Tango', 'Chyarli' ] ) {
def n = "${name} ${i}"
buildStages.put(n, prepareOneBuildStage(n))
}
buildList.add(buildStages)
}
return buildList
}
def prepareOneBuildStage(String name) {
def proj_name = name.split(' ')[0]
def proj_parallel_sub_step = name.split(' ')[1]
//Return the whole chunkoni
return {
stage("Build\nProject-${proj_name}\nStep ${proj_parallel_sub_step}") {
println("Building ${proj_name} - ${proj_parallel_sub_step}")
sh(script:'sleep 15', returnStatus:true)
}
}
}
When I'm putting the above Groovy Script (which is creating DYNAMIC Stages) inside Pipeline Script or Pipeline Script from SCM (i.e. the same code available in a .groovy file) -- it runs successfully and creates dynamic stages under BUILD step for each of the 3 projects and runs 3 steps (Nth) for all 3 projects in parallel and then starts the next Nth step for all 3 projects and so on.
If you see below, we also got individual columns in Jenkins job dashboard for them.
Now, When I put the above script in Jenkinsfile (Pipeline DSL) where I have pipeline { .... } section, it's not working and giving me the following error.
Using my JOB DSL, I created a new Jenkins Pipeline job where Pipeline Script from SCM calls a groovy file (which now contains):
//----------------------------------------------------
// Both - Parallel Run and GUI View in JF Jenkins job.
//----------------------------------------------------
def runParallel = true
def buildStages
def wkspace = /var/lib/jenkins/workspaces/ignore_this_variale_or_its_value_for_now
// Create List of build stages to suit
def prepareBuildStages() {
def buildList = []
for (i=1; i<3; i++) {
def buildStages = [:]
for (name in [ 'Alpha', 'Tango', 'Chyarli' ] ) {
def n = "${name} ${i}"
buildStages.put(n, prepareOneBuildStage(n))
}
buildList.add(buildStages)
}
return buildList
}
//---
def prepareOneBuildStage(String name) {
def proj_name = name.split(' ')[0]
def proj_parallel_sub_step = name.split(' ')[1]
// return the whole chunkoni (i.e. for a given stage) - will be named dynamically.
return {
stage("Build\nProject-${proj_name}\nStep ${proj_parallel_sub_step}") {
println("Building ${proj_name} - ${proj_parallel_sub_step}")
sh(script:'sleep 15', returnStatus:true)
}
}
}
// Set up List<Map<String,Closure>> describing the builds
buildStages = prepareBuildStages()
//---------------------
String jenkinsBaselines
// SEE NOW --- we have this section called 'pipeline'
pipeline {
agent {
node {
label 'rhat6'
customWorkspace wkspace
}
}
options {
ansiColor('xterm')
timeout(time: 8, unit: 'HOURS')
skipDefaultCheckout()
timestamps()
}
environment {
someEnvVar = 'aValue'
}
//------------- Stages
stages {
stage('Initializing Parallel Dynamic Stages') {
// Set up List<Map<String,Closure>> describing the builds
println("Initialised pipeline.")
}
for (builds in buildStages) {
if (runParallel) {
parallel(builds)
} else {
// run serially (nb. Map is unordered! )
for (build in builds.values()) {
build.call()
}
}
}
stage('Done') {
println('The whole SHENZI is complete.')
}
}
//---------------------
}
Running the Jenkinsfile Jenkins job now gives me this error:
[BFA] Scanning build for known causes...
[BFA] No failure causes found
[BFA] Done. 1s
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 69: Not a valid stage section definition: "buildStages = prepareBuildStages()". Some extra configuration is required. # line 69, column 5.
stage('Initializing Parallel Dynamic Stages') {
^
WorkflowScript: 69: Unknown stage section "println". Starting with version 0.5, steps in a stage must be in a steps block. # line 69, column 5.
stage('Initializing Parallel Dynamic Stages') {
^
WorkflowScript: 75: Expected a stage # line 75, column 5.
for (builds in buildStages) {
^
WorkflowScript: 86: Unknown stage section "println". Starting with version 0.5, steps in a stage must be in a steps block. # line 86, column 5.
stage('Done') {
^
WorkflowScript: 69: No "steps" or "parallel" to execute within stage "Initializing Parallel Dynamic Stages" # line 69, column 5.
stage('Initializing Parallel Dynamic Stages') {
^
WorkflowScript: 86: No "steps" or "parallel" to execute within stage "Done" # line 86, column 5.
stage('Done') {
^
6 errors
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:133)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:557)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:518)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:290)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
How can I get this working in Jenkinsfile pipeline section and still able to get individual columns per dynamically created stage for a given project N and Step M?
Tried the following way, still errors the say way.
//------------- Stages
stages {
stage('Initializing Parallel Dynamic Stages') {
// Set up List<Map<String,Closure>> describing the builds
buildStages = prepareBuildStages()
println("Initialised pipeline.")
// tried this way too. within a stage
buildStages.each { bld -->
parallel(bld)
}
}
stage('Done') {
println('The whole SHENZI is complete.')
}
}
//---------------------
So I did some poking into this and got it working.
The new code within Jenkinsfile (for STAGES) is now:
//------------- Stages
stages {
stage('Start Pipeline') {
steps {
script {
sh "echo HELLO moto razr!"
}
}
}
stage('Initializing Parallel Dynamic Stages'){
steps {
script {
// Run all Nth step for all Projects in Parallel.
buildStages.each { bs -> parallel(bs) }
// OR uncomment the following code (if conditional on boolean variable).
/*
for (builds in buildStages) {
if (runParallel) {
parallel(builds)
} else {
// run serially (nb. Map is unordered! )
for (build in builds.values()) {
build.call()
}
}
}
*/
}
}
}
stage('Done') {
println('The whole SHENZI is complete.')
}
}
//---------------------
That's all it took to work.
For clear messages / stage names, I tweaked the function as well and We won't set this variable buildStages within pipeline { ... }
//---
def prepareOneBuildStage(String name) {
def proj_name = name.split(' ')[0]
def proj_parallel_sub_step = name.split(' ')[1]
// return the whole chunkoni (i.e. for a given stage) - will be named dynamically.
return {
stage("BUILD Project-${proj_name} Parallel_Step_${proj_parallel_sub_step}") {
println("Parallel_Step # ${proj_parallel_sub_step} of Project => ${proj_name}")
sh(script:"echo \"Parallel_Step # ${proj_parallel_sub_step} of Project => ${proj_name}\" && sleep 20", returnStatus:true)
// -- OR -- you can call Gradle task i.e. rpm / any other / any other command here.
}
}
}
// Set up List<Map<String,Closure>> describing the builds. section now.
buildStages = prepareBuildStages()
//---------------------
This implementation is now creating N no. of parallel stages i.e. a separate column per project for a given Nth step (when looking at Jenkinsfile job's dashboard) for P no. of projects.
It will run All P projects for a given Nth step in parallel.
It will wait for Nth step for all Projects to complete first and then jump to the next Nth step.
What this means is, if Project ALPHA Step #1 is complete, it'll still wait for all Step #1 of other 2 projects and then launch Step #2 of all projects in parallel.
Challenge: How can we make ALPHA Project's Step #2 to start as soon as ALPHA project's Step #1 is complete i.e. it won't wait for Step 1 of other 2 projects to complete and could possibly run Step #2 of ALPHA project 1 in parallel with Step N(=1) or N+1 of other projects.
This assumes all projects are independent of each other and projects don't share contents generated by a given project/their stage/steps in any other project/stage/step.
Depending upon your own requirements, you may want to wait (i.e. don't run Step 2 of all projects until Step 1 of all projects are fully complete) --OR-- you may want to run Step 2 of ALPHA project with let's say - Step 2 of TANGO project while project CHYARLI's step 1 is still in progress.
As the main scope of this post was to get separate dynamically created columns/stages per project (running in parallel within pipeline { ... } section), I think, I got what I was looking for.
NOTE: Go easy with parallel if you want to run concurrent builds of a pipeline. For more info about issues related to running parallel build actions concurrently, see here: Jenkins - java.lang.IllegalArgumentException: Last unit does not have enough valid bits & Gradle error: Task 'null' not found in root project
You can also run the stages parallel as follow
pipeline {
agent { node { label 'master' } }
stages {
stage('Add regression tests') {
steps {
script {
def testList = ["a", "b", "c", "d"]
def branches = [:]
for (int i = 0; i < 4 ; i++) {
int index=i, branch = i+1
stage ("branch_${branch}"){
branches["branch_${branch}"] = {
node ('master'){
sh "echo 'node: ${NODE_NAME}, index: ${index}, i: ${i}, testListVal: " + testList[index] + "'"
}
}
}
}
parallel branches
}
}
}
}
}
I have a Jenkinsfile with multiple stages and one of them is in fact another job (the deploy one) which can fail in some cases.
I know that I can made prompts using Jenkinsfile but I don't really know how to implement a retry mechanism for this job.
I want to be able to click on the failed stage and choose to retry it.
You should be able to combine retry + input to do that
Something like that
stage('deploy-test') {
try {
build 'yourJob'
} catch(error) {
echo "First build failed, let's retry if accepted"
retry(2) {
input "Retry the job ?"
build 'yourJob'
}
}
}
you could also use timeout for the input if you want it to finish if nobody validates.
There is also waitUntil that might be useful but i haven't used it yet
Edit :
WaitUntil seems definitely the best, you should play with it a bit but something like that is cleaner :
stage('deploy-test') {
waitUntil {
try {
build 'yourJob'
} catch(error) {
input "Retry the job ?"
false
}
}
}
By the way, there is doc all of the steps here https://jenkins.io/doc/pipeline/steps
This one with a nice incremental wait
stage('deploy-test') {
def retryAttempt = 0
retry(2) {
if (retryAttempt > 0) {
sleep(1000 * 2 + 2000 * retryAttempt)
}
retryAttempt = retryAttempt + 1
input "Retry the job ?"
build 'yourJob'
}
}
This gist (not mine) was one of the better options that I found while trying to implement this functionality too. https://gist.github.com/beercan1989/b66b7643b48434f5bdf7e1c87094acb9
Changed it to a method in a shared library that just did retry or abort for my needs. Also added a max retries and made the timeout variable so that we could change it depending on the job or stage that needs it.
package com.foo.bar.jenkins
def class PipelineHelper {
def steps
PipelineHelper(steps) {
this.steps = steps
}
void retryOrAbort(final Closure<?> action, int maxAttempts, int timeoutSeconds, final int count = 0) {
steps.echo "Trying action, attempt count is: ${count}"
try {
action.call();
} catch (final exception) {
steps.echo "${exception.toString()}"
steps.timeout(time: timeoutSeconds, unit: 'SECONDS') {
def userChoice = false
try {
userChoice = steps.input(message: 'Retry?', ok: 'Ok', parameters: [
[$class: 'BooleanParameterDefinition', defaultValue: true, description: '', name: 'Check to retry from failed stage']])
} catch (org.jenkinsci.plugins.workflow.steps.FlowInterruptedException e) {
userChoice = false
}
if (userChoice) {
if (count <= maxAttempts) {
steps.echo "Retrying from failed stage."
return retryOrAbort(action, maxAttempts, timeoutMinutes, count + 1)
} else {
steps.echo "Max attempts reached. Will not retry."
throw exception
}
} else {
steps.echo 'Aborting'
throw exception;
}
}
}
}
}
Example usage with a max of 2 retries that waits for 60s for input.
def pipelineHelper = new PipelineHelper(this)
stage ('Retry Example'){
pipelineHelper.retryOrAbort({
node{
echo 'Here is an example'
throw new RuntimeException('This example will fail.')
}
}, 2, 60)
}
Just remember to put nodes inside of the closure so that waiting for an input doesn't block an executor.
If you have the paid jenkins enterprise Cloudbees has a Checkpoint plugin that can better handle this, but it is not planned to be release for open source Jenkins (JENKINS-33846).
Is There a way to check if(do.condion=='11') in Groovy DSL.
if(object.member == '2') //then do my logic
I am not able to use ==. its not throwing any error it just going to next statement
If I'm understanding your DSL correctly, formatted for clarity it looks like this:
AcceptILPN {
input scanCase
if(workflowParameters.reserveVerificationModeParm.equals("1")) next AcceptSKUQuantity
if(workflowParameters.validateCarton.equals("1")) next AcceptOLPNOrTote
if(workflowDO.nextDtlPresent) next AcceptILPN else next AcceptPickCart
}
To see what's happening, here's the same code with a more formal syntax:
AcceptILPN {
input(scanCase)
if(workflowParameters.reserveVerificationModeParm.equals("1")) {
next(AcceptSKUQuantity)
}
if(workflowParameters.validateCarton.equals("1")) {
next(AcceptOLPNOrTote)
)
if(workflowDO.nextDtlPresent) {
next(AcceptILPN)
} else {
next(AcceptPickCart)
}
}
As you can see, even when the first if expression evaluates to true the following if blocks will execute because there's nothing (at least visible in the DSL) that exits the Closure prematurely. It seems you're looking for something like this:
AcceptILPN {
input(scanCase)
if(workflowParameters.reserveVerificationModeParm.equals("1")) {
next(AcceptSKUQuantity)
}
else if(workflowParameters.validateCarton.equals("1")) {
next(AcceptOLPNOrTote)
)
else if(workflowDO.nextDtlPresent) {
next(AcceptILPN)
} else {
next(AcceptPickCart)
}
}