Multibranch Pipeline Jenkinsfile Looking for advice/Feedback - dsl

Ok, Below you find my opus.. (lol not really) but still this is my attempt to take a mess and automate it. Those who are willing, any advice would be appreciated.
The project all lives in one git repo. very much like https://github.com/jmvanryn/multibranchPipeline
some of my questions are..
could I be doing some of this better? how do I use Global variables?
you can see I'm defining some things twice. why does GIT pull twice?
is there a way to get the cobertura report plugin to work?
env.BRANCH_NAME gives me "feature/FeatureName" is there a
DSL/Groovy way to trim it to just FeatureName?
node {
wrap([$class: 'BuildUser']) {
def user = env.BUILD_USER_ID
}
env.JAVA_HOME = tool 'JDK1.7.0.21'
def mvnHome = tool 'M3'
def nodeJS = tool 'NodeJSv5.6.0'
env.MAVEN_OPTS = "-Xmx1024m -XX:MaxPermSize=512m"
env.PATH = "${mvnHome}/bin:${nodeJS}/bin:${env.JAVA_HOME}/bin:${env.PATH}"
//checkout scm
sh 'git rev-parse --verify HEAD > hash.txt'
env.GIT_COMMIT = readFile('hash.txt')
env.GIT_COMMIT = env.GIT_COMMIT.trim()
setVersion();
stage("Build Core")
hipchatSend color: 'GREEN', notify: true, room: '1654572'
BuildIt ("sharedLib1")
BuildIt ("sharedLib2")
stage("Build Modules")
parallel "first-ui": {
BuildIt ("first-ui")
}, "second-ui": {
BuildIt ("second-ui")
}, "first-ws": {
BuildIt ("first-ws")
}, "second-ws": {
BuildIt ("second-ws")
}
stage("Archive files")
echo("================================== Archive Stage ==================================")
step([$class: 'ArtifactArchiver', artifacts: '**/target/*.war', fingerprint: true])
//step([$class: 'ArtifactArchiver', artifacts: 'CONFIG/*.*', fingerprint: true])
step([$class: 'JUnitResultArchiver', testResults: '**/target/surefire-reports/TEST-*.xml'])
step([$class: 'hipchat', room: '1654572', startNotification: false, notifySuccess: true, notifyAborted: false, notifyNotBuil: false, notifyUnstable: false, notifyFailure: true, notifyBackToNormal: false])
stage("Notification")
step([$class: 'Mailer', notifyEveryUnstableBuild: true, recipients: 'jvanryn#liaison-intl.com', sendToIndividuals: true])
}
def BuildIt(module) {
sh "echo '================================== Build Stage : ${module} ==================================';\
cd ${module};\
mvn --batch-mode -V -U -e clean deploy -U -DskipITs sonar:sonar -Dsonar.branch=${env.BRANCH_NAME} -Dsurefire.useFile=false"
}
def setVersion () {
def JobCode = env.BRANCH_NAME
def JobName = env.JOB_NAME
def BuildURL = env.BUILD_URLD
def BuildNum = env.BUILD_NUMBER
def RevNum = env.GIT_COMMIT
def WrkSpce = pwd()
echo "Running Build for: "+JobCode
// update version to a unique version
def script=WrkSpce+"/Tools/PomVersionUpdater.sh "+JobCode+"-"+RevNum+"-"+BuildNum
sh script
}

First, you can create a .mvn folder with jvm.config and maven.config to store all those xmx parameters and others, so everyone is using the same build settings.
You also need a try/catch block to send some notification if the build fails.
Other than that it looks ok, could be better but it works

Related

how to write the correct pipline jenkins docker grovy node

I am rewriting my pipline in node, I need to understand how to perform a step with a gait in node now an error is coming from stage('Deploy')
node {
checkout scm
def customImage = docker.build("python-web-tests:${env.BUILD_ID}")
customImage.inside {
sh "python ${env.CMD_PARAMS}"
}
stage('Deploy') {
post {
always {
allure([
includeProperties: false,
jdk: '',
properties: [],
reportBuildPolicy: 'ALWAYS',
results: [[path: 'report']]
])
cleanWs()
}
}
}
and this is the old pipeline
pipeline {
agent {label "slave_first"}
stages {
stage("Создание контейнера image") {
steps {
catchError {
script {
docker.build("python-web-tests:${env.BUILD_ID}", "-f Dockerfile .")
}
}
}
}
stage("Running and debugging the test") {
steps {
sh 'ls'
sh 'docker run --rm -e REGION=${REGION} -e DATA=${DATA} -e BUILD_DESCRIPTION=${BUILD_URL} -v ${WORKSPACE}:/tmp python-web-tests:${BUILD_ID} /bin/bash -c "python ${CMD_PARAMS} || exit_code=$?; chmod -R 777 /tmp; exit $exit_code"'
}
}
}
post {
always {
allure([
includeProperties: false,
jdk: '',
properties: [],
reportBuildPolicy: 'ALWAYS',
results: [[path: 'report']]
])
cleanWs()
}
}
}
I tried to transfer the method of creating an allure report, but nothing worked, I use the version above, almost everything turned out, you can still add environment variables to the build, for example, those that are specified -e DATA=${DATA} how do I add it
I don't recommend to switch from declarative to scriptive pipeline.
You are losing possibility to use multiple tooling connected with declarative approach like syntax checkers.
If you still want to use scriptive approach try this:
node('slave_first') {
stage('Build') {
checkout scm
def customImage = docker.build("python-web-tests:${env.BUILD_ID}")
customImage.inside {
sh "python ${env.CMD_PARAMS}"
}
}
stage('Deploy') {
allure([
includeProperties: false,
jdk: '',
properties: [],
reportBuildPolicy: 'ALWAYS',
results: [[path: 'report']]])
cleanWs()
}
}
There is no post and always directive in scriptive pipelines. It's on your head to catch all exceptions and set status of the job. I guess you were using this page: https://www.jenkins.io/doc/book/pipeline/syntax/, but it's a mistake.
This page only refers to declarative approach and in few cases you have hidden scriptive code as examples.
Also i don't know if you have default agent label set in your Jenkins config, but by looking at your declarative one I think you missed 'slave_first' arg in node object.
those that are specified -e DATA=${DATA} how do I add it
That's a docker question not a Jenkins. If you want to launch docker image and then also have access to some reports located in this container you should mount workspace/file where those output files landed. You should also pass location of those files to allure.
I suggest you to try this:
mount some subfolder in workspace to docker container
cat test report file if it's visible
add allure report with passing this file location to allure step

How to set conditions in the parallel build to proceed to next stage if one step is success

I am creating a declarative pipeline in Jenkins. There are 6 stages in it.
First Stage: Scenario Upload
Second Stage: Pull code from Git
Third Stage: Maven Build
Fourth Stage: Its a parallel stage. First step will launch mobile emulator and second step will check device connected or not.
Fifth Stage: I want to start this stage when the second step BUILD SUCCESS else stop the job
Sixth Stage: Send email
I am stuck with point 5 (Fifth Stage). Please help
pipeline {
agent any
stages {
stage("Scenario Upload") {
steps {
script {
def inputFile = input message: 'Upload file', parameters: [file(name: 'CyclosAppStatus.xlsx')]
new hudson.FilePath(new File("$workspace/Cucumber_BDD master/Result/CyclosAppStatus.xlsx")).copyFrom(inputFile)
inputFile.delete()
}
}
}
stage('Git Pull Code') {
steps {
git credentialsId: '708a126a-66bb-4eb5-8826-55cedf6497c3', url: 'https://github.com/divakar-ragupathy/Mobile_Automation_BDD.git'
}
}
stage('Maven Clean Build') {
steps {
bat label: '', script: '''Echo Maven Clean Build...
cd %WORKSPACE%\\ADB_Devices
mvn clean compile'''
}
}
stage('Building Android Setup') {
steps {
parallel(
Invoke_Emulator: {
bat label: '', script: '''Echo Invoking Emulator...
#echo off
set emulName=%Emulator_Name%
echo %emulName%
for /f "tokens=1 delims=:" %%e in ("%emulName%") do (
%ANDROID_AVD_PATH%emulator -avd "%%e" -no-boot-anim -no-snapshot-save -no-snapshot-load
)
endlocal'''
},
Checking_Device: {
bat label: '', script: '''Echo Checking Connected Device...
cd %WORKSPACE%\\ADB_Devices
mvn exec:java -Dexec.mainClass=com.expleo.adbListner.CheckConnectedAdbDevices -Dlog4j.configuration=file:///%WORKSPACE%\\ADB_Devices\\src\\log4j.properties -Dexec.args="%Emulator_Name%"'''
}
)
}
}
}
}
If you declare a variable without the "def" keyword it is global. You can use that to store the condition in the previous stages. In the 5th stage you can use a when block to check this condition.

How to validate the output of grep in Jenkinsfile groovy

I am trying to run a command to retrieve the latest commit message in git and then do a grep to see if it contains certain string and then use that as a condition but getting errors.
def commitmsg = sh(returnStdout: true, script: "git log -1 --pretty='%s' | grep Merge").trim()
if (env.BRANCH_NAME == 'release' && commitmsg == 'Merge')
{
..........
}
How do i get this to work? I just want that if the output of
git log -1 --pretty='%s'
contains the string Merge then i can use it under if condition
I was able to get this to work.
def commitmsg = sh(returnStdout: true, script: "git log -1 --pretty='%s'").trim()
if (env.BRANCH_NAME == 'release' && commitmsg.contains("Merge pull request") == true)
{
....
}

Calling Jenkins git plugin from a shared library class

I have a long standing declarative pipeline infrastructure
I would like to start putting repeated code into shared libraries
The problem I am facing is calling the git plugin from a shared library function/class. I'm a bit lost as my experience is really only with Jenkins declarative stuff, not the Groovy/Java specifics.
Here is a snippet of the Jenkinsfile, (before using shared library):
pipeline {
agent any
stages {
stage('Prep Workspace') {
steps {
script {
if ((env.BRANCH_NAME == 'staging') || (env.BRANCH_NAME == 'production')) {
BRANCH=env.BRANCH_NAME
} else {
BRANCH='master'
}
}
echo "||------ Get ProjectOne Dependency ------||"
dir('deps/ProjectOne') {
git branch: "${BRANCH}",
changelog: false,
credentialsId: 'jenkinsgit',
poll: false,
url: 'git#github.com:myprivateorg/ProjectOne.git'
}
echo "||------ Get ProjectTwo Dependency ------||"
dir('deps/ProjectTwo') {
git branch: "${BRANCH}",
changelog: false,
credentialsId: 'jenkinsgit',
poll: false,
url: 'git#github.com:myprivateorg/ProjectTwo.git'
}
}
}
}
}
Note the repeated calls to pull down project files from git repos. The goal here, is to move the repeated code to a shared function call.
I've read the following portion in the manual, on how to use git in shared library:
https://www.jenkins.io/doc/book/pipeline/shared-libraries/#accessing-steps
Using the example in the documentation I've created the shared library file
In src/org/test/gitHelper.groovy:
package org.test;
def checkOutFrom(String repo, String branch='master') {
echo "||------ CLONING $repo ------||"
git branch: branch, changelog: false, credentialsId: 'jenkinsgit', poll: false, url: "git#github.com:myprivateorg/$repo.git"
}
return this
Then in the Jenkinsfile:
#Library('jenkins-shared-library') _
pipeline {
agent any
stages {
stage('Prep Workspace') {
steps {
script {
if ((env.BRANCH_NAME == 'staging') || (env.BRANCH_NAME == 'production')) {
BRANCH=env.BRANCH_NAME
} else {
BRANCH='master'
}
def g = new org.test.gitHelper()
g.checkOutFrom('ProjectOne')
g.checkOutFrom('ProjectTwo')
}
}
}
}
}
This loads the class and calls the function fine, but fails when it hits git itself:
groovy.lang.MissingPropertyException: No such property: git for class: java.lang.String
I used g.getClass() to confirm it's of type class org.test.gitHelper and NOT java.lang.String so I'm not sure where it's getting that type from.
Please note I have also tried this way:
vars/pullRepo.groovy
def call(String repo, String branch) {
echo "||------ CLONING $repo ------||"
dir("deps/$repo") {
git branch: branch, changelog: false, credentialsId: 'jenkinsgit', poll: false, url: "git#github.com:myprivateorg/$repo.git"
}
}
Jenkinsfile:
pullRepo('ProjectOne', 'master')
I get the exact same error: groovy.lang.MissingPropertyException: No such property: git for class: java.lang.String
For me, it works to pass the Jenkins context to the shared library like so:
Jenkinsfile:
pullRepo(this, repo, branch)
vars/pullRepo.groovy:
def call(def context, String repo, String branch) {
echo "||------ CLONING $repo ------||"
dir("deps/$repo") {
context.git branch: branch, changelog: false, credentialsId: 'jenkinsgit', poll: false, url: "git#github.com:myprivateorg/$repo.git"
}
}
Note that I'm passing the Jenkins context into the context variable, and calling git as a method of the context. You should also be able to do this by passing the context up to your class.

How can I use the Jenkins Copy Artifacts Plugin from within the pipelines (jenkinsfile)?

I am trying to find an example of using the Jenkins Copy Artifacts Plugin from within Jenkins pipelines (workflows).
Can anyone point to a sample Groovy code that is using it?
With a declarative Jenkinsfile, you can use following pipeline:
pipeline {
agent any
stages {
stage ('push artifact') {
steps {
sh 'mkdir archive'
sh 'echo test > archive/test.txt'
zip zipFile: 'test.zip', archive: false, dir: 'archive'
archiveArtifacts artifacts: 'test.zip', fingerprint: true
}
}
stage('pull artifact') {
steps {
copyArtifacts filter: 'test.zip', fingerprintArtifacts: true, projectName: env.JOB_NAME, selector: specific(env.BUILD_NUMBER)
unzip zipFile: 'test.zip', dir: './archive_new'
sh 'cat archive_new/test.txt'
}
}
}
}
Before version 1.39 of the CopyArtifact, you must replace second stage with following (thanks #Yeroc) :
stage('pull artifact') {
steps {
step([ $class: 'CopyArtifact',
filter: 'test.zip',
fingerprintArtifacts: true,
projectName: '${JOB_NAME}',
selector: [$class: 'SpecificBuildSelector', buildNumber: '${BUILD_NUMBER}']
])
unzip zipFile: 'test.zip', dir: './archive_new'
sh 'cat archive_new/test.txt'
}
}
With CopyArtifact, I use '${JOB_NAME}' as project name which is the current running project.
Default selector used by CopyArtifact use last successful project build number, never current one (because it's not yet successful, or not). With SpecificBuildSelector you can choose '${BUILD_NUMBER}' which contains current running project build number.
This pipeline works with parallel stages and can manage huge files (I'm using a 300Mb file, it not works with stash/unstash)
This pipeline works perfectly with my Jenkins 2.74, provided you have all needed plugins
If you are using agents in your controller and you want to copy artifacts between each other you can use stash/unstash, for example:
stage 'build'
node{
git 'https://github.com/cloudbees/todo-api.git'
stash includes: 'pom.xml', name: 'pom'
}
stage name: 'test', concurrency: 3
node {
unstash 'pom'
sh 'cat pom.xml'
}
You can see this example in this link:
https://dzone.com/refcardz/continuous-delivery-with-jenkins-workflow
If builds are not running in the same pipeline you can use direct CopyArtifact plugin, here is example: https://www.cloudbees.com/blog/copying-artifacts-between-builds-jenkins-workflow and example code:
node {
// setup env..
// copy the deployment unit from another Job...
step ([$class: 'CopyArtifact',
projectName: 'webapp_build',
filter: 'target/orders.war']);
// deploy 'target/orders.war' to an app host
}
name = "/" + "${env.JOB_NAME}"
def archiveName = 'relNum'
try {
step($class: 'hudson.plugins.copyartifact.CopyArtifact', projectName: name, filter: archiveName)
} catch (none) {
echo 'No artifact to copy from ' + name + ' with name relNum'
writeFile file: archiveName, text: '3'
}
def content = readFile(archiveName).trim()
echo 'value archived: ' + content
try that using copy artifact plugin

Resources