We want to split up our project into smaller pieces. Our current CI process goes through a short test phase and then runs a deployment script. However, if nothing has changed in one of the sub project, we do not want to go through the build for this.
Jenkins without pipelines supports exclusions in the SCM configuration (we use git) and based on this, you can configure a specific job to run. However, when using a pipeline, how can I know, if I should build this part or not? How do I get access to the paths that were affected by the last push?
At the moment our script is very simple, and we would like to keep it as simple as possible.
We were playing around with the scripted and the declarative syntax, but could not find a good solution.
Declarative:
#!groovy
pipeline {
agent any
tools {
nodejs '8.1'
}
stages {
stage('Checkout') {
steps {
checkout scm
}
}
# Only continue, if something has changed
stage('Install') {
steps {
sh 'npm install'
}
}
stage('Test') {
steps {
sh 'npm run test-jenkins'
}
post {
always {
junit "artifacts/test/report.xml"
}
}
}
}
}
Scripted:
#!groovy
node {
def nodejs = tool name: '8.1', type: 'jenkins.plugins.nodejs.tools.NodeJSInstallation'
env.PATH = "${nodejs}/bin:${env.PATH}"
stage('Checkout') {
checkout scm
}
# Only continue, if something has changed
stage('Install') {
sh 'npm install'
}
stage('Test') {
try {
sh 'npm run test-jenkins'
} finally {
junit "artifacts/test/report.xml"
}
}
}
Thanks to ElpieKay's fast comment on my question, we now have an elegant solution:
Make a tag to the current commit on a successful build
In the next build compare the new commit and the tag for changes
We are using a multi-branch pipeline and a parallel build for the multiple projects we have under the same source root. We iterate through the projects (serviceX) and check in the corresponding directory for a change:
def projects = ['service1', 'service2']
def builders = [:]
for (p in projects) {
def label = p
builders[label] = {
def tag = "${BRANCH_NAME}_last"
node {
echo "Checking for changes compared to ${tag} in directory ${label}"
try {
sh "./check-for-changes ${tag} ${label}"
} catch (ignored) {
echo "Nothing to do"
return
}
dir (label) {
stage(label + ": Install") {
sh "npm install"
}
stage(label + ": Test") {
try {
sh "npm run test-jenkins"
} finally {
junit 'artifacts/test/report.xml'
}
}
echo "Setting tag for the last build on this branch"
sh "git tag -f ${tag}"
}
}
}
}
parallel builders
... and the script to check for changes:
#!/bin/bash
SHA_PREV=$1
if [ -z ${SHA_PREV} ]; then
echo "Usage: `basename $0` <tag> <path>"
exit 1
fi
CHECK_PATH=$2
if [ -z ${CHECK_PATH} ]; then
echo "Usage: `basename $0` <tag> <path>"
exit 1
fi
if `git rev-parse ${SHA_PREV} >/dev/null 2>&1`; then
echo "Found previous tag: ${SHA_PREV}"
else
SHA_PREV=`git rev-list --max-parents=0 HEAD`
echo "Using initial commit: ${SHA_PREV}"
fi
changes=`git diff --name-only ${SHA_PREV} HEAD | grep ${CHECK_PATH}/`
if [ ! -n "${changes}" ]; then
echo "No changes found"
exit 2 # no changes found
fi
Related
I am rewriting my pipline in node, I need to understand how to perform a step with a gait in node now an error is coming from stage('Deploy')
node {
checkout scm
def customImage = docker.build("python-web-tests:${env.BUILD_ID}")
customImage.inside {
sh "python ${env.CMD_PARAMS}"
}
stage('Deploy') {
post {
always {
allure([
includeProperties: false,
jdk: '',
properties: [],
reportBuildPolicy: 'ALWAYS',
results: [[path: 'report']]
])
cleanWs()
}
}
}
and this is the old pipeline
pipeline {
agent {label "slave_first"}
stages {
stage("Создание контейнера image") {
steps {
catchError {
script {
docker.build("python-web-tests:${env.BUILD_ID}", "-f Dockerfile .")
}
}
}
}
stage("Running and debugging the test") {
steps {
sh 'ls'
sh 'docker run --rm -e REGION=${REGION} -e DATA=${DATA} -e BUILD_DESCRIPTION=${BUILD_URL} -v ${WORKSPACE}:/tmp python-web-tests:${BUILD_ID} /bin/bash -c "python ${CMD_PARAMS} || exit_code=$?; chmod -R 777 /tmp; exit $exit_code"'
}
}
}
post {
always {
allure([
includeProperties: false,
jdk: '',
properties: [],
reportBuildPolicy: 'ALWAYS',
results: [[path: 'report']]
])
cleanWs()
}
}
}
I tried to transfer the method of creating an allure report, but nothing worked, I use the version above, almost everything turned out, you can still add environment variables to the build, for example, those that are specified -e DATA=${DATA} how do I add it
I don't recommend to switch from declarative to scriptive pipeline.
You are losing possibility to use multiple tooling connected with declarative approach like syntax checkers.
If you still want to use scriptive approach try this:
node('slave_first') {
stage('Build') {
checkout scm
def customImage = docker.build("python-web-tests:${env.BUILD_ID}")
customImage.inside {
sh "python ${env.CMD_PARAMS}"
}
}
stage('Deploy') {
allure([
includeProperties: false,
jdk: '',
properties: [],
reportBuildPolicy: 'ALWAYS',
results: [[path: 'report']]])
cleanWs()
}
}
There is no post and always directive in scriptive pipelines. It's on your head to catch all exceptions and set status of the job. I guess you were using this page: https://www.jenkins.io/doc/book/pipeline/syntax/, but it's a mistake.
This page only refers to declarative approach and in few cases you have hidden scriptive code as examples.
Also i don't know if you have default agent label set in your Jenkins config, but by looking at your declarative one I think you missed 'slave_first' arg in node object.
those that are specified -e DATA=${DATA} how do I add it
That's a docker question not a Jenkins. If you want to launch docker image and then also have access to some reports located in this container you should mount workspace/file where those output files landed. You should also pass location of those files to allure.
I suggest you to try this:
mount some subfolder in workspace to docker container
cat test report file if it's visible
add allure report with passing this file location to allure step
I am using latest Jenkins in my Linux box.
I would like to implement a Test stage like below:
pipeline {
agent any
stages {
stage('Test') {
HTTP_CODE = sh (
script: 'echo $(curl --write-out \\"%{http_code}\\" --silent --output /dev/null http://localhost/)',
returnStdout: true
).trim()
}
}
}
In this stage, I want to execute a bash script to check whether the web server is up or not. HTTP_CODE variable will have the value 200 if everything is fine, and if there is any other value, then it can be treated as error.
How can I implement this logic as a testing stage in my Jenkins pipeline?
Thanks.
You should update your pipeline as follow:
pipeline {
agent any
stages {
stage('Test') {
HTTP_CODE = sh (
script: 'echo $(curl --write-out \\"%{http_code}\\" --silent --output /dev/null http://localhost/)',
returnStdout: true
).trim()
if ('200' != HTTP_CODE) {
currentBuild.result = "FAILURE"
error('Test stage failed!)
}
}
}
}
Regards.
I am creating a declarative pipeline in Jenkins. There are 6 stages in it.
First Stage: Scenario Upload
Second Stage: Pull code from Git
Third Stage: Maven Build
Fourth Stage: Its a parallel stage. First step will launch mobile emulator and second step will check device connected or not.
Fifth Stage: I want to start this stage when the second step BUILD SUCCESS else stop the job
Sixth Stage: Send email
I am stuck with point 5 (Fifth Stage). Please help
pipeline {
agent any
stages {
stage("Scenario Upload") {
steps {
script {
def inputFile = input message: 'Upload file', parameters: [file(name: 'CyclosAppStatus.xlsx')]
new hudson.FilePath(new File("$workspace/Cucumber_BDD master/Result/CyclosAppStatus.xlsx")).copyFrom(inputFile)
inputFile.delete()
}
}
}
stage('Git Pull Code') {
steps {
git credentialsId: '708a126a-66bb-4eb5-8826-55cedf6497c3', url: 'https://github.com/divakar-ragupathy/Mobile_Automation_BDD.git'
}
}
stage('Maven Clean Build') {
steps {
bat label: '', script: '''Echo Maven Clean Build...
cd %WORKSPACE%\\ADB_Devices
mvn clean compile'''
}
}
stage('Building Android Setup') {
steps {
parallel(
Invoke_Emulator: {
bat label: '', script: '''Echo Invoking Emulator...
#echo off
set emulName=%Emulator_Name%
echo %emulName%
for /f "tokens=1 delims=:" %%e in ("%emulName%") do (
%ANDROID_AVD_PATH%emulator -avd "%%e" -no-boot-anim -no-snapshot-save -no-snapshot-load
)
endlocal'''
},
Checking_Device: {
bat label: '', script: '''Echo Checking Connected Device...
cd %WORKSPACE%\\ADB_Devices
mvn exec:java -Dexec.mainClass=com.expleo.adbListner.CheckConnectedAdbDevices -Dlog4j.configuration=file:///%WORKSPACE%\\ADB_Devices\\src\\log4j.properties -Dexec.args="%Emulator_Name%"'''
}
)
}
}
}
}
If you declare a variable without the "def" keyword it is global. You can use that to store the condition in the previous stages. In the 5th stage you can use a when block to check this condition.
I have made a Jenkins pipeline for my Angular application.
This Angular application uses SSR, so you have to run it in the background.
Thus I decided to use a screen for this, here is my JenkinsFile:
pipeline {
agent any
environment {
HOME = '.'
}
stages {
stage('build') {
steps {
sh 'npm i'
sh 'npm run build:ssr'
}
}
stage('move') {
steps {
script {
if (BRANCH_NAME == 'master') {
sh 'rm -R /var/www/AngularJenkinsTest_Master/client || true'
sh 'mkdir /var/www/AngularJenkinsTest_Master/client || true'
sh 'cp -R $WORKSPACE/dist/AngularJenkinsTest/* /var/www/AngularJenkinsTest_Master/client'
}
}
}
}
stage('publish') {
steps {
script {
if (BRANCH_NAME == 'master') {
sh 'screen -X -S AngularJenkinsTest_Master kill | true'
sh 'screen -dmS AngularJenkinsTest_Master'
sh 'screen -S AngularJenkinsTest_Master -X stuff "node /var/www/AngularJenkinsTest_Master/client/server/main.js\n"'
}
}
}
}
}
}
As you can see in "publish", I kill the screen (as it is running a node application), I then create a new screen and send a command to it.
I added a screen -ls to the end of it, and it did show that it existed, but when I go to my linux console.
jenkins#server:/root$ screen -ls
No Sockets found in /run/screen/S-jenkins.
This is the output that jenkins gives me:
Jenkins Output
I am new to Jenkins, so maybe I am just being dumb, but is there any reason for this to happen?
When i am passing value of a variable declared in jenkins Groovy script its value is not retained in for loop which is running on a remote server. Strange thing is i am able to access the same value outside the for loop.
Here is the sample code i am trying to use
#!/usr/bin/env groovy
def config
def COMMANDS_TO_CHECK='curl grep hello awk tr mkdir bc'
pipeline {
agent {
label "master"
}
stages {
stage ('Validation of commands') {
steps {
script {
sh """
#!/bin/bash
/usr/bin/sshpass -p passwrd ssh user#host << EOF
hostname
echo $COMMANDS_TO_CHECK ---> This is printed
for CURRENT_COMMAND in \$COMMANDS_TO_CHECK
do
echo ${CURRENT_COMMAND} ---> Why This is not printed?
echo \${CURRENT_COMMAND} ----> Why This is not printed?
done
hostname
EOF
exit
"""
}
}
}
}
}
Output
[workspace#3] Running shell script
+ /usr/bin/sshpass -p passwrd ssh user#host
Pseudo-terminal will not be allocated because stdin is not a terminal.
illinsduck01
curl grep hello awk tr mkdir bc
illinsduck01
+ exit
You can wrap sh in """ ... """ as below
#!/usr/bin/env groovy
def config
pipeline {
agent {
label "master"
}
stages {
stage ('Validation of commands') {
steps {
script {
sh """#!/bin/sh
/usr/bin/sshpass -p password ssh username#hostname << EOF
COMMANDS_TO_CHECK="curl grep hello awk tr mkdir bc"
hostname
echo \$COMMANDS_TO_CHECK
for CURRENT_COMMAND in \$COMMANDS_TO_CHECK
do
echo \$CURRENT_COMMAND
which \$CURRENT_COMMAND
status=\$?
if [ \${status} -eq 0 ]
then
echo \${CURRENT_COMMAND} command is OK
else
echo "Failed to find the \${CURRENT_COMMAND} command"
fi
done
hostname
EOF
exit
"""
}
}
}
}
}