I have a long standing declarative pipeline infrastructure
I would like to start putting repeated code into shared libraries
The problem I am facing is calling the git plugin from a shared library function/class. I'm a bit lost as my experience is really only with Jenkins declarative stuff, not the Groovy/Java specifics.
Here is a snippet of the Jenkinsfile, (before using shared library):
pipeline {
agent any
stages {
stage('Prep Workspace') {
steps {
script {
if ((env.BRANCH_NAME == 'staging') || (env.BRANCH_NAME == 'production')) {
BRANCH=env.BRANCH_NAME
} else {
BRANCH='master'
}
}
echo "||------ Get ProjectOne Dependency ------||"
dir('deps/ProjectOne') {
git branch: "${BRANCH}",
changelog: false,
credentialsId: 'jenkinsgit',
poll: false,
url: 'git#github.com:myprivateorg/ProjectOne.git'
}
echo "||------ Get ProjectTwo Dependency ------||"
dir('deps/ProjectTwo') {
git branch: "${BRANCH}",
changelog: false,
credentialsId: 'jenkinsgit',
poll: false,
url: 'git#github.com:myprivateorg/ProjectTwo.git'
}
}
}
}
}
Note the repeated calls to pull down project files from git repos. The goal here, is to move the repeated code to a shared function call.
I've read the following portion in the manual, on how to use git in shared library:
https://www.jenkins.io/doc/book/pipeline/shared-libraries/#accessing-steps
Using the example in the documentation I've created the shared library file
In src/org/test/gitHelper.groovy:
package org.test;
def checkOutFrom(String repo, String branch='master') {
echo "||------ CLONING $repo ------||"
git branch: branch, changelog: false, credentialsId: 'jenkinsgit', poll: false, url: "git#github.com:myprivateorg/$repo.git"
}
return this
Then in the Jenkinsfile:
#Library('jenkins-shared-library') _
pipeline {
agent any
stages {
stage('Prep Workspace') {
steps {
script {
if ((env.BRANCH_NAME == 'staging') || (env.BRANCH_NAME == 'production')) {
BRANCH=env.BRANCH_NAME
} else {
BRANCH='master'
}
def g = new org.test.gitHelper()
g.checkOutFrom('ProjectOne')
g.checkOutFrom('ProjectTwo')
}
}
}
}
}
This loads the class and calls the function fine, but fails when it hits git itself:
groovy.lang.MissingPropertyException: No such property: git for class: java.lang.String
I used g.getClass() to confirm it's of type class org.test.gitHelper and NOT java.lang.String so I'm not sure where it's getting that type from.
Please note I have also tried this way:
vars/pullRepo.groovy
def call(String repo, String branch) {
echo "||------ CLONING $repo ------||"
dir("deps/$repo") {
git branch: branch, changelog: false, credentialsId: 'jenkinsgit', poll: false, url: "git#github.com:myprivateorg/$repo.git"
}
}
Jenkinsfile:
pullRepo('ProjectOne', 'master')
I get the exact same error: groovy.lang.MissingPropertyException: No such property: git for class: java.lang.String
For me, it works to pass the Jenkins context to the shared library like so:
Jenkinsfile:
pullRepo(this, repo, branch)
vars/pullRepo.groovy:
def call(def context, String repo, String branch) {
echo "||------ CLONING $repo ------||"
dir("deps/$repo") {
context.git branch: branch, changelog: false, credentialsId: 'jenkinsgit', poll: false, url: "git#github.com:myprivateorg/$repo.git"
}
}
Note that I'm passing the Jenkins context into the context variable, and calling git as a method of the context. You should also be able to do this by passing the context up to your class.
Related
I am rewriting my pipline in node, I need to understand how to perform a step with a gait in node now an error is coming from stage('Deploy')
node {
checkout scm
def customImage = docker.build("python-web-tests:${env.BUILD_ID}")
customImage.inside {
sh "python ${env.CMD_PARAMS}"
}
stage('Deploy') {
post {
always {
allure([
includeProperties: false,
jdk: '',
properties: [],
reportBuildPolicy: 'ALWAYS',
results: [[path: 'report']]
])
cleanWs()
}
}
}
and this is the old pipeline
pipeline {
agent {label "slave_first"}
stages {
stage("Создание контейнера image") {
steps {
catchError {
script {
docker.build("python-web-tests:${env.BUILD_ID}", "-f Dockerfile .")
}
}
}
}
stage("Running and debugging the test") {
steps {
sh 'ls'
sh 'docker run --rm -e REGION=${REGION} -e DATA=${DATA} -e BUILD_DESCRIPTION=${BUILD_URL} -v ${WORKSPACE}:/tmp python-web-tests:${BUILD_ID} /bin/bash -c "python ${CMD_PARAMS} || exit_code=$?; chmod -R 777 /tmp; exit $exit_code"'
}
}
}
post {
always {
allure([
includeProperties: false,
jdk: '',
properties: [],
reportBuildPolicy: 'ALWAYS',
results: [[path: 'report']]
])
cleanWs()
}
}
}
I tried to transfer the method of creating an allure report, but nothing worked, I use the version above, almost everything turned out, you can still add environment variables to the build, for example, those that are specified -e DATA=${DATA} how do I add it
I don't recommend to switch from declarative to scriptive pipeline.
You are losing possibility to use multiple tooling connected with declarative approach like syntax checkers.
If you still want to use scriptive approach try this:
node('slave_first') {
stage('Build') {
checkout scm
def customImage = docker.build("python-web-tests:${env.BUILD_ID}")
customImage.inside {
sh "python ${env.CMD_PARAMS}"
}
}
stage('Deploy') {
allure([
includeProperties: false,
jdk: '',
properties: [],
reportBuildPolicy: 'ALWAYS',
results: [[path: 'report']]])
cleanWs()
}
}
There is no post and always directive in scriptive pipelines. It's on your head to catch all exceptions and set status of the job. I guess you were using this page: https://www.jenkins.io/doc/book/pipeline/syntax/, but it's a mistake.
This page only refers to declarative approach and in few cases you have hidden scriptive code as examples.
Also i don't know if you have default agent label set in your Jenkins config, but by looking at your declarative one I think you missed 'slave_first' arg in node object.
those that are specified -e DATA=${DATA} how do I add it
That's a docker question not a Jenkins. If you want to launch docker image and then also have access to some reports located in this container you should mount workspace/file where those output files landed. You should also pass location of those files to allure.
I suggest you to try this:
mount some subfolder in workspace to docker container
cat test report file if it's visible
add allure report with passing this file location to allure step
I am creating a declarative pipeline in Jenkins. There are 6 stages in it.
First Stage: Scenario Upload
Second Stage: Pull code from Git
Third Stage: Maven Build
Fourth Stage: Its a parallel stage. First step will launch mobile emulator and second step will check device connected or not.
Fifth Stage: I want to start this stage when the second step BUILD SUCCESS else stop the job
Sixth Stage: Send email
I am stuck with point 5 (Fifth Stage). Please help
pipeline {
agent any
stages {
stage("Scenario Upload") {
steps {
script {
def inputFile = input message: 'Upload file', parameters: [file(name: 'CyclosAppStatus.xlsx')]
new hudson.FilePath(new File("$workspace/Cucumber_BDD master/Result/CyclosAppStatus.xlsx")).copyFrom(inputFile)
inputFile.delete()
}
}
}
stage('Git Pull Code') {
steps {
git credentialsId: '708a126a-66bb-4eb5-8826-55cedf6497c3', url: 'https://github.com/divakar-ragupathy/Mobile_Automation_BDD.git'
}
}
stage('Maven Clean Build') {
steps {
bat label: '', script: '''Echo Maven Clean Build...
cd %WORKSPACE%\\ADB_Devices
mvn clean compile'''
}
}
stage('Building Android Setup') {
steps {
parallel(
Invoke_Emulator: {
bat label: '', script: '''Echo Invoking Emulator...
#echo off
set emulName=%Emulator_Name%
echo %emulName%
for /f "tokens=1 delims=:" %%e in ("%emulName%") do (
%ANDROID_AVD_PATH%emulator -avd "%%e" -no-boot-anim -no-snapshot-save -no-snapshot-load
)
endlocal'''
},
Checking_Device: {
bat label: '', script: '''Echo Checking Connected Device...
cd %WORKSPACE%\\ADB_Devices
mvn exec:java -Dexec.mainClass=com.expleo.adbListner.CheckConnectedAdbDevices -Dlog4j.configuration=file:///%WORKSPACE%\\ADB_Devices\\src\\log4j.properties -Dexec.args="%Emulator_Name%"'''
}
)
}
}
}
}
If you declare a variable without the "def" keyword it is global. You can use that to store the condition in the previous stages. In the 5th stage you can use a when block to check this condition.
Looking help on Jenkins, i have written jenkinsfile where in one stage i am running sh git submodule update --init --recursive command my job is getting failed with Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password) error. but i can see on log common-ui-layout directory is present with full access, as per my analysis i found that while hitting the submodule command its not getting authenticated or its not finding the common-ui-layout folder. Im posting my jenkinsfile here, Please provide the fix of this issue.
pipeline {
agent {
label 'agent.com'
}
stages {stage("submodule clone"){
steps
{
checkout(
[
$class: 'GitSCM',
branches: [
[
name: 'master'
]
],
doGenerateSubmoduleConfigurations: false,
extensions: [
[
$class: 'SubmoduleOption',
disableSubmodules: false,
parentCredentials: true,
recursiveSubmodules: true,
reference: '',
trackingSubmodules: false
]
],
submoduleCfg: [],
userRemoteConfigs: [
[
credentialsId: '<***ID****>',
url: 'https://*****gitlab.com/****/common-ui-layout.git'
]
]
]
)
}
}
stage("fetch data"){
steps {
git branch: 'patch-1',
credentialsId: '<***ID****>',
url: 'https://****.gitlab.com/*****/****.git'
sh "pwd"
sh "ls -lat"
}
}
stage ("Installing pre-req"){
steps{
sh '''
yarn install;
yarn global add #angular/cli
'''
}
}
stage('Build app') {
steps {
sh "yarn install";
sh "pwd";
sh 'git submodule update --init --recursive';
//sh "git submodule update --recursive –remote";
sh "yarn run ng build";
println "BUILD NUMBER = $BUILD_NUMBER"
println "Build Success.."
}
}
}
}
Please refer the error snipt here
The above error got fixed after adding the rsa key. after that i got another error when i hit the git submodule update --init --recursive command in pipeline and return the with below error-
Cloning into 'common-ui-layout'...
fatal: could not read Username for 'https://xxx.xxx.com': No such device or address
please refer the snippet here
Please suggest me where i am missing?
I want to publish an artifact to Nexus 3 in the Gradle project.
Due to requirements, publish from Jenkins instead of gradle publish using build.gradle maven-publish plug-in.
I created and executed a JenkinsPipeline script, but the artifact jar and pom have been separated.Why?
The following site is referred to this and this.
https://plugins.jenkins.io/nexus-artifact-uploader
And I make JenkinsPipeline script.
floowing pipeline script.
pipeline {
...
stage("publish to nexus") {
steps {
script {
pom = readMavenPom file: "build/pom.xml";
artifactPath = "build/libs/gs-managing-transactions-0.1.0.jar"
artifactExists = fileExists artifactPath;
if(artifactExists) {
nexusArtifactUploader(
nexusVersion: NEXUS_VERSION,
protocol: NEXUS_PROTOCOL,
nexusUrl: NEXUS_URL,
groupId: pom.groupId,
version: pom.version,
repository: NEXUS_REPOSITORY,
credentialsId: NEXUS_CREDENTIAL_ID,
artifacts: [
[artifactId: pom.artifactId, classifier: '', file: artifactPath , type: pom.packaging],
[artifactId: pom.artifactId, classifier: '', file: "build/pom.xml", type: "pom"]
]
);
} else {
error "*** File: ${artifactPath}, could not be found";
}
}
}
}
...
}
part of execute log
[Pipeline] nexusArtifactUploader
...
Uploading: http://localhost:7777/repository/maven-snapshots/com/sample/sample-spring-managing-transactions/0.0.1-SNAPSHOT/sample-spring-managing-transactions-0.0.1-20190621.123700-7-debug.jar
...
Uploading artifact gs-managing-transactions-0.1.0.jar completed.
...
Uploading: http://localhost:7777/repository/maven-snapshots/com/sample/sample-spring-managing-transactions/0.0.1-SNAPSHOT/sample-spring-managing-transactions-0.0.1-20190621.123701-8-debug.pom
Uploading artifact pom.xml completed.
Why stored separete artifact like "sample-spring-managing-transactions-0.0.1-20190621.123700-7-debug.jar", "sample-spring-managing-transactions-0.0.1-20190621.123701-8-debug.pom".
I will expect following.
"sample-spring-managing-transactions-0.0.1-20190621.123700-7-debug.jar" "sample-spring-managing-transactions-0.0.1-20190621.123701-7-debug.pom"
solved. I missed the description "Uploading maven artifacts snapshots is not supported by this plugin." plugins.jenkins.io/nexus-artifact-uploader.
I am trying to find an example of using the Jenkins Copy Artifacts Plugin from within Jenkins pipelines (workflows).
Can anyone point to a sample Groovy code that is using it?
With a declarative Jenkinsfile, you can use following pipeline:
pipeline {
agent any
stages {
stage ('push artifact') {
steps {
sh 'mkdir archive'
sh 'echo test > archive/test.txt'
zip zipFile: 'test.zip', archive: false, dir: 'archive'
archiveArtifacts artifacts: 'test.zip', fingerprint: true
}
}
stage('pull artifact') {
steps {
copyArtifacts filter: 'test.zip', fingerprintArtifacts: true, projectName: env.JOB_NAME, selector: specific(env.BUILD_NUMBER)
unzip zipFile: 'test.zip', dir: './archive_new'
sh 'cat archive_new/test.txt'
}
}
}
}
Before version 1.39 of the CopyArtifact, you must replace second stage with following (thanks #Yeroc) :
stage('pull artifact') {
steps {
step([ $class: 'CopyArtifact',
filter: 'test.zip',
fingerprintArtifacts: true,
projectName: '${JOB_NAME}',
selector: [$class: 'SpecificBuildSelector', buildNumber: '${BUILD_NUMBER}']
])
unzip zipFile: 'test.zip', dir: './archive_new'
sh 'cat archive_new/test.txt'
}
}
With CopyArtifact, I use '${JOB_NAME}' as project name which is the current running project.
Default selector used by CopyArtifact use last successful project build number, never current one (because it's not yet successful, or not). With SpecificBuildSelector you can choose '${BUILD_NUMBER}' which contains current running project build number.
This pipeline works with parallel stages and can manage huge files (I'm using a 300Mb file, it not works with stash/unstash)
This pipeline works perfectly with my Jenkins 2.74, provided you have all needed plugins
If you are using agents in your controller and you want to copy artifacts between each other you can use stash/unstash, for example:
stage 'build'
node{
git 'https://github.com/cloudbees/todo-api.git'
stash includes: 'pom.xml', name: 'pom'
}
stage name: 'test', concurrency: 3
node {
unstash 'pom'
sh 'cat pom.xml'
}
You can see this example in this link:
https://dzone.com/refcardz/continuous-delivery-with-jenkins-workflow
If builds are not running in the same pipeline you can use direct CopyArtifact plugin, here is example: https://www.cloudbees.com/blog/copying-artifacts-between-builds-jenkins-workflow and example code:
node {
// setup env..
// copy the deployment unit from another Job...
step ([$class: 'CopyArtifact',
projectName: 'webapp_build',
filter: 'target/orders.war']);
// deploy 'target/orders.war' to an app host
}
name = "/" + "${env.JOB_NAME}"
def archiveName = 'relNum'
try {
step($class: 'hudson.plugins.copyartifact.CopyArtifact', projectName: name, filter: archiveName)
} catch (none) {
echo 'No artifact to copy from ' + name + ' with name relNum'
writeFile file: archiveName, text: '3'
}
def content = readFile(archiveName).trim()
echo 'value archived: ' + content
try that using copy artifact plugin