Using:
Groovy Version: 3.0.8 JVM: 11.0.10 Vendor: Oracle Corporation OS: Linux
I have this script:
def shellCommand(String cmd) {
def process = cmd.execute()
def output = new StringWriter(), error = new StringWriter()
process.waitForProcessOutput(output, error)
println "exit value=${process.exitValue()}"
println "OUT: $output"
println "ERR: $error"
}
def gitRelease() {
def cmd001 = "git tag -a -m \"Release 0.0.777\" 0.0.45"
shellCommand(cmd001)
}
gitRelease()
When I run it from command line I get below error:
$ groovy myScript.groovy
exit value=128
OUT:
ERR: fatal: Failed to resolve '0.0.45' as a valid ref.
Same error if I try with slashy string:
def cmd001 = /git tag -a -m "Release 0.0.777" 0.0.45/
If I instead run git directly it works:
$ git tag -a -m "Release 0.0.777" 0.0.45
$ git tag
0.0.45
Creating a simple tag from the above groovy script works:
def gitRelease() {
//def cmd001 = "git tag -a -m \"Release 0.0.777\" 0.0.45"
def cmd001 = "git tag 0.0.46"
shellCommand(cmd001)
}
gives:
$ groovy myScript.groovy
exit value=0
OUT:
ERR:
Any suggestions?
The String.execute method often gives issues in unexpected places
There's another List.execute method that gives much more expected results
def cmd1 = ["git", "tag", "-a", "-m", "Release 0.0.777", "0.0.45"]
You should also change
def shellCommand(String cmd) {
To
def shellCommand(List cmd) {
Related
I'm very new to working with jenkins, so far I was able to run simple pipeline with simple pip install, but I need to pass global credentials from Jenkins into python script test.py invoked by jenkinsfile.
pipeline {
options { timeout(time: 30, unit: 'MINUTES')
buildDiscarder(logRotator(numToKeepStr: '30', artifactNumToKeepStr: '30')) }
agent { label 'ops_slave' }
stages {
stage('Environment Build') {
steps {
echo "Hello World!"
sh "echo Hello from the shell"
sh "hostname"
sh "uptime"
sh "python3 -m venv test_env"
sh "source ./test_env/bin/activate"
sh "pip3 install pandas psycopg2"
sh """echo the script is working"""
withCredentials([[
$class: 'UsernamePasswordMultiBinding',
credentialsId: 98,
usernameVariable: 'user',
passwordVariable: 'pw',
]])
sh """python3 bartek-jenkins-testing/python/test.py"""
}
}
}
}
I've seen implementation where you use argparse, but it's way above my level at this point, and I believe there is a way to reference it from python script or jenkins directly to pass to python script. I was googling for some time now, but I'm not sure questions I'm asking are correct.
My python script should be able to get username and password from Jenkins global credentials ID98:
print('Hello World this is python')
import pandas as pd
print(pd.__version__)
import pyodbc
import psycopg2
# can pass environemtn variables
connection = psycopg2.connect(
host="saturn-dv",
database="saturn_dv",
port='8080',
user='saturn_user_bartek_malysz',
password='')
connection.set_session(readonly=True)
query = """
SELECT table_name FROM information_schema.tables
WHERE table_schema = 'public'
ORDER BY table_schema,table_name;"""
data = pd.read_sql(query, connection)
print(data)
A straightforward way is to leverage environment variable as following
// Jenkinsfile
withCredentials([[
$class: 'UsernamePasswordMultiBinding',
credentialsId: 98,
usernameVariable: 'user',
passwordVariable: 'pw',
]]) {
sh """
export DB_USERNAME="${user}"
export DB_PASSWORD="${pw}"
python3 bartek-jenkins-testing/python/test.py
"""
}
// test.py
connection = psycopg2.connect(
host="saturn-dv",
database="saturn_dv",
port='8080',
user=os.getenv('DB_USERNAME'),
password=os.getenv('DB_PASSWORD'))
When i am passing value of a variable declared in jenkins Groovy script its value is not retained in for loop which is running on a remote server. Strange thing is i am able to access the same value outside the for loop.
Here is the sample code i am trying to use
#!/usr/bin/env groovy
def config
def COMMANDS_TO_CHECK='curl grep hello awk tr mkdir bc'
pipeline {
agent {
label "master"
}
stages {
stage ('Validation of commands') {
steps {
script {
sh """
#!/bin/bash
/usr/bin/sshpass -p passwrd ssh user#host << EOF
hostname
echo $COMMANDS_TO_CHECK ---> This is printed
for CURRENT_COMMAND in \$COMMANDS_TO_CHECK
do
echo ${CURRENT_COMMAND} ---> Why This is not printed?
echo \${CURRENT_COMMAND} ----> Why This is not printed?
done
hostname
EOF
exit
"""
}
}
}
}
}
Output
[workspace#3] Running shell script
+ /usr/bin/sshpass -p passwrd ssh user#host
Pseudo-terminal will not be allocated because stdin is not a terminal.
illinsduck01
curl grep hello awk tr mkdir bc
illinsduck01
+ exit
You can wrap sh in """ ... """ as below
#!/usr/bin/env groovy
def config
pipeline {
agent {
label "master"
}
stages {
stage ('Validation of commands') {
steps {
script {
sh """#!/bin/sh
/usr/bin/sshpass -p password ssh username#hostname << EOF
COMMANDS_TO_CHECK="curl grep hello awk tr mkdir bc"
hostname
echo \$COMMANDS_TO_CHECK
for CURRENT_COMMAND in \$COMMANDS_TO_CHECK
do
echo \$CURRENT_COMMAND
which \$CURRENT_COMMAND
status=\$?
if [ \${status} -eq 0 ]
then
echo \${CURRENT_COMMAND} command is OK
else
echo "Failed to find the \${CURRENT_COMMAND} command"
fi
done
hostname
EOF
exit
"""
}
}
}
}
}
I wrote some functions to help me writing Jenkins pipelines.
The following functions, are responsible to returning a shell command output:
def gen_uuid(){
randomUUID() as String
}
def sh_out(cmd){
String uuid = gen_uuid()
sh """( ${cmd} )> ${uuid}"""
String out = readFile(uuid).trim()
sh "set +x ; rm ${uuid}"
return out
}
That shown, here's another function:
Map get_started_by(){
withCredentials([ // Use Jenkins credentials ID of artifactory
[$class: 'UsernamePasswordMultiBinding', credentialsId: '0b8d591a-f4ce-XXXX-XXXX-faecb504d3d0', usernameVariable: 'J_USER', passwordVariable: 'J_PASS'],
]){
List startedBy = sh_out("""
set +x; curl -u ${J_USER}:${J_PASS} '${env.BUILD_URL}api/json' 2>/dev/null | \
python -mjson.tool | \
awk -F'"' '/(userId|userName)/{print \$4}'
""").split(/(\n)/)
return [
userId: startedBy[0],
userName: startedBy[1]
]
}
}
Which returns the userId and userName of the user who issue the job run.
Then, my problem is in this function:
def run_in_stage_func(String stage_name, Closure command, String sendTo){
String started_by = get_started_by()
String ulink = "<#${started_by['userId']}>"
String jlink = "(<${env.BUILD_URL}|Open>)"
println "============================================================"
stage (stage_name) {
try {
command()
if (currentBuild.result == 'FAILURE') {
error "Build failed, see log for further details."
}
println "============================================================"
} catch (Exception ex) {
def except = "${ex}"
slackSend channel: channel, color: 'danger', teamDomain: null, token: null,
message: " :${ulink} *Failed to build ${env.JOB_NAME}*! :x: ${jlink} (<!here|here>)"
echo "Pipeline failed at stage: ${stage_name}"
throw ex
}
}
}
When I run the job, I get the following error:
groovy.lang.MissingPropertyException: No such property: userId for class: java.lang.String
What could be the reason that the line ' String ulink = "<#${started_by['userId']}>" ' - is not working as intended?
You cast the result of get_started_by() to String explicitly. To fix it, change your code to
def started_by = get_started_by()
or
Map started_by = get_started_by()
The following groovy scripts fail using command line
#Grab("org.apache.poi:poi:3.9")
println "test"
Error:
unexpected token: println # line 2, column 1.
println "test"
^
1 error
Removing the Grab, it works!
Anything I missed?
$>groovy -v
Groovy Version: 2.1.7 JVM: 1.7.0_25 Vendor: Oracle Corporation OS: Linux
Annotations can only be applied to certain targets. See SO: Why can't I do a method call after a #Grab declaration in a Groovy script?
#Grab("org.apache.poi:poi:3.9")
dummy = null
println "test"
Alternatively you can use grab as a method call:
import static groovy.grape.Grape.grab
grab(group: "org.apache.poi", module: "poi", version: "3.9")
println "test"
For more information refer to Groovy Language Documentation > Dependency management with Grape.
File 'Grabber.groovy'
package org.taste
import groovy.grape.Grape
//List<List[]> artifacts => [[<group>,<module>,<version>,[<Maven-URL>]],..]
static def grab (List<List[]> artifacts) {
ClassLoader classLoader = new groovy.lang.GroovyClassLoader()
def eal = Grape.getEnableAutoDownload()
artifacts.each { artifact -> {
Map param = [
classLoader: classLoader,
group : artifact.get(0),
module : artifact.get(1),
version : artifact.get(2),
classifier : (artifact.size() < 4) ? null : artifact.get(3)
]
println param
Grape.grab(param)
}
}
Grape.setEnableAutoDownload(eal)
}
Usage :
package org.taste
import org.taste.Grabber
Grabber.grab([
[ "org.codehaus.groovy.modules.http-builder", "http-builder", '0.7.1'],
[ "org.postgresql", "postgresql", '42.3.1', null ],
[ "com.oracle.database.jdbc", "ojdbc8", '12.2.0.1', null]
])
my question is :
when i pass the script with :
groovy MyScript.groovy -o mtest -f filetest
the script can get the -o option.
but when i change the place of the option.
groovy MyScript.groovy -f filetest -o mtest
it can't get the option of -o
why?
do i miss something?
the groovy code is:
def cli = new CliBuilder()
cli.with {
usage: 'Self'
h longOpt:'help', 'U should input a analyze script with -o dataFileName!'
o longOpt:'output', 'The file which should be analyzed.', args:1, required:true
f longOpt:'file', 'File'
}
def opt = cli.parse(args)
def action
if( args.length == 0) {
cli.usage()
return
}
if( opt.h ) {
cli.usage()
return
}
println(args);
println(opt);
println(opt.o);
groovy MyScript.groovy -f filetest -o mtest
print result is :
[-f, filetest,-o,mtest]
groovy.util.OptionAccessor#66b51404
false
groovy MyScript.groovy -o mtest -f filetest
print result is :
[-o,mtest,-f, filetest]
groovy.util.OptionAccessor#66b51404
mtest
Think you need to specify args on your -f option as well (as it takes an argument) ie:
def cli = new CliBuilder().with {
usage: 'Self'
h longOpt:'help', 'U should input a analyze script with -o dataFileName!'
o longOpt:'output', 'The file which should be analyzed.', args:1, required:true
f longOpt:'file', 'File', args:1
it
}
def opt = cli.parse( args )
if( opt ) {
println args
println opt
println opt.o
}