So I have a simple script running in an IF statement. I always get:
syntax error: unexpected end of file (expecting "fi")
I am wondering what could be the possible solution for this ?
def call(Map config) {
withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', credentialsId: 'JENKINS_AWS'],
sshUserPrivateKey(credentialsId: 'JENKINS-SSH', keyFileVariable: 'SSH_KEY')]) {
sh """
#!/bin/bash
source add_ssh_key "${SSH_KEY}"
source init_env "${TARGET_STAGE}"
source create-bastion-pod "${PROMETHEUS_PUSHGATEWAY}" "${PROMETHEUS_PUSHGATEWAY_PORT}"
if [ ${TARGET_STAGE} == 'dev' ]; then
cat <<-EOF | curl --data-binary #- \${BASTION_URL}/metrics/job/sone_job
# TYPE some_metric counter
some_metric{label="val1"} 42
EOF
fi
delete-bastion-pod
"""
}
}
<<- only strips tabs from the here-document; your closing delimiter appears to be indented (according to what Groovy actually presents to the shell) with a couple of spaces. Try something like
sh """
#!/bin/bash
if [ ${TARGET_STAGE} == 'dev' ]; then
cat <<EOF | curl --data-binary #- \${BASTION_URL}/metrics/job/some_job
# TYPE some_metric counter
some_metric{label="val1"} 42
EOF
fi
"""
Note that as far as the shell executing the script is concerned, the here-document and the closing EOF aren't indented at all.
Related
Is there a way to get the list of return values from python script in the bash script?
Tried:
test_python_return_list.sh:
#!/bin/bash
echo "check return list from python"
out=($(python3 ./test_return_list.py))
echo "return from python script: "
echo $(out)
test_return_list.py:
ret = [1, 2, 3]
exit(ret)
Test result:
$./test_python_return_list.sh
check return list from python
[1,2,3] --> why/where this printout is from?
return from python script:
---> echo $(out) print out nothing, how do I get the [1,2,3] in the bash script?
$
Thanks!
A few issues with the current code:
invalid shebang (missing leading /)
echo $(out) should be generating an error that out is an invalid command unless ... a) OP does in fact have a command named out or b) code displayed in question is different than OP's actual script; regardless, out is being populated as an array so need a different syntax to display
the python/exit(ret) command appears to send its ouput to stderr, but the out=(...) call is only capturing stdout
re: python/exit(ret) result:
$ python3 ./test_return_list.py > py.stdout 2>py.stderr
==> py.stderr <==
[1, 2, 3]
==> py.stdout <==
# empty file
Making a few changes to OP's current bash script:
$ cat test_python_return_list.sh
#!/bin/bash # fix shebang
echo "check return list from python"
out=( $(python3 ./test_return_list.py 2>&1) ) # redirect stderr to stdout
echo "return from python script: "
echo "${out[#]}" # echo contents of array out[]
typeset -p out # alternative display of out[] contents
Taking for a test drive:
$ ./test_python_return_list.sh
check return list from python
return from python script:
[1, 2, 3]
declare -a out=([0]="[1," [1]="2," [2]="3]")
NOTE: while this should address the syntax issues I can't speak to whether or not this is OP's expected contents of the out[] array ...
I'm writing a script to send SQL output in mail, but it is not executing successfully and is not generating the output I want.
The query generates two columns with multiple rows. How can I generate the output in table format as below?
Below is my code:
#!/bin/bash
ORACLE_HOME= **PATH
export ORACLE_HOME
PATH=$PATH:$ORACLE_HOME/bin
export PATH
TNS_ADMIN= ** PATH
export TNS_ADMIN
today=$(date +%d-%m-%Y)
output=$(sqlplus -S user/pass#service <<EOF
set heading off;
SELECT distinct list_name ,max(captured_dttm) as Last_received FROM db.table1
group by list_name having max(captured_dttm) <= trunc(sysdate - interval '2' hour);
EOF)
if [ -z "$output" ];
then
echo"its fine"
exit
else
echo "
Dear All,
Kindly check we've not received the list for last 2 hour : $output
Regards,
Team" | mailx -S smtp=XX.XX.X.XX:XX -s "URGENT! Please check list FOR $today" user#abc.com
fi
When using a here document, the closing string can't be followed by anything but a newline. Move the closing parenthesis to the next line:
output=$(sqlplus -S user/pass#service <<EOF
...
EOF
)
Below is my bash script.
#!/bin/bash
message=$1
hostname=$2
severity=$3
eventname=$4
tagpath=$5
appname=$6
data="{"action":"EventsRouter","method":"add_event","data":[{"summary":"$message"},"device":"$hostname","message": "$message $eventname $tagpath","component":"$appname","severity":"$severity","evclasskey":"nxlog","evclass":"/nxlog/perf","monitor":"localhost"}],"type":"rpc","tid":1}"
echo "Total number of args : $#"
echo "message = $message"
echo "hostname = $hostname"
echo "appname = $appname"
echo "data = $data"
curl -u uname:password -k https://myurlcom/zport/dmd/evconsole_router -d $data
and when i try to run with sh tcp.sh value value value value value value
host:
'host,component:host,severity:host,evclasskey:nxlog,evclass:/nxlog/perf,monitor:localhost}],type:rpc,tid:1}'
is not a legal name (unexpected end of input) Total number of args : 6
message = message hostname = test appname = host data = curl: option
-d: requires parameter
I see that data has no value included.
This json has to be sent in this order for it to be accepted in the endpoint. Help me correct this.
Using jq to safely generate the desired JSON:
#!/bin/bash
parameters=(
--arg message "$1"
--arg hostname "$2"
--arg severity "$3"
--arg eventname "$4"
--arg tagpath "$5"
--arg appname "$6"
)
data=$(jq -n "${parameters[#]}" '
{action: "EventsRouter",
method: "add_event",
data: [ {summary: $message,
device: $hostname,
message: "\($message) \($eventname\) \($tagpath)",
component: $appname,
severity: $severity,
evclasskey: "nxlog",
evclass: "/nxlog/perf",
monitor: "localhost"
}
],
type: "rpc",
tid: 1
}'
curl -u uname:password -k https://myurlcom/zport/dmd/evconsole_router -d "$data"
Assuming:
message="my_message"
hostname="my_host"
severity="my_severity"
eventname="my_event"
tagpath="my_path"
appname="my_app"
If you run:
data="{"action":"EventsRouter","method":"add_event","data":[{"summary":"$message"},"device":"$hostname","message": "$message $eventname $tagpath","component":"$appname","severity":"$severity","evclasskey":"nxlog","evclass":"/nxlog/perf","monitor":"localhost"}],"type":"rpc","tid":1}"
you will get an error, because there is a not escaped white space before the string "my_event"
my_event: command not found
What happened? Since your json input has a lot of words between double quotes, you will have to enclose the whole string into single quotes, in order to preserve the double quotes inside of the string. But between single quotes, the bash variables will not be replaced by their value. So you will need to close the single quotes before each variable and reopen these again immediately after.
So that line of your script must become:
data='{"action":"EventsRouter","method":"add_event","data":[{"summary":"'$message'"},"device":"'$hostname'","message": "'$message $eventname $tagpath'","component":"'$appname'","severity":"'$severity'","evclasskey":"nxlog","evclass":"/nxlog/perf","monitor":"localhost"}],"type":"rpc","tid":1}'
If you execute:
echo "$data"
you will get:
{"action":"EventsRouter","method":"add_event","data":[{"summary":"my_message"},"device":"my_host","message": "my_message my_event my_path","component":"my_app","severity":"my_severity","evclasskey":"nxlog","evclass":"/nxlog/perf","monitor":"localhost"}],"type":"rpc","tid":1}
which is correct, I assume: the double quotes didn't disappear from your json data structure.
I have something like this on a Jenkinsfile (Groovy) and I want to record the stdout and the exit code in a variable in order to use the information later.
sh "ls -l"
How can I do this, especially as it seems that you cannot really run any kind of groovy code inside the Jenkinsfile?
The latest version of the pipeline sh step allows you to do the following;
// Git committer email
GIT_COMMIT_EMAIL = sh (
script: 'git --no-pager show -s --format=\'%ae\'',
returnStdout: true
).trim()
echo "Git committer email: ${GIT_COMMIT_EMAIL}"
Another feature is the returnStatus option.
// Test commit message for flags
BUILD_FULL = sh (
script: "git log -1 --pretty=%B | grep '\\[jenkins-full]'",
returnStatus: true
) == 0
echo "Build full flag: ${BUILD_FULL}"
These options where added based on this issue.
See official documentation for the sh command.
For declarative pipelines (see comments), you need to wrap code into script step:
script {
GIT_COMMIT_EMAIL = sh (
script: 'git --no-pager show -s --format=\'%ae\'',
returnStdout: true
).trim()
echo "Git committer email: ${GIT_COMMIT_EMAIL}"
}
Current Pipeline version natively supports returnStdout and returnStatus, which make it possible to get output or status from sh/bat steps.
An example:
def ret = sh(script: 'uname', returnStdout: true)
println ret
An official documentation.
quick answer is this:
sh "ls -l > commandResult"
result = readFile('commandResult').trim()
I think there exist a feature request to be able to get the result of sh step, but as far as I know, currently there is no other option.
EDIT: JENKINS-26133
EDIT2: Not quite sure since what version, but sh/bat steps now can return the std output, simply:
def output = sh returnStdout: true, script: 'ls -l'
If you want to get the stdout AND know whether the command succeeded or not, just use returnStdout and wrap it in an exception handler:
scripted pipeline
try {
// Fails with non-zero exit if dir1 does not exist
def dir1 = sh(script:'ls -la dir1', returnStdout:true).trim()
} catch (Exception ex) {
println("Unable to read dir1: ${ex}")
}
output:
[Pipeline] sh
[Test-Pipeline] Running shell script
+ ls -la dir1
ls: cannot access dir1: No such file or directory
[Pipeline] echo
unable to read dir1: hudson.AbortException: script returned exit code 2
Unfortunately hudson.AbortException is missing any useful method to obtain that exit status, so if the actual value is required you'd need to parse it out of the message (ugh!)
Contrary to the Javadoc https://javadoc.jenkins-ci.org/hudson/AbortException.html the build is not failed when this exception is caught. It fails when it's not caught!
Update:
If you also want the STDERR output from the shell command, Jenkins unfortunately fails to properly support that common use-case. A 2017 ticket JENKINS-44930 is stuck in a state of opinionated ping-pong whilst making no progress towards a solution - please consider adding your upvote to it.
As to a solution now, there could be a couple of possible approaches:
a) Redirect STDERR to STDOUT 2>&1
- but it's then up to you to parse that out of the main output though, and you won't get the output if the command failed - because you're in the exception handler.
b) redirect STDERR to a temporary file (the name of which you prepare earlier) 2>filename (but remember to clean up the file afterwards) - ie. main code becomes:
def stderrfile = 'stderr.out'
try {
def dir1 = sh(script:"ls -la dir1 2>${stderrfile}", returnStdout:true).trim()
} catch (Exception ex) {
def errmsg = readFile(stderrfile)
println("Unable to read dir1: ${ex} - ${errmsg}")
}
c) Go the other way, set returnStatus=true instead, dispense with the exception handler and always capture output to a file, ie:
def outfile = 'stdout.out'
def status = sh(script:"ls -la dir1 >${outfile} 2>&1", returnStatus:true)
def output = readFile(outfile).trim()
if (status == 0) {
// output is directory listing from stdout
} else {
// output is error message from stderr
}
Caveat: the above code is Unix/Linux-specific - Windows requires completely different shell commands.
this is a sample case, which will make sense I believe!
node('master'){
stage('stage1'){
def commit = sh (returnStdout: true, script: '''echo hi
echo bye | grep -o "e"
date
echo lol''').split()
echo "${commit[-1]} "
}
}
For those who need to use the output in subsequent shell commands, rather than groovy, something like this example could be done:
stage('Show Files') {
environment {
MY_FILES = sh(script: 'cd mydir && ls -l', returnStdout: true)
}
steps {
sh '''
echo "$MY_FILES"
'''
}
}
I found the examples on code maven to be quite useful.
All the above method will work. but to use the var as env variable inside your code you need to export the var first.
script{
sh " 'shell command here' > command"
command_var = readFile('command').trim()
sh "export command_var=$command_var"
}
replace the shell command with the command of your choice. Now if you are using python code you can just specify os.getenv("command_var") that will return the output of the shell command executed previously.
How to read the shell variable in groovy / how to assign shell return value to groovy variable.
Requirement : Open a text file read the lines using shell and store the value in groovy and get the parameter for each line .
Here , is delimiter
Ex: releaseModule.txt
./APP_TSBASE/app/team/i-home/deployments/ip-cc.war/cs_workflowReport.jar,configurable-wf-report,94,23crb1,artifact
./APP_TSBASE/app/team/i-home/deployments/ip.war/cs_workflowReport.jar,configurable-temppweb-report,394,rvu3crb1,artifact
========================
Here want to get module name 2nd Parameter (configurable-wf-report) , build no 3rd Parameter (94), commit id 4th (23crb1)
def module = sh(script: """awk -F',' '{ print \$2 "," \$3 "," \$4 }' releaseModules.txt | sort -u """, returnStdout: true).trim()
echo module
List lines = module.split( '\n' ).findAll { !it.startsWith( ',' ) }
def buildid
def Modname
lines.each {
List det1 = it.split(',')
buildid=det1[1].trim()
Modname = det1[0].trim()
tag= det1[2].trim()
echo Modname
echo buildid
echo tag
}
If you don't have a single sh command but a block of sh commands, returnstdout wont work then.
I had a similar issue where I applied something which is not a clean way of doing this but eventually it worked and served the purpose.
Solution -
In the shell block , echo the value and add it into some file.
Outside the shell block and inside the script block , read this file ,trim it and assign it to any local/params/environment variable.
example -
steps {
script {
sh '''
echo $PATH>path.txt
// I am using '>' because I want to create a new file every time to get the newest value of PATH
'''
path = readFile(file: 'path.txt')
path = path.trim() //local groovy variable assignment
//One can assign these values to env and params as below -
env.PATH = path //if you want to assign it to env var
params.PATH = path //if you want to assign it to params var
}
}
Easiest way is use this way
my_var=`echo 2`
echo $my_var
output
: 2
note that is not simple single quote is back quote ( ` ).
I wrote simple script as follow
#!/bin/bash
auth_type=""
SM_Read-only="Yes"
SM_write-only="No"
echo -e ${SM_Read-only}
echo -e ${SM_Write-only}
if [ "${SM_Read-only}" == "Yes" ] && [ "${SM_Write-only}" == "Yes" ]
then
auth_type="Read Write"
else
auth_type="Read"
fi
echo -e $auth_type
And when i execute it i got following output with errors.
./script.bash: line 5: SM_Read-only=Yes: command not found
./script.bash: line 6: SM_write-only=No: command not found
only
only
Read
Any one know correct way to declare the variable with "-" (dash)?
EDIT:
have getting response from c code and evaluate the variables for example
RESP=`getValue SM_ Read-only ,Write-only 2>${ERR_DEV}`
RC=$?
eval "$RESP"
from above scripts code my c binary getValue know that script want Read-only and Write-only and return value to script.So during eval $RESP in cause error and in my script i access variable by
echo -e ${SM_Read-only}
echo -e ${SM_Write-only}
which also cause error.
Rename the variable name as follows:
SM_Read_only="Yes"
SM_write_only="No"
Please, don't use - minus sign in variable names in bash, please refer to the answer, on how to set the proper variable name in bash.
However if you generate the code, based on others output, you can simply process their output with sed:
RESP=$(getValue SM_ Read-rule,Write-rule 2>${ERR_DEV}|sed "s/-/_/g")
RC=$?
eval "$RESP"
- is not allowed in shell variable names. Only letters, numbers, and underscore, and the first character must be a letter or underscore.
I think you cant have a dash in your variables names, only letters, digits and "_"
Try:
SM_Read_only
Or
SM_ReadOnly