I have something like this on a Jenkinsfile (Groovy) and I want to record the stdout and the exit code in a variable in order to use the information later.
sh "ls -l"
How can I do this, especially as it seems that you cannot really run any kind of groovy code inside the Jenkinsfile?
The latest version of the pipeline sh step allows you to do the following;
// Git committer email
GIT_COMMIT_EMAIL = sh (
script: 'git --no-pager show -s --format=\'%ae\'',
returnStdout: true
).trim()
echo "Git committer email: ${GIT_COMMIT_EMAIL}"
Another feature is the returnStatus option.
// Test commit message for flags
BUILD_FULL = sh (
script: "git log -1 --pretty=%B | grep '\\[jenkins-full]'",
returnStatus: true
) == 0
echo "Build full flag: ${BUILD_FULL}"
These options where added based on this issue.
See official documentation for the sh command.
For declarative pipelines (see comments), you need to wrap code into script step:
script {
GIT_COMMIT_EMAIL = sh (
script: 'git --no-pager show -s --format=\'%ae\'',
returnStdout: true
).trim()
echo "Git committer email: ${GIT_COMMIT_EMAIL}"
}
Current Pipeline version natively supports returnStdout and returnStatus, which make it possible to get output or status from sh/bat steps.
An example:
def ret = sh(script: 'uname', returnStdout: true)
println ret
An official documentation.
quick answer is this:
sh "ls -l > commandResult"
result = readFile('commandResult').trim()
I think there exist a feature request to be able to get the result of sh step, but as far as I know, currently there is no other option.
EDIT: JENKINS-26133
EDIT2: Not quite sure since what version, but sh/bat steps now can return the std output, simply:
def output = sh returnStdout: true, script: 'ls -l'
If you want to get the stdout AND know whether the command succeeded or not, just use returnStdout and wrap it in an exception handler:
scripted pipeline
try {
// Fails with non-zero exit if dir1 does not exist
def dir1 = sh(script:'ls -la dir1', returnStdout:true).trim()
} catch (Exception ex) {
println("Unable to read dir1: ${ex}")
}
output:
[Pipeline] sh
[Test-Pipeline] Running shell script
+ ls -la dir1
ls: cannot access dir1: No such file or directory
[Pipeline] echo
unable to read dir1: hudson.AbortException: script returned exit code 2
Unfortunately hudson.AbortException is missing any useful method to obtain that exit status, so if the actual value is required you'd need to parse it out of the message (ugh!)
Contrary to the Javadoc https://javadoc.jenkins-ci.org/hudson/AbortException.html the build is not failed when this exception is caught. It fails when it's not caught!
Update:
If you also want the STDERR output from the shell command, Jenkins unfortunately fails to properly support that common use-case. A 2017 ticket JENKINS-44930 is stuck in a state of opinionated ping-pong whilst making no progress towards a solution - please consider adding your upvote to it.
As to a solution now, there could be a couple of possible approaches:
a) Redirect STDERR to STDOUT 2>&1
- but it's then up to you to parse that out of the main output though, and you won't get the output if the command failed - because you're in the exception handler.
b) redirect STDERR to a temporary file (the name of which you prepare earlier) 2>filename (but remember to clean up the file afterwards) - ie. main code becomes:
def stderrfile = 'stderr.out'
try {
def dir1 = sh(script:"ls -la dir1 2>${stderrfile}", returnStdout:true).trim()
} catch (Exception ex) {
def errmsg = readFile(stderrfile)
println("Unable to read dir1: ${ex} - ${errmsg}")
}
c) Go the other way, set returnStatus=true instead, dispense with the exception handler and always capture output to a file, ie:
def outfile = 'stdout.out'
def status = sh(script:"ls -la dir1 >${outfile} 2>&1", returnStatus:true)
def output = readFile(outfile).trim()
if (status == 0) {
// output is directory listing from stdout
} else {
// output is error message from stderr
}
Caveat: the above code is Unix/Linux-specific - Windows requires completely different shell commands.
this is a sample case, which will make sense I believe!
node('master'){
stage('stage1'){
def commit = sh (returnStdout: true, script: '''echo hi
echo bye | grep -o "e"
date
echo lol''').split()
echo "${commit[-1]} "
}
}
For those who need to use the output in subsequent shell commands, rather than groovy, something like this example could be done:
stage('Show Files') {
environment {
MY_FILES = sh(script: 'cd mydir && ls -l', returnStdout: true)
}
steps {
sh '''
echo "$MY_FILES"
'''
}
}
I found the examples on code maven to be quite useful.
All the above method will work. but to use the var as env variable inside your code you need to export the var first.
script{
sh " 'shell command here' > command"
command_var = readFile('command').trim()
sh "export command_var=$command_var"
}
replace the shell command with the command of your choice. Now if you are using python code you can just specify os.getenv("command_var") that will return the output of the shell command executed previously.
How to read the shell variable in groovy / how to assign shell return value to groovy variable.
Requirement : Open a text file read the lines using shell and store the value in groovy and get the parameter for each line .
Here , is delimiter
Ex: releaseModule.txt
./APP_TSBASE/app/team/i-home/deployments/ip-cc.war/cs_workflowReport.jar,configurable-wf-report,94,23crb1,artifact
./APP_TSBASE/app/team/i-home/deployments/ip.war/cs_workflowReport.jar,configurable-temppweb-report,394,rvu3crb1,artifact
========================
Here want to get module name 2nd Parameter (configurable-wf-report) , build no 3rd Parameter (94), commit id 4th (23crb1)
def module = sh(script: """awk -F',' '{ print \$2 "," \$3 "," \$4 }' releaseModules.txt | sort -u """, returnStdout: true).trim()
echo module
List lines = module.split( '\n' ).findAll { !it.startsWith( ',' ) }
def buildid
def Modname
lines.each {
List det1 = it.split(',')
buildid=det1[1].trim()
Modname = det1[0].trim()
tag= det1[2].trim()
echo Modname
echo buildid
echo tag
}
If you don't have a single sh command but a block of sh commands, returnstdout wont work then.
I had a similar issue where I applied something which is not a clean way of doing this but eventually it worked and served the purpose.
Solution -
In the shell block , echo the value and add it into some file.
Outside the shell block and inside the script block , read this file ,trim it and assign it to any local/params/environment variable.
example -
steps {
script {
sh '''
echo $PATH>path.txt
// I am using '>' because I want to create a new file every time to get the newest value of PATH
'''
path = readFile(file: 'path.txt')
path = path.trim() //local groovy variable assignment
//One can assign these values to env and params as below -
env.PATH = path //if you want to assign it to env var
params.PATH = path //if you want to assign it to params var
}
}
Easiest way is use this way
my_var=`echo 2`
echo $my_var
output
: 2
note that is not simple single quote is back quote ( ` ).
I am trying to get a file name and pass it to a command using popen. Then I want to print the output. This is my code:
filePath = tkinter.filedialog.askopenfilename(filetypes=[("All files", "*.*")])
fileNameStringForm = (basename(filePath ))
fileNameByteForm = fileNameStringForm.encode(encoding='utf-8')
process = subprocess.Popen(['gagner','-arg1'], shell = True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
process .communicate(fileNameByteForm )
stdout, stderr = process .communicate() <<------ERROR POINTS TO THIS LINE
stringOutput = stdout.decode('urf-8')
print(stringOutput)
I am getting the following error:
ValueError: Invalid file object: <_io.BufferedReader name=9>
I have looked at other similar questions but nothing seems to have solved my problem. Can some show me where I am going wrong in the code?
Edit:
If I were to run the command in a command line it would be:
gagner -arg1 < file1
What you are doing is not what you are describing in the supposed command line argument. You are actually executing this:
echo "file1" | gagner -arg1
You will need to make sure that you pass in the file contents yourself. Popen will not open and read the file for you.
According to the documentation, what communicate() does is
interact with process: Send data to stdin. Read data from stdout and stderr, until end-of-file is reached. Wait for process to terminate.
So, once you have run
process.communicate(fileNameByteForm)
your sub process has finished and the pipes have been closed. The second call will then fail as a result.
What you want to do instead is
stdout, stderr = process.communicate(input_data)
which will pipe your input data into the sub process and read stdout and stderr.
I have to run two scripts, but 2nd script will only run if first script success.
First script
server_name=$1;
src_path=$2;
dst_path=$3;
dst_dir=$4;
cd $src_path;
smbclient //$server_name -A ~/\.smbclient -c "cd $dst_path;mkdir $dst_dir;cd $dst_dir;pwd; recurse; prompt; mput *; exit;";
Above script copy the contents from source path to a directory in remote server.
second script
#!/usr/bin/python
import sys
import os
import subprocess
project = sys.argv[1]
path = sys.argv[2]
print "\n inside second script"
fileName = "myfile.html"
file = open(fileName, "a")
if file:
file.write('Release Locations:<br>\n' +
'<ul>\n' +
'<li>file://' + path + '</li>\n' +
'<li>smb://' + path + '</li>\n' +
'</ul>')
else:
exit(1)
command(control don't go to 2nd script)
./first_script.sh server_name/Shared /home/myname/work/target Software/target required_files && python second_script.py sony server_name/Shared/Software/target/required_files
command(control don't go to 2nd script)
./first_script.sh server_name/Shared /home/myname/work/target Software/target required_files || python second_script.py sony server_name/Shared/Software/target/required_files
command(control go to 2nd script)
./first_script.sh server_name/Shared /home/myname/work/target Software/target required_files;python second_script.py sony server_name/Shared/Software/target/required_files
On executing first script && or || with second script, control is not going into second script.
status of first_script when run individually: 0 (checked by running echo $?)
First script is successfully copying contents with some warnings.
If i run 2nd script individually it runs.
warnings
WARNING: The "idmap uid" option is deprecated
WARNING: The "idmap gid" option is deprecated
Questions
Because of this warnings 2nd script is not executing ?
How to make second script script execute ?
This is my current directory structure
/ <-- current working Dir
/php/
file1.php
file2.php
file3.txt
I am trying to execute the following groovy commands
def cp = 'cp -f *.php /tmp/'
def cpProc = cp.execute(null, new File('./php/')
cpProc.waitfor()
log.info 'current exitvalue :' + cpProc.exitValue()
log.info 'current proc out : ' + cpProc.text
but I keep getting cp: cannot stat *.php': No such file or directory, I've verified the files exist and I've verified my current working directory
if I execute log.info 'ls -la'.execute(null, new File('./php/')) I see the PHP and .txt files.
This seems like a long shot but I think there might be a bug with using wild cards for commands when executing them in a specified working directory, unless there's something I'm missing?
I'm using groovy 1.7.5
this version works for me, just try it out:
#!/usr/bin/env groovy
command = ["sh", "-c", "cp -f *.php /tmp/"]
def cpProc = command.execute(null, new File('./php/'))
cpProc.waitFor()
print 'current exitvalue :' + cpProc.exitValue() + '\n'
print 'current proc out : ' + cpProc.text + '\n'
print 'ls -la'.execute(null, new File('/tmp/')).text
The first answer on this question explains why your version did not work: Groovy execute "cp *" shell command
i am creating my own plugin when i try to create a task type Exec. Its not running any command. i am getting command line error stating command is not correct. I have displayed the generated command in Exec command to command prompt and when i run it works. The code is not wroking inside gradle plugin scope.
task myrun (type: Exec) {
def cp = project.files(
project.sourceSets.main.output.classesDir,
project.sourceSets.main.resources,
project.configurations.runtime
).getAsPath()
String myCommand = "visage -cp ${cp} visage.javafx.scene.effect.EffectTest"
println "RUN COMMAND : ${myCommand}"
workingDir project.sourceSets.main.output.classesDir
// classpath project.files([project.sourceSets.main.output.classesDir,project.sourceSets.main.resources, project.configurations.runtime,])
commandLine = [myCommand]
}
Can anyone tell me is there any error in the code?
Here is the answer to the same post of yours in the Gradle forum:
commandLine is a list, and every argument becomes a separate element of that list:
commandLine "visage", "-cp", cp, "visage.javafx.scene.effect.EffectTest"
An alternative approach would be to use something like this
exec clause
{
executable = "bash"
args = [ "-c", """ "command arg1 arg2... argn" """]
...
}