I am new to using Linux/Unix command. I have a command that creates a Jmeter file and then in log checks if the generated Jmeter file (If there is any error in Jmeter file it will show error in percentage). If the log has some errors it provides Failure. During this process I get outcome as Failure or Success but the file does not get generated. How can I modify the command so that file gets generated as well as able to check error.
if jmeter -n -t $JMX_FILE -Jurl=$TEST_URL -Jthreads=1 -Jrampup=1 -Jcount=1 -Jsla=$SLA -f -l jmeter-results.jtl | grep "0 (0.00%)$"
then
echo Success
exit 0
else
echo Failure
exit 1
fi
Related
I have created a shell scripts which connects to teradata and executes the given condition, lets say test.sh. i have created a wrapper script to call test.sh if there is only an input file for test.sh, wrapper.sh.
Wrapper.sh:
cd ${FILELOC
COUNT=$(ls -l 1001_*.txt | wc -l)
if [ "$COUNT" -ne 0 ]
then
/u/w/us/bin/test.sh.sh`
fi
when i executes wrapper.sh manually, the test.sh is been called and getting executed without an error. but when i schedule it in cron it throws an error as shown below
EXIT ERRORCODE;
*** Exiting BTEQ...
*** RC (return code) = 999
Please help me to understand the issue
Shell script name seems to be Invalid
Can you share the the content of the test.sh.sh fine so that it will be easy to debug
There might be chances that file permission are not proper. due to this it got failed while executing from cron scheduler
This question already has answers here:
Exit when one process in pipe fails
(2 answers)
Closed 4 years ago.
Even if the mycode.sh has non-0 exit code this command returns 0 as ssh connection was successful. How to get the actual return code of the .sh on remote server?
/home/mycode.sh '20'${ODATE} 1 | ssh -L 5432:localhost:5432 myuser#myremotehost cat
This is not related to SSH, but to how bash handles the exit status in pipelines. From the bash manual page:
The return status of a pipeline is the exit status of the last command, unless the pipefail option is enabled. If pipefail is enabled, the pipeline's return status is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully. If the reserved word ! precedes a pipeline, the exit status of that pipeline is the logical negation of the exit status as described above. The shell waits for all commands in the pipeline to terminate before returning a value.
If you want to check that there was an error in the pipeline due to any of the commands involved, just set the pipefail option:
set -o pipefail
your_pipeline_here
echo $? # Prints non-zero if something went wrong
It is not possible to actually send the exit status to the next command in the pipeline (in your case, ssh) without additional steps. If you really want to do that, the command will have to be split like this:
res="$(/home/mycode.sh '20'${ODATE} 1)"
if (( $? == 0 )); then
echo -n "$res" | ssh -L 5432:localhost:5432 myuser#myremotehost cat
else
# You can do anything with the exit status here - even pass it on as an argument to the remote command
echo "mycode.sh failed" >&2
fi
You may want to save the output of mycode.sh to a temporary file instead of the $res variable if it's too large.
/home/mycode.sh is located onto the local host.
the ssh command is running cat on the remote server.
All text printed to the standard output of the /home/mycode.sh is redirected to the cat standard input.
The man ssh reads:
EXIT STATUS
ssh exits with the exit status of the remote command or with 255 if an error occurred.
Conclusion: the ssh exists with the EXIT STATUS of the cat or 255 if an error occurred.
if /home/mycode.sh script prints commands to the standard input, they can be run on the remote server when the cat is not present:
/home/mycode.sh '20'${ODATE} 1 | ssh -L 5432:localhost:5432 myuser#myremotehost
In my test, the EXIT STATUS of the last command executed on the remote server is returned by ssh:
printf "%s\n" "uname -r" date "ls this_file_does_not_exist" |\
ssh -L 5432:localhost:5432 myuser#myremotehost ;\
printf "EXIT STATUS of the last command, executed remotely with ssh is %d\n" $?
4.4.0-119-generic
Wed Aug 29 02:55:04 EDT 2018
ls: cannot access 'this_file_does_not_exist': No such file or directory
EXIT STATUS of the last command, executed remotely with ssh is 2
I am trying to run a simple one-rule snakemake file as following:
resources_dir='resources'
rule downloadReference:
output:
fa = resources_dir+'/human_g1k_v37.fasta',
fai = resources_dir+'/human_g1k_v37.fasta.fai',
shell:
('mkdir -p '+resources_dir+'; cd '+resources_dir+'; ' +
'wget ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/human_g1k_v37.fasta.gz; gunzip human_g1k_v37.fasta.gz; ' +
'wget ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/human_g1k_v37.fasta.fai;')
But I get an error as :
Error in job downloadReference while creating output files
resources/human_g1k_v37.fasta, resources/human_g1k_v37.fasta.fai.
RuleException:
CalledProcessError in line 10 of
/lustre4/home/masih/projects/NGS_pipeline/snake_test:
Command 'mkdir -p resources; cd resources; wget ftp://ftp-
trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/human_g1k_v37.fasta.gz; gunzip human_g1k_v37.fasta.gz; wget ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/human_g1k_v37.fasta.fai;' returned non-zero exit status 2.
File "/lustre4/home/masih/projects/NGS_pipeline/snake_test", line 10, in __rule_downloadReference
File "/home/masih/miniconda3/lib/python3.6/concurrent/futures/thread.py", line 55, in run
Removing output files of failed job downloadReference since they might be corrupted:
resources/human_g1k_v37.fasta
Will exit after finishing currently running jobs.
Exiting because a job execution failed. Look above for error message
I am not using the threads option in snakemake. I can not figure out how this is related with thread.py. Anybody has experience with this error?
When a shell command fails, it has an exit status which is not 0.
This is what "returned non-zero exit status 2" indicates.
One of your shell command fails, and the failure is propagated to snakemake. I suppose that snakemake uses threads and that the failure manifests itself at the level of some code in the threads.py file1.
In order to better understand what is happening, we can capture the first error using the || operator followed by a function issuing an error message:
# Define functions to be used in shell portions
shell.prefix("""
# http://linuxcommand.org/wss0150.php
PROGNAME=$(basename $0)
function error_exit
{{
# ----------------------------------------------------------------
# Function for exit due to fatal program error
# Accepts 1 argument:
# string containing descriptive error message
# ----------------------------------------------------------------
echo "${{PROGNAME}}: ${{1:-"Unknown Error"}}" 1>&2
exit 1
}}
""")
resources_dir='resources'
rule downloadReference:
output:
fa = resources_dir+'/human_g1k_v37.fasta',
fai = resources_dir+'/human_g1k_v37.fasta.fai',
params:
resources_dir = resources_dir
shell:
"""
mkdir -p {params.resources_dir}
cd {params.resources_dir}
wget ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/human_g1k_v37.fasta.gz || error_exit "fasta download failed"
gunzip human_g1k_v37.fasta.gz || error_exit "fasta gunzip failed"
wget ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/human_g1k_v37.fasta.fai || error_exit "fai download failed"
"""
When I run this, I get the following message after the messages of the first download:
gzip: human_g1k_v37.fasta.gz: decompression OK, trailing garbage ignored
bash: fasta gunzip failed
It turns out that gzip uses a non-zero exit code in case of warnings:
Exit status is normally 0; if an error occurs, exit status is 1. If a warning occurs, exit status is 2.
(from the DIAGNOSTICS section of man gzip)
If I remove the error-capturing || error_exit "fasta gunzip failed", the workflow is able to complete. So I don't understand why you had this error in the first place.
I'm surprised that gzip authors decided to use a non-zero status in case of a simple warning. They added a -q option to turn off this specific warning, due to the presence of trailing zeroes, but strangely, the exit status is still non-zero when this option is used.
1 According to Johannes Köster, author of snakemake:
Sorry for the misleading thread.py thing, this is just the place where snakemake detects the problem. The real issue is that your command exits with exit code 2, which indicates an error not related to Snakemake
I have written a script where it checks the PID of that JVM. If PID is not existing for that specific JVM, it will give exit status 2 and terminates the script. To check the PID, I do a cat of specific file which contains the PID. When there is no PID or no file which contains PID, it is throwing "No such file or directory" even after redirecting the output using > /dev/null 2>&1 at the end of the command.
I don't want the output displayed on the screen. If someone could help me on that would be really helpful.
EDIT this is the code snippet:
ONLINE=grep -ir "$HOSTNAME-"$VAR" is currently online" /home/logs/$GF_OWNER/$HOSTNAME-"$VAR"/$HOSTNAME-"$VAR".log | wc -l
OFFLINE=0
PID=cat /home/logs/$GF_OWNER/$HOSTNAME-"$VAR"/gemstart.pid
It think you should try to test if your file exists before using cat/ls/whatever on it. For that, use the if/test structure :
if [ -e $PETH_TO_THE_FILE ]; then
//processing here
else
fi
This way you won't have unnecessary output.
I have a small problem regarding "sftp".
I have a script, which simply transfers a file to a remote sftp server. But when this script runs it fails at sftp and my script fails.
So, i have to manually transfer the file,using command which is same as the command that i have used in the script, and it works fine.
So my problem is that the sftp command runs smoothly when i run it manually, but creates problem when the same command is run through the script.
this is the code that I'm using
sftp -v -b sftp_input.txt UserId#aa.bb.cc.dd
if (($? > 0 ));
then
echo "sftp error. Exiting.."
exit
fi
where sftp_input.txt contains the cmd to put the file to remote server.
Please advice.....
The script can't work because it's malformed. You forgot to separate the if statement and also forgot the closing fi. Here's the correct form for your script:
sftp -v -b sftp_input.txt UserId#aa.bb.cc.ddd
if (($? > 0 )); then
echo "sftp error. Exiting.."
exit
fi
If you want it all in one line, then:
sftp -v -b sftp_input.txt UserId#aa.bb.cc.ddd; if (($? > 0 )); then echo "sftp error. Exiting.."; exit; fi
But as you can see it's a bad idea. Better to write readable and well indented code.