I am trying to run a simple one-rule snakemake file as following:
resources_dir='resources'
rule downloadReference:
output:
fa = resources_dir+'/human_g1k_v37.fasta',
fai = resources_dir+'/human_g1k_v37.fasta.fai',
shell:
('mkdir -p '+resources_dir+'; cd '+resources_dir+'; ' +
'wget ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/human_g1k_v37.fasta.gz; gunzip human_g1k_v37.fasta.gz; ' +
'wget ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/human_g1k_v37.fasta.fai;')
But I get an error as :
Error in job downloadReference while creating output files
resources/human_g1k_v37.fasta, resources/human_g1k_v37.fasta.fai.
RuleException:
CalledProcessError in line 10 of
/lustre4/home/masih/projects/NGS_pipeline/snake_test:
Command 'mkdir -p resources; cd resources; wget ftp://ftp-
trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/human_g1k_v37.fasta.gz; gunzip human_g1k_v37.fasta.gz; wget ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/human_g1k_v37.fasta.fai;' returned non-zero exit status 2.
File "/lustre4/home/masih/projects/NGS_pipeline/snake_test", line 10, in __rule_downloadReference
File "/home/masih/miniconda3/lib/python3.6/concurrent/futures/thread.py", line 55, in run
Removing output files of failed job downloadReference since they might be corrupted:
resources/human_g1k_v37.fasta
Will exit after finishing currently running jobs.
Exiting because a job execution failed. Look above for error message
I am not using the threads option in snakemake. I can not figure out how this is related with thread.py. Anybody has experience with this error?
When a shell command fails, it has an exit status which is not 0.
This is what "returned non-zero exit status 2" indicates.
One of your shell command fails, and the failure is propagated to snakemake. I suppose that snakemake uses threads and that the failure manifests itself at the level of some code in the threads.py file1.
In order to better understand what is happening, we can capture the first error using the || operator followed by a function issuing an error message:
# Define functions to be used in shell portions
shell.prefix("""
# http://linuxcommand.org/wss0150.php
PROGNAME=$(basename $0)
function error_exit
{{
# ----------------------------------------------------------------
# Function for exit due to fatal program error
# Accepts 1 argument:
# string containing descriptive error message
# ----------------------------------------------------------------
echo "${{PROGNAME}}: ${{1:-"Unknown Error"}}" 1>&2
exit 1
}}
""")
resources_dir='resources'
rule downloadReference:
output:
fa = resources_dir+'/human_g1k_v37.fasta',
fai = resources_dir+'/human_g1k_v37.fasta.fai',
params:
resources_dir = resources_dir
shell:
"""
mkdir -p {params.resources_dir}
cd {params.resources_dir}
wget ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/human_g1k_v37.fasta.gz || error_exit "fasta download failed"
gunzip human_g1k_v37.fasta.gz || error_exit "fasta gunzip failed"
wget ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/human_g1k_v37.fasta.fai || error_exit "fai download failed"
"""
When I run this, I get the following message after the messages of the first download:
gzip: human_g1k_v37.fasta.gz: decompression OK, trailing garbage ignored
bash: fasta gunzip failed
It turns out that gzip uses a non-zero exit code in case of warnings:
Exit status is normally 0; if an error occurs, exit status is 1. If a warning occurs, exit status is 2.
(from the DIAGNOSTICS section of man gzip)
If I remove the error-capturing || error_exit "fasta gunzip failed", the workflow is able to complete. So I don't understand why you had this error in the first place.
I'm surprised that gzip authors decided to use a non-zero status in case of a simple warning. They added a -q option to turn off this specific warning, due to the presence of trailing zeroes, but strangely, the exit status is still non-zero when this option is used.
1 According to Johannes Köster, author of snakemake:
Sorry for the misleading thread.py thing, this is just the place where snakemake detects the problem. The real issue is that your command exits with exit code 2, which indicates an error not related to Snakemake
Related
I am new to using Linux/Unix command. I have a command that creates a Jmeter file and then in log checks if the generated Jmeter file (If there is any error in Jmeter file it will show error in percentage). If the log has some errors it provides Failure. During this process I get outcome as Failure or Success but the file does not get generated. How can I modify the command so that file gets generated as well as able to check error.
if jmeter -n -t $JMX_FILE -Jurl=$TEST_URL -Jthreads=1 -Jrampup=1 -Jcount=1 -Jsla=$SLA -f -l jmeter-results.jtl | grep "0 (0.00%)$"
then
echo Success
exit 0
else
echo Failure
exit 1
fi
I would like to conduct analysis using program Arlsumstat_64bit with thousand of input files.
Arlsumstat_64bit reads input files (.arp) and write result file (sumstat.out).
Each input will append new line on the result file (sumstat.out) based on the argument "0 1"
Therefore, I wrote a shell script to execute all the input (*.arp) in the same folder.
However, if the input files contain error, the shell script will be stuck without any subsequently process. Therefore, I found a command with "timeout" to deal my issue.
I made a shell script as following
#!/bin/bash
for sp in $(ls *.arp) ;
do
echo "process start: $sp"
timeout 10 arlsumstat_64bit ${sp}.arp sumstat.out 1 0
rm -r ${sp}.res
echo "process done: $sp"
done
However, I still need to know which input files failed.
How could make a list telling me which input files are "timeout"?
See the man page for the timeout command http://man7.org/linux/man-pages/man1/timeout.1.html
If the command times out, and --preserve-status is not set, then exit
with status 124. Otherwise, exit with the status of COMMAND. If no
signal is specified, send the TERM signal upon timeout. The TERM
signal kills any process that does not block or catch that signal.
It may be necessary to use the KILL (9) signal, since this signal
cannot be caught, in which case the exit status is 128+9 rather than
124.
You should find out which exit codes are possible for the program arlsumstat_64bit. I assume it should exit with status 0 on success. Otherwise the script below will not work. If you need to distinguish between timeout and other errors it should not use exit status 124 or which is used by timeout to indicate a timeout. So you can check the exit status of your command to distinguish between success, error or timeout as necessary.
To keep the script simple I assume you don't need to distingish between timeout and other errors.
I added some comments where I modified your script to improve it or to show alternatives.
#!/bin/bash
# don't parse the output of ls
for sp in *.arp
do
echo "process start: $sp"
# instead of using "if timeout 10 arlsumstat_64bit ..." you could also run
# timeout 10 arlsumstat_64bit... and check the value of `$?` afterwards,
# e.g. if you want to distinguish between error and timeout.
# $sp will already contain .arp so ${sp}.arp is wrong
# use quotes in case a file name contains spaces
if timeout 10 arlsumstat_64bit "${sp}" sumstat.out 1 0
then
echo "process done: $sp"
else
echo "processing failed or timeout: $sp"
fi
# If the result for foo.arp is foo.res, the .arp must be removed
# If it is foo.arp.res, rm -r "${sp}.res" would be correct
# use quotes
rm -r "${sp%.arp}.res"
done
Below code should work for you:
#!/bin/bash
for sp in $(ls *.arp) ;
do
echo "process start: $sp"
timeout 10 arlsumstat_64bit ${sp}.arp sumstat.out 1 0
if [ $? -eq 0 ]
then
echo "process done sucessfully: $sp"
else
echo "process failed: $sp"
fi
echo "Deleting ${sp}.res"
rm -r ${sp}.res
done
When you want to set a time limit for a process, you can simply use timeout before the process:
timeout 1.5s COMMAND
This will kill the COMMAND if it was not done after 1.5 seconds.
I used that command in some bash scripts; How can i know if one process was completely done before the time limit, or it was killed (because of exceeding the time limit)?
The Gnu timeout command normally returns a status code of 124 if the timeout was exceeded. Otherwise, it returns the status code returned by the command itself. So you can test the status code by grabbing the value of $? immediately after executing timeout:
timeout 1.5s COMMAND
status=$?
if ((status==124)); then
# command timed out
elif (status!=0)); then
# command terminated in time, but it returned an error status
else
# command terminated in time and reported success
fi
If your command might return the status code 124, then you would have to use the --preserve-status option and check to see if the command was terminated by the signal you tell timeout to send. See the man timeout for details.
Add && echo >> time_limit.txt after COMMAND:
timeout 1.5s COMMAND && echo >> time_limit.txt
So, if you want to see if the COMMAND was killed, check the existence of file time_limit.txt. If that file exists, it means the command was NOT killed. Otherwise, the command was killed.
In bash script, you can check the existence of that file as follow:
if [[ -r time_limit.txt ]]then;
{
echo "The command was NOT killed"
}
else
{
echo "The command was killed"
}
I am trying to make a script where I strictly want to control the standard output. so I am redirecting everything to /dev/null and printing my own messages.
Example:
user#root> tar zxvf samplefile.tar;> /dev/null 2>&1 ;echo $?
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error exit delayed from previous errors
0
tar command failed but $? is storing exit status of redirection. I want exit status of tar command.
Query:
How can I get the status of tar command also redirect all std output to /dev/null?
As mentioned by Subbeh and Ploutox, I should not have used ";" in between the two commands.after removing ";" ,redirection became part of the orignal command and gave the correct return code.
However I am still not sure what could be used if we need to get exit status of ($?-1).
Sometimes when I execute a bash script with the curl command to upload some files to my ftp server, it will return some error like:
56 response reading failed
and I have to find the wrong line and re-run them manually and it will be OK.
I'm wondering if that could be re-run automatically when the error occurs.
My scripts is like this:
#there are some files(A,B,C,D,E) in my to_upload directory,
# which I'm trying to upload to my ftp server with curl command
for files in `ls` ;
do curl -T $files ftp.myserver.com --user ID:pw ;
done
But sometimes A,B,C, would be uploaded successfully, only D were left with an "error 56", so I have to rerun curl command manually. Besides, as Will Bickford said, I prefer that no confirmation will be required, because I'm always asleep at the time the script is running. :)
Here's a bash snippet I use to perform exponential back-off:
# Retries a command a configurable number of times with backoff.
#
# The retry count is given by ATTEMPTS (default 5), the initial backoff
# timeout is given by TIMEOUT in seconds (default 1.)
#
# Successive backoffs double the timeout.
function with_backoff {
local max_attempts=${ATTEMPTS-5}
local timeout=${TIMEOUT-1}
local attempt=1
local exitCode=0
while (( $attempt < $max_attempts ))
do
if "$#"
then
return 0
else
exitCode=$?
fi
echo "Failure! Retrying in $timeout.." 1>&2
sleep $timeout
attempt=$(( attempt + 1 ))
timeout=$(( timeout * 2 ))
done
if [[ $exitCode != 0 ]]
then
echo "You've failed me for the last time! ($#)" 1>&2
fi
return $exitCode
}
Then use it in conjunction with any command that properly sets a failing exit code:
with_backoff curl 'http://monkeyfeathers.example.com/'
Perhaps this will help. It will try the command, and if it fails, it will tell you and pause, giving you a chance to fix run-my-script.
COMMAND=./run-my-script.sh
until $COMMAND; do
read -p "command failed, fix and hit enter to try again."
done
I have faced a similar problem where I need to make contact with servers using curl that are in the process of starting up and haven't started up yet, or services that are temporarily unavailable for whatever reason. The scripting was getting out of hand, so I made a dedicated retry tool that will retry a command until it succeeds:
#there are some files(A,B,C,D,E) in my to_upload directory,
# which I'm trying to upload to my ftp server with curl command
for files in `ls` ;
do retry curl -f -T $files ftp.myserver.com --user ID:pw ;
done
The curl command has the -f option, which returns code 22 if the curl fails for whatever reason.
The retry tool will by default run the curl command over and over forever until the command returns status zero, backing off for 10 seconds between retries. In addition retry will read from stdin once and once only, and writes to stdout once and once only, and writes all stdout to stderr if the command fails.
Retry is available from here: https://github.com/minfrin/retry