Related
Does crontab have an argument for creating cron jobs without using the editor (crontab -e)? If so, what would be the code to create a cron job from a Bash script?
You can add to the crontab as follows:
#write out current crontab
crontab -l > mycron
#echo new cron into cron file
echo "00 09 * * 1-5 echo hello" >> mycron
#install new cron file
crontab mycron
rm mycron
Cron line explaination
* * * * * "command to be executed"
- - - - -
| | | | |
| | | | ----- Day of week (0 - 7) (Sunday=0 or 7)
| | | ------- Month (1 - 12)
| | --------- Day of month (1 - 31)
| ----------- Hour (0 - 23)
------------- Minute (0 - 59)
Source nixCraft.
You may be able to do it on-the-fly
crontab -l | { cat; echo "0 0 0 0 0 some entry"; } | crontab -
crontab -l lists the current crontab jobs, cat prints it, echo prints the new command and crontab - adds all the printed stuff into the crontab file. You can see the effect by doing a new crontab -l.
This shorter one requires no temporary file, it is immune to multiple insertions, and it lets you change the schedule of an existing entry.
Say you have these:
croncmd="/home/me/myfunction myargs > /home/me/myfunction.log 2>&1"
cronjob="0 */15 * * * $croncmd"
To add it to the crontab, with no duplication:
( crontab -l | grep -v -F "$croncmd" ; echo "$cronjob" ) | crontab -
To remove it from the crontab whatever its current schedule:
( crontab -l | grep -v -F "$croncmd" ) | crontab -
Notes:
grep -F matches the string literally, as we do not want to interpret it as a regular expression
We also ignore the time scheduling and only look for the command. This way; the schedule can be changed without the risk of adding a new line to the crontab
Thanks everybody for your help. Piecing together what I found here and elsewhere I came up with this:
The Code
command="php $INSTALL/indefero/scripts/gitcron.php"
job="0 0 * * 0 $command"
cat <(fgrep -i -v "$command" <(crontab -l)) <(echo "$job") | crontab -
I couldn't figure out how to eliminate the need for the two variables without repeating myself.
command is obviously the command I want to schedule. job takes $command and adds the scheduling data. I needed both variables separately in the line of code that does the work.
Details
Credit to duckyflip, I use this little redirect thingy (<(*command*)) to turn the output of crontab -l into input for the fgrep command.
fgrep then filters out any matches of $command (-v option), case-insensitive (-i option).
Again, the little redirect thingy (<(*command*)) is used to turn the result back into input for the cat command.
The cat command also receives echo "$job" (self explanatory), again, through use of the redirect thingy (<(*command*)).
So the filtered output from crontab -l and the simple echo "$job", combined, are piped ('|') over to crontab - to finally be written.
And they all lived happily ever after!
In a nutshell:
This line of code filters out any cron jobs that match the command, then writes out the remaining cron jobs with the new one, effectively acting like an "add" or "update" function.
To use this, all you have to do is swap out the values for the command and job variables.
EDIT (fixed overwriting):
cat <(crontab -l) <(echo "1 2 3 4 5 scripty.sh") | crontab -
There have been a lot of good answers around the use of crontab, but no mention of a simpler method, such as using cron.
Using cron would take advantage of system files and directories located at /etc/crontab, /etc/cron.daily,weekly,hourly or /etc/cron.d/:
cat > /etc/cron.d/<job> << EOF
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root HOME=/
01 * * * * <user> <command>
EOF
In this above example, we created a file in /etc/cron.d/, provided the environment variables for the command to execute successfully, and provided the user for the command, and the command itself. This file should not be executable and the name should only contain alpha-numeric and hyphens (more details below).
To give a thorough answer though, let's look at the differences between crontab vs cron/crond:
crontab -- maintain tables for driving cron for individual users
For those who want to run the job in the context of their user on the system, using crontab may make perfect sense.
cron -- daemon to execute scheduled commands
For those who use configuration management or want to manage jobs for other users, in which case we should use cron.
A quick excerpt from the manpages gives you a few examples of what to and not to do:
/etc/crontab and the files in /etc/cron.d must be owned by root, and must not be group- or other-writable. In contrast to the spool area, the files under /etc/cron.d or the files under /etc/cron.hourly, /etc/cron.daily, /etc/cron.weekly and /etc/cron.monthly may also be symlinks, provided that both the symlink and the file it points to are owned by root. The files under /etc/cron.d do not need to be executable, while the files under /etc/cron.hourly, /etc/cron.daily, /etc/cron.weekly and /etc/cron.monthly do, as they are run by run-parts (see run-parts(8) for more information).
Source: http://manpages.ubuntu.com/manpages/trusty/man8/cron.8.html
Managing crons in this manner is easier and more scalable from a system perspective, but will not always be the best solution.
So, in Debian, Ubuntu, and many similar Debian based distros...
There is a cron task concatenation mechanism that takes a config file, bundles them up and adds them to your cron service running.
You can put a file under the /etc/cron.d/somefilename where somefilename is whatever you want.
sudo echo "0,15,30,45 * * * * ntpdate -u time.nist.gov" >> /etc/cron.d/vmclocksync
Let's disassemble this:
sudo - because you need elevated privileges to change cron configs under the /etc directory
echo - a vehicle to create output on std out. printf, cat... would work as well
" - use a doublequote at the beginning of your string, you're a professional
0,15,30,45 * * * * - the standard cron run schedule, this one runs every 15 minutes
ntpdate -u time.nist.gov - the actual command I want to run
" - because my first double quotes needs a buddy to close the line being output
>> - the double redirect appends instead of overwrites*
/etc/cron.d/vmclocksync - vmclocksync is the filename I've chosen, it goes in /etc/cron.d/
* if we used the > redirect, we could guarantee we only had one task entry. But, we would be at risk of blowing away any other rules in an existing file. You can decide for yourself if possible destruction with > is right or possible duplicates with >> are for you. Alternatively, you could do something convoluted or involved to check if the file name exists, if there is anything in it, and whether you are adding any kind of duplicate-- but, I have stuff to do and I can't do that for you right now.
For a nice quick and dirty creation/replacement of a crontab from with a BASH script, I used this notation:
crontab <<EOF
00 09 * * 1-5 echo hello
EOF
Chances are you are automating this, and you don't want a single job added twice.
In that case use:
__cron="1 2 3 4 5 /root/bin/backup.sh"
cat <(crontab -l) |grep -v "${__cron}" <(echo "${__cron}")
This only works if you're using BASH. I'm not aware of the correct DASH (sh) syntax.
Update: This doesn't work if the user doesn't have a crontab yet. A more reliable way would be:
(crontab -l ; echo "1 2 3 4 5 /root/bin/backup.sh") | sort - | uniq - | crontab -
Alternatively, if your distro supports it, you could also use a separate file:
echo "1 2 3 4 5 /root/bin/backup.sh" |sudo tee /etc/crond.d/backup
Found those in another SO question.
echo "0 * * * * docker system prune --force >/dev/null 2>&1" | sudo tee /etc/cron.daily/dockerprune
A variant which only edits crontab if the desired string is not found there:
CMD="/sbin/modprobe fcpci"
JOB="#reboot $CMD"
TMPC="mycron"
grep "$CMD" -q <(crontab -l) || (crontab -l>"$TMPC"; echo "$JOB">>"$TMPC"; crontab "$TMPC")
(2>/dev/null crontab -l ; echo "0 3 * * * /usr/local/bin/certbot-auto renew") | crontab -
cat <(crontab -l 2>/dev/null) <(echo "0 3 * * * /usr/local/bin/certbot-auto renew") | crontab -
#write out current crontab
crontab -l > mycron 2>/dev/null
#echo new cron into cron file
echo "0 3 * * * /usr/local/bin/certbot-auto renew" >> mycron
#install new cron file
crontab mycron
rm mycron
If you're using the Vixie Cron, e.g. on most Linux distributions, you can just put a file in /etc/cron.d with the individual cronjob.
This only works for root of course. If your system supports this you should see several examples in there. (Note the username included in the line, in the same syntax as the old /etc/crontab)
It's a sad misfeature in cron that there is no way to handle this as a regular user, and that so many cron implementations have no way at all to handle this.
My preferred solution to this would be this:
(crontab -l | grep . ; echo -e "0 4 * * * myscript\n") | crontab -
This will make sure you are handling the blank new line at the bottom correctly. To avoid issues with crontab you should usually end the crontab file with a blank new line. And the script above makes sure it first removes any blank lines with the "grep ." part, and then add in a new blank line at the end with the "\n" in the end of the script. This will also prevent getting a blank line above your new command if your existing crontab file ends with a blank line.
Bash script for adding cron job without the interactive editor.
Below code helps to add a cronjob using linux files.
#!/bin/bash
cron_path=/var/spool/cron/crontabs/root
#cron job to run every 10 min.
echo "*/10 * * * * command to be executed" >> $cron_path
#cron job to run every 1 hour.
echo "0 */1 * * * command to be executed" >> $cron_path
Here is a bash function for adding a command to crontab without duplication
function addtocrontab () {
local frequency=$1
local command=$2
local job="$frequency $command"
cat <(fgrep -i -v "$command" <(crontab -l)) <(echo "$job") | crontab -
}
addtocrontab "0 0 1 * *" "echo hello"
CRON="1 2 3 4 5 /root/bin/backup.sh"
cat < (crontab -l) |grep -v "${CRON}" < (echo "${CRON}")
add -w parameter to grep exact command, without -w parameter adding the cronjob "testing" cause deletion of cron job "testing123"
script function to add/remove cronjobs. no duplication entries :
cronjob_editor () {
# usage: cronjob_editor '<interval>' '<command>' <add|remove>
if [[ -z "$1" ]] ;then printf " no interval specified\n" ;fi
if [[ -z "$2" ]] ;then printf " no command specified\n" ;fi
if [[ -z "$3" ]] ;then printf " no action specified\n" ;fi
if [[ "$3" == add ]] ;then
# add cronjob, no duplication:
( crontab -l | grep -v -F -w "$2" ; echo "$1 $2" ) | crontab -
elif [[ "$3" == remove ]] ;then
# remove cronjob:
( crontab -l | grep -v -F -w "$2" ) | crontab -
fi
}
cronjob_editor "$1" "$2" "$3"
tested :
$ ./cronjob_editor.sh '*/10 * * * *' 'echo "this is a test" > export_file' add
$ crontab -l
$ */10 * * * * echo "this is a test" > export_file
No, there is no option in crontab to modify the cron files.
You have to: take the current cron file (crontab -l > newfile), change it and put the new file in place (crontab newfile).
If you are familiar with perl, you can use this module Config::Crontab.
LLP, Andrea
script function to add cronjobs. check duplicate entries,useable expressions * > "
cronjob_creator () {
# usage: cronjob_creator '<interval>' '<command>'
if [[ -z $1 ]] ;then
printf " no interval specified\n"
elif [[ -z $2 ]] ;then
printf " no command specified\n"
else
CRONIN="/tmp/cti_tmp"
crontab -l | grep -vw "$1 $2" > "$CRONIN"
echo "$1 $2" >> $CRONIN
crontab "$CRONIN"
rm $CRONIN
fi
}
tested :
$ ./cronjob_creator.sh '*/10 * * * *' 'echo "this is a test" > export_file'
$ crontab -l
$ */10 * * * * echo "this is a test" > export_file
source : my brain ;)
Say you're logged in as the user "ubuntu", but you want to add a job to a different user's crontab, like "john", for example. You can do the following:
(sudo crontab -l -u john; echo "* * * * * command") | awk '!x[$0]++' | sudo crontab -u john -
Source for most of this solution: https://www.baeldung.com/linux/create-crontab-script
I was having tons of issues trying to add a job to another user's crontab. It kept duplicating crontabs, or just flat-out deleting them. After some testing, though, I'm confident this line of code will append a new job to a specified user's crontab, non-destructively, including not creating a job that already exists.
I wanted to find an example like this, so maybe it helps:
COMMAND="/var/lib/postgresql/backup.sh"
CRON="0 0 * * *"
USER="postgres"
CRON_FILE="postgres-backup"
# At CRON times, the USER will run the COMMAND
echo "$CRON $USER $COMMAND" | sudo tee /etc/cron.d/$CRON_FILE
echo "Cron job created. Remove /etc/cron.d/$CRON_FILE to stop it."
Debian 8.6. No root.
I can use cron.
I need to check if application ( php ./somescript & ) running in background stopped, and restart it. How can I check it using bash?
Of course, there is ps aux | grep ....., but how do I automate it?
I suggest to take a look at keyword #reboot from man 5 crontab to start a job once at server startup.
One way to go about it would be:
Cron:
* * * * * env DISPLAY=:0 /folder/myscript >/dev/null 2>&1
The env DISPLAY=:0 might not be needed in your case, or it might be needed, depending on your script (note: you might need to adapt this to your case, run echo $DISPLAY to find out your variable on the case).
Script:
#!/bin/bash
testvar="$(ps aux | grep -s "somescript" | grep -sv "grep")"
if [ -z "$testvar" ]; then nohup /folder/somescript &; fi
exit 0
This all could and should be fine tuned to your needs, but I believe this example could serve you well.
Edit: I fixed a small oversight on the code (I added | grep -sv "grep" to get rid of the own grep process of looking for the file from the tesvar results).
Does crontab have an argument for creating cron jobs without using the editor (crontab -e)? If so, what would be the code to create a cron job from a Bash script?
You can add to the crontab as follows:
#write out current crontab
crontab -l > mycron
#echo new cron into cron file
echo "00 09 * * 1-5 echo hello" >> mycron
#install new cron file
crontab mycron
rm mycron
Cron line explaination
* * * * * "command to be executed"
- - - - -
| | | | |
| | | | ----- Day of week (0 - 7) (Sunday=0 or 7)
| | | ------- Month (1 - 12)
| | --------- Day of month (1 - 31)
| ----------- Hour (0 - 23)
------------- Minute (0 - 59)
Source nixCraft.
You may be able to do it on-the-fly
crontab -l | { cat; echo "0 0 0 0 0 some entry"; } | crontab -
crontab -l lists the current crontab jobs, cat prints it, echo prints the new command and crontab - adds all the printed stuff into the crontab file. You can see the effect by doing a new crontab -l.
This shorter one requires no temporary file, it is immune to multiple insertions, and it lets you change the schedule of an existing entry.
Say you have these:
croncmd="/home/me/myfunction myargs > /home/me/myfunction.log 2>&1"
cronjob="0 */15 * * * $croncmd"
To add it to the crontab, with no duplication:
( crontab -l | grep -v -F "$croncmd" ; echo "$cronjob" ) | crontab -
To remove it from the crontab whatever its current schedule:
( crontab -l | grep -v -F "$croncmd" ) | crontab -
Notes:
grep -F matches the string literally, as we do not want to interpret it as a regular expression
We also ignore the time scheduling and only look for the command. This way; the schedule can be changed without the risk of adding a new line to the crontab
Thanks everybody for your help. Piecing together what I found here and elsewhere I came up with this:
The Code
command="php $INSTALL/indefero/scripts/gitcron.php"
job="0 0 * * 0 $command"
cat <(fgrep -i -v "$command" <(crontab -l)) <(echo "$job") | crontab -
I couldn't figure out how to eliminate the need for the two variables without repeating myself.
command is obviously the command I want to schedule. job takes $command and adds the scheduling data. I needed both variables separately in the line of code that does the work.
Details
Credit to duckyflip, I use this little redirect thingy (<(*command*)) to turn the output of crontab -l into input for the fgrep command.
fgrep then filters out any matches of $command (-v option), case-insensitive (-i option).
Again, the little redirect thingy (<(*command*)) is used to turn the result back into input for the cat command.
The cat command also receives echo "$job" (self explanatory), again, through use of the redirect thingy (<(*command*)).
So the filtered output from crontab -l and the simple echo "$job", combined, are piped ('|') over to crontab - to finally be written.
And they all lived happily ever after!
In a nutshell:
This line of code filters out any cron jobs that match the command, then writes out the remaining cron jobs with the new one, effectively acting like an "add" or "update" function.
To use this, all you have to do is swap out the values for the command and job variables.
EDIT (fixed overwriting):
cat <(crontab -l) <(echo "1 2 3 4 5 scripty.sh") | crontab -
There have been a lot of good answers around the use of crontab, but no mention of a simpler method, such as using cron.
Using cron would take advantage of system files and directories located at /etc/crontab, /etc/cron.daily,weekly,hourly or /etc/cron.d/:
cat > /etc/cron.d/<job> << EOF
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root HOME=/
01 * * * * <user> <command>
EOF
In this above example, we created a file in /etc/cron.d/, provided the environment variables for the command to execute successfully, and provided the user for the command, and the command itself. This file should not be executable and the name should only contain alpha-numeric and hyphens (more details below).
To give a thorough answer though, let's look at the differences between crontab vs cron/crond:
crontab -- maintain tables for driving cron for individual users
For those who want to run the job in the context of their user on the system, using crontab may make perfect sense.
cron -- daemon to execute scheduled commands
For those who use configuration management or want to manage jobs for other users, in which case we should use cron.
A quick excerpt from the manpages gives you a few examples of what to and not to do:
/etc/crontab and the files in /etc/cron.d must be owned by root, and must not be group- or other-writable. In contrast to the spool area, the files under /etc/cron.d or the files under /etc/cron.hourly, /etc/cron.daily, /etc/cron.weekly and /etc/cron.monthly may also be symlinks, provided that both the symlink and the file it points to are owned by root. The files under /etc/cron.d do not need to be executable, while the files under /etc/cron.hourly, /etc/cron.daily, /etc/cron.weekly and /etc/cron.monthly do, as they are run by run-parts (see run-parts(8) for more information).
Source: http://manpages.ubuntu.com/manpages/trusty/man8/cron.8.html
Managing crons in this manner is easier and more scalable from a system perspective, but will not always be the best solution.
So, in Debian, Ubuntu, and many similar Debian based distros...
There is a cron task concatenation mechanism that takes a config file, bundles them up and adds them to your cron service running.
You can put a file under the /etc/cron.d/somefilename where somefilename is whatever you want.
sudo echo "0,15,30,45 * * * * ntpdate -u time.nist.gov" >> /etc/cron.d/vmclocksync
Let's disassemble this:
sudo - because you need elevated privileges to change cron configs under the /etc directory
echo - a vehicle to create output on std out. printf, cat... would work as well
" - use a doublequote at the beginning of your string, you're a professional
0,15,30,45 * * * * - the standard cron run schedule, this one runs every 15 minutes
ntpdate -u time.nist.gov - the actual command I want to run
" - because my first double quotes needs a buddy to close the line being output
>> - the double redirect appends instead of overwrites*
/etc/cron.d/vmclocksync - vmclocksync is the filename I've chosen, it goes in /etc/cron.d/
* if we used the > redirect, we could guarantee we only had one task entry. But, we would be at risk of blowing away any other rules in an existing file. You can decide for yourself if possible destruction with > is right or possible duplicates with >> are for you. Alternatively, you could do something convoluted or involved to check if the file name exists, if there is anything in it, and whether you are adding any kind of duplicate-- but, I have stuff to do and I can't do that for you right now.
For a nice quick and dirty creation/replacement of a crontab from with a BASH script, I used this notation:
crontab <<EOF
00 09 * * 1-5 echo hello
EOF
Chances are you are automating this, and you don't want a single job added twice.
In that case use:
__cron="1 2 3 4 5 /root/bin/backup.sh"
cat <(crontab -l) |grep -v "${__cron}" <(echo "${__cron}")
This only works if you're using BASH. I'm not aware of the correct DASH (sh) syntax.
Update: This doesn't work if the user doesn't have a crontab yet. A more reliable way would be:
(crontab -l ; echo "1 2 3 4 5 /root/bin/backup.sh") | sort - | uniq - | crontab -
Alternatively, if your distro supports it, you could also use a separate file:
echo "1 2 3 4 5 /root/bin/backup.sh" |sudo tee /etc/crond.d/backup
Found those in another SO question.
echo "0 * * * * docker system prune --force >/dev/null 2>&1" | sudo tee /etc/cron.daily/dockerprune
A variant which only edits crontab if the desired string is not found there:
CMD="/sbin/modprobe fcpci"
JOB="#reboot $CMD"
TMPC="mycron"
grep "$CMD" -q <(crontab -l) || (crontab -l>"$TMPC"; echo "$JOB">>"$TMPC"; crontab "$TMPC")
(2>/dev/null crontab -l ; echo "0 3 * * * /usr/local/bin/certbot-auto renew") | crontab -
cat <(crontab -l 2>/dev/null) <(echo "0 3 * * * /usr/local/bin/certbot-auto renew") | crontab -
#write out current crontab
crontab -l > mycron 2>/dev/null
#echo new cron into cron file
echo "0 3 * * * /usr/local/bin/certbot-auto renew" >> mycron
#install new cron file
crontab mycron
rm mycron
If you're using the Vixie Cron, e.g. on most Linux distributions, you can just put a file in /etc/cron.d with the individual cronjob.
This only works for root of course. If your system supports this you should see several examples in there. (Note the username included in the line, in the same syntax as the old /etc/crontab)
It's a sad misfeature in cron that there is no way to handle this as a regular user, and that so many cron implementations have no way at all to handle this.
My preferred solution to this would be this:
(crontab -l | grep . ; echo -e "0 4 * * * myscript\n") | crontab -
This will make sure you are handling the blank new line at the bottom correctly. To avoid issues with crontab you should usually end the crontab file with a blank new line. And the script above makes sure it first removes any blank lines with the "grep ." part, and then add in a new blank line at the end with the "\n" in the end of the script. This will also prevent getting a blank line above your new command if your existing crontab file ends with a blank line.
Bash script for adding cron job without the interactive editor.
Below code helps to add a cronjob using linux files.
#!/bin/bash
cron_path=/var/spool/cron/crontabs/root
#cron job to run every 10 min.
echo "*/10 * * * * command to be executed" >> $cron_path
#cron job to run every 1 hour.
echo "0 */1 * * * command to be executed" >> $cron_path
Here is a bash function for adding a command to crontab without duplication
function addtocrontab () {
local frequency=$1
local command=$2
local job="$frequency $command"
cat <(fgrep -i -v "$command" <(crontab -l)) <(echo "$job") | crontab -
}
addtocrontab "0 0 1 * *" "echo hello"
CRON="1 2 3 4 5 /root/bin/backup.sh"
cat < (crontab -l) |grep -v "${CRON}" < (echo "${CRON}")
add -w parameter to grep exact command, without -w parameter adding the cronjob "testing" cause deletion of cron job "testing123"
script function to add/remove cronjobs. no duplication entries :
cronjob_editor () {
# usage: cronjob_editor '<interval>' '<command>' <add|remove>
if [[ -z "$1" ]] ;then printf " no interval specified\n" ;fi
if [[ -z "$2" ]] ;then printf " no command specified\n" ;fi
if [[ -z "$3" ]] ;then printf " no action specified\n" ;fi
if [[ "$3" == add ]] ;then
# add cronjob, no duplication:
( crontab -l | grep -v -F -w "$2" ; echo "$1 $2" ) | crontab -
elif [[ "$3" == remove ]] ;then
# remove cronjob:
( crontab -l | grep -v -F -w "$2" ) | crontab -
fi
}
cronjob_editor "$1" "$2" "$3"
tested :
$ ./cronjob_editor.sh '*/10 * * * *' 'echo "this is a test" > export_file' add
$ crontab -l
$ */10 * * * * echo "this is a test" > export_file
No, there is no option in crontab to modify the cron files.
You have to: take the current cron file (crontab -l > newfile), change it and put the new file in place (crontab newfile).
If you are familiar with perl, you can use this module Config::Crontab.
LLP, Andrea
script function to add cronjobs. check duplicate entries,useable expressions * > "
cronjob_creator () {
# usage: cronjob_creator '<interval>' '<command>'
if [[ -z $1 ]] ;then
printf " no interval specified\n"
elif [[ -z $2 ]] ;then
printf " no command specified\n"
else
CRONIN="/tmp/cti_tmp"
crontab -l | grep -vw "$1 $2" > "$CRONIN"
echo "$1 $2" >> $CRONIN
crontab "$CRONIN"
rm $CRONIN
fi
}
tested :
$ ./cronjob_creator.sh '*/10 * * * *' 'echo "this is a test" > export_file'
$ crontab -l
$ */10 * * * * echo "this is a test" > export_file
source : my brain ;)
Say you're logged in as the user "ubuntu", but you want to add a job to a different user's crontab, like "john", for example. You can do the following:
(sudo crontab -l -u john; echo "* * * * * command") | awk '!x[$0]++' | sudo crontab -u john -
Source for most of this solution: https://www.baeldung.com/linux/create-crontab-script
I was having tons of issues trying to add a job to another user's crontab. It kept duplicating crontabs, or just flat-out deleting them. After some testing, though, I'm confident this line of code will append a new job to a specified user's crontab, non-destructively, including not creating a job that already exists.
I wanted to find an example like this, so maybe it helps:
COMMAND="/var/lib/postgresql/backup.sh"
CRON="0 0 * * *"
USER="postgres"
CRON_FILE="postgres-backup"
# At CRON times, the USER will run the COMMAND
echo "$CRON $USER $COMMAND" | sudo tee /etc/cron.d/$CRON_FILE
echo "Cron job created. Remove /etc/cron.d/$CRON_FILE to stop it."
There are some commands I'd like to run on a grid using qsub (SGE 8.1.3, CentOS 5.9) that need to use a pipe (|) or a redirect (>). For example, let's say I have to parallelize the command
echo 'hello world' > hello.txt
(Obviously a simplified example: in reality I might need to redirect the output of a program like bowtie directly to samtools). If I did:
qsub echo 'hello world' > hello.txt
the resulting content of hello.txt would look like
Your job 123454321 ("echo") has been submitted
Similarly if I used a pipe (echo "hello world" | myprogram), that message is all that would be passed to myprogram, not the actual stdout.
I'm aware I could write a small bash script that each contain the command with the pipe/redirect, and then do qsub ./myscript.sh. However, I'm trying to run many parallelized jobs at the same time using a script, so I'd have to write many such bash scripts each with a slightly different command. When scripting this solution can start to feel very hackish. An example of such a script in Python:
for i, (infile1, infile2, outfile) in enumerate(files):
command = ("bowtie -S %s %s | " +
"samtools view -bS - > %s\n") % (infile1, infile2, outfile)
script = "job" + str(counter) + ".sh"
open(script, "w").write(command)
os.system("chmod 755 %s" % script)
os.system("qsub -cwd ./%s" % script)
This is frustrating for a few reasons, among them that my program can't even delete the many jobXX.sh scripts afterwards to clean up after itself, since I don't know how long the job will be waiting in the queue, and the script has to be there when the job starts.
Is there a way to provide my full echo 'hello world' > hello.txt command to qsub without having to create another file containing the command?
You can do this by turning it into a bash -c command, which lets you put the | in a quoted statement:
qsub bash -c "cmd <options> | cmd2 <options>"
As #spuder has noted in the comments, it seems that in other versions of qsub (not SGE 8.1.3, which I'm using), one can solve the problem with:
echo "cmd <options> | cmd2 <options>" | qsub
as well.
Although my answer is a bit late I am adding it for any incoming viewers. To use a pipe/direct and submit that as a qsub job you need to do a couple of things. But first, using qsub at the end of a pipe like you're doing will only result in one job being sent to the queue (i.e. Your code will run serially rather than get parallelized).
Run qsub with enabling binary mode since the default qsub behavior rather expects compiled code. For that you use the "-b y" flag to qsub and you'll avoid any errors of the sort "command required for a binary mode" or "script length does not match declared length".
echo each call to qsub and then pipe that to shell.
Suppose you have a file params-query.txt which hold several bowtie commands and piped calls to samtools of the following form:
bowtie -q query -1 param1 -2 param2 ... | samtools ...
To send each query as a separate job first prepare your command line units from STDIN through xargs STDIN. Notice the quotes around the braces are important if you are submitting a command of piped parts. That way your entire query is treated a single unit.
cat params-query.txt | xargs -i echo qsub -b y -o output_log -e error_log -N job_name \"{}\" | sh
If that didn't work as expected then you're probably better off generating an intermediate output between bowtie and samtools before calling samtools to accept that intermediate output. You won't need to change the qsub call through xargs but the code in params-query.txt should look like:
bowtie -q query -o intermediate_query_out -1 param1 -2 param2 && samtools read_from_intermediate_query_out
This page has interesting qsub tricks you might like
grep http *.job | awk -F: '{print $1}' | sort -u | xargs -I {} qsub {}
This
#!/bin/bash
if [ `ps -ef | grep "91.34.124.35" | grep -v grep | wc -l` -eq 0 ]; then sh home/asfd.sh; fi
or this?
ps -ef | grep "91\.34\.124\.35" | grep -v grep > /dev/null
if [ "$?" -ne "0" ]
then
sh home/asfd.sh
else
echo "Process is running fine"
fi
Hello, how can I write a shell script that looks in running processes and if there isn't a process name CONTAINING 91.34.124.35 then execute a file in a certain place and I want to make this run every 30 seconds in a continuous loop, I think there was a sleep command.
you can't use cron since on the implementation I know the smallest unit is one minute. You can use sleep but then your process will always be running (with cron it will started every time).
To use sleep just
while true ; do
if ! pgrep -f '91\.34\.124\.35' > /dev/null ; then
sh /home/asfd.sh
fi
sleep 30
done
If your pgrep has the option -q to suppress output (as on BSD) you can also use pgrep -q without redirecting the output to /dev/null
First of all, you should be able to reduce your script to simply
if ! pgrep "91\.34\.124\.35" > /dev/null; then ./your_script.sh; fi
To run this every 30 seconds via cron (because cron only runs every minute) you need 2 entries - one to run the command, another to delay for 30 seconds before running the same command again. For example:
* * * * * root if ! pgrep "91\.34\.124\.35" > /dev/null; then ./your_script.sh; fi
* * * * * root sleep 30; if ! pgrep "91\.34\.124\.35" > /dev/null; then ./your_script.sh; fi
To make this cleaner, you might be able to first store the command in a variable and use it for both entries. (I haven't tested this).
CHECK_COMMAND="if ! pgrep '91\.34\.124\.35' > /dev/null; then ./your_script.sh; fi"
* * * * * root eval "$CHECK_COMMAND"
* * * * * root sleep 30; eval "$CHECK_COMMAND"
p.s. The above assumes you're adding that to /etc/crontab. To use it in a user's crontab (crontab -e) simply leave out the username (root) before the command.
I would suggest using watch:
watch -n 30 launch_my_script_if_process_is_dead.sh
Either way is fine, you can save it in a .sh file and add it to the crontab to run every 30 seconds. Let me know if you want to know how to use crontab.
Try this:
if ps -ef | grep "91\.34\.124\.35" | grep -v grep > /dev/null
then
sh home/asfd.sh
else
echo "Process is running fine"
fi
No need to use test. if itself will examine the exit code.
You can save your script in file name, myscript.sh
then you can run your script through cron,
*/30 * * * * /full/path/for/myscript.sh
or you can use while
# cat script1.sh
#!/bin/bash
while true; do /bin/sh /full/path/for/myscript.sh ; sleep 30; done &
# ./script1.sh
Thanks.
I have found deamonizing critical scripts very effective.
http://cr.yp.to/daemontools.html
You can use monit for this task. See docu. It is available on most linux distributions and has a straightforward config. Find some examples in this post
For your app it will look something like
check process myprocessname
matching "91\.34\.124\.35"
start program = "/home/asfd.sh"
stop program = "/home/dfsa.sh"
If monit is not available on your platform you can use supervisord.
I also found this question very similar Repeat command automatically in Linux. It suggests to use watch.
Use cron for the "loop every 30 seconds" part.