I would like to run a specific script at a certain time (only once!). If I run it normally like this:
marc#Marc-Linux:~/tennis_betting_strategy1/wrappers$ Rscript write_csv2.R
It does work. I however would like to program it in a cronjob to run at 10:50 and therefore did the following:
50 10 11 05 * Rscript ~/csv_file/write_csv.R
This does not seem to work however. Any thoughts where I go wrong? These are the details of the cron package im
using:
PID COMMAND
1015 cron
My system time also checks out:
marc#Marc-Linux:~/tennis_betting_strategy1/wrappers$ date
wo mei 11 10:56:46 CEST 2016
There is a special tool for running commands only once - at.
With at you can schedule a command like this:
at 09:05 am today
at> enter you commands...
Note, you'll need the atd daemon running.
Your crontab entry looks okay, however. I'd suggest checking if the cron daemon is running(exact daemon name depends on the cron package; it could be cron, crond, or vixie-cron, for instance). One way to check if the daemon is running is to use the ps command, e.g.:
$ ps -C cron -o pid,args
PID COMMAND
306 /usr/sbin/cron
Some advices.
Read more about the PATH variable. Notice that it is set differently in interactive shells (see your ~/.bashrc) and in cron or at jobs. See also this about Rscript.
Replace your command by a shell script, e.g. in ~/bin/myrscriptjob.sh
That myrscriptjob.sh file should start with #!/bin/sh
Be sure to make that shell script executable:
chmod u+x ~/bin/myrscriptjob.sh
Add some logging in your shell script, near the start; either use logger(1) or at least some date(1) command suitably redirected, or even both:
#!/bin/sh
# file myrscriptjob.sh
/bin/date +"myrscriptjob starting %c %n" > /tmp/myrscriptjob.start
/usr/bin/logger -t myrscript job starting $$
/usr/local/bin/Rscript $HOME/csv_file/write_csv.R
in the last line above, replace /usr/local/bin/Rscript by the output of which Rscript done in some interactive terminal.
Notice that you should not use ~ (but replace them with $HOME when appropriate) in shell scripts.
Finally, use at to run your script once. If you want to run it periodically in a crontab job, give the absolute path, e.g.
5 09 11 05 * $HOME/bin/myrscriptjob.sh
and check in /tmp/myrscriptjob.start and in your system log if it has started successfully.
BTW, in your myrscriptjob.sh script, you might replace the first line #!/bin/sh with #!/bin/sh -vx (then the shell is verbose about execution, and cron or at will send you some email). See dash(1), bash(1), execve(2)
Use full path (started from /) for both Rscript and write_csv2.R. Check sample command as follows
/tmp/myscript/Rscript /tmp/myfile/write_csv2.R
Ensure you have execution permission of Rscript and write permission in folder where write_csv2.R will be created(/tmp/myfile)
I am going to create a crontab task to schedule my task.
My /etc/crontab looks like this,
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/
17 15 * * * root sh /opt/app/tool/ReviewSummaryTool/runf.sh
The task script runf.sh has content looks like this,
#!/usr/bin/env bash
java -Dhostname=$(hostname) -jar ReviewSummaryTool.jar -full
But the crontab task could not be executed (I checked the output log) when time is arrived.
However, the task script could be execute by command below,
sh /opt/app/tool/ReviewSummaryTool/runf.sh
And I checked the log of crontab at /var/log/cron and it seems that the task has already been executed. See brief log content below,
Aug 31 15:17:01 SSECBIGDATA01 crond[1677]: (*system*) RELOAD (/etc/crontab)
Aug 31 15:17:01 SSECBIGDATA01 CROND[29248]: (root) CMD (sh /opt/app/tool/ReviewSummaryTool/runf.sh)
Now, I have no idea what's wrong with my configuration. My operating system is CentOS.
Any help would be appreciate. Thanks in advance.
For debugging purposes replace sh with /bin/sh -vx in the crontab entry:
17 15 * * * root /bin/sh -vx /opt/app/tool/ReviewSummaryTool/runf.sh
Then the trace of the executed script will be printed, so emailed to you.
And add logger commands inside your runf.sh script, e.g. have it be
#!/bin/sh
logger -t runfjob start of runf pid $$ in $(pwd) on host $(hostname)
java -Dhostname=$(hostname) -jar ReviewSummaryTool.jar -full
logger -t runfjob end of runf pid $$
The logger command makes entries to the system log using syslog(3). You should find these messages under /var/log, perhaps in /var/log/messages or /var/log/syslog etc...
I strongly suggest to put the full path of the ReviewSummaryTool.jar file in the java command of your runf.sh script. It is likely that your cronjob is run in a current directory not having that file. Or perhaps put a cd command before the java one.
Be sure that the $PATH is correct and that java is found there.
I'm trying to use a cronjob to run a ruby script (Using Rails3 runner) with the following Cronjobs:
#!/bin/bash
0-59 * * * * echo 'script test'
# Begin Whenever generated tasks for: test1
* * * * * /bin/bash -l -c '/home/administrator/test1/script/rails runner /home/administrator/test1/app/create_flag.rb >> /home/administrator/test1/test.log 2>&1'
# End Whenever generated tasks for: test1
test1 is the name of the Rails3 project folder.
the "echo 'script test'" was added as a test, but neither seems to be executing. I'm currently using Ubuntu 10.04 LTS.
Have I written the cronjob incorrectly?
Crontab file is not a shell script. So you don't need #!/bin/bash at the beginning of the file. Plus, spaces there are suspicious. Try something like this:
SHELL=/bin/bash
MAILTO=administrator#localhost
BASH_ENV=/home/administrator/.bash_profile
* * * * * /home/administrator/test1/script/rails runner /home/administrator/test1/app/create_flag.rb >> /home/administrator/test1/test.log 2>&1'
Plus, make sure you call crontab -e as administrator to edit the crontab file.
You need to specify the user which runs the commands (you can see the format here. Also the echo will output 'script test' to what? If you want a test try doing a touch on a file, so you can physically see the action of the cron job.
Cron does not use your user environment, so it will not have the same path set that you have. This means that you should use absolute paths for commands.
I have a crontab running every hour. The user running it has environment variabless in the .bash_profile that work when the user runs the job from the terminal, however, obviously these don't get picked up by crontab when it runs.
I've tried setting them in .profile and .bashrc but they still don't seem to get picked up. Does anyone know where I can put environment vars that crontab can pick up?
You can define environment variables in the crontab itself when running crontab -e from the command line.
LANG=nb_NO.UTF-8
LC_ALL=nb_NO.UTF-8
# m h dom mon dow command
* * * * * sleep 5s && echo "yo"
This feature is only available to certain implementations of cron. Ubuntu and Debian currently use vixie-cron which allows these to be declared in the crontab file (also GNU mcron).
Archlinux and RedHat use cronie which does not allow environment variables to be declared and will throw syntax errors in the cron.log. Workaround can be done per-entry:
# m h dom mon dow command
* * * * * export LC_ALL=nb_NO.UTF-8; sleep 5s && echo "yo"
I got one more solution for this problem:
0 5 * * * . $HOME/.profile; /path/to/command/to/run
In this case it will pick all the environment variable defined in your $HOME/.profile file.
Of course $HOME is also not set, you have to replace it with the full path of your $HOME.
Setting vars in /etc/environment also worked for me in Ubuntu. As of 12.04, variables in /etc/environment are loaded for cron.
Have 'cron' run a shell script that sets the environment before running the command.
Always.
# #(#)$Id: crontab,v 4.2 2007/09/17 02:41:00 jleffler Exp $
# Crontab file for Home Directory for Jonathan Leffler (JL)
#-----------------------------------------------------------------------------
#Min Hour Day Month Weekday Command
#-----------------------------------------------------------------------------
0 * * * * /usr/bin/ksh /work1/jleffler/bin/Cron/hourly
1 1 * * * /usr/bin/ksh /work1/jleffler/bin/Cron/daily
23 1 * * 1-5 /usr/bin/ksh /work1/jleffler/bin/Cron/weekday
2 3 * * 0 /usr/bin/ksh /work1/jleffler/bin/Cron/weekly
21 3 1 * * /usr/bin/ksh /work1/jleffler/bin/Cron/monthly
The scripts in ~/bin/Cron are all links to a single script, 'runcron', which looks like:
: "$Id: runcron.sh,v 2.1 2001/02/27 00:53:22 jleffler Exp $"
#
# Commands to be performed by Cron (no debugging options)
# Set environment -- not done by cron (usually switches HOME)
. $HOME/.cronfile
base=`basename $0`
cmd=${REAL_HOME:-/real/home}/bin/$base
if [ ! -x $cmd ]
then cmd=${HOME}/bin/$base
fi
exec $cmd ${#:+"$#"}
(Written using an older coding standard - nowadays, I'd use a shebang '#!' at the start.)
The '~/.cronfile' is a variation on my profile for use by cron - rigorously non-interactive and no echoing for the sake of being noisy. You could arrange to execute the .profile and so on instead. (The REAL_HOME stuff is an artefact of my environment - you can pretend it is the same as $HOME.)
So, this code reads the appropriate environment and then executes the non-Cron version of the command from my home directory. So, for example, my 'weekday' command looks like:
: "#(#)$Id: weekday.sh,v 1.10 2007/09/17 02:42:03 jleffler Exp $"
#
# Commands to be done each weekday
# Update ICSCOPE
n.updics
The 'daily' command is simpler:
: "#(#)$Id: daily.sh,v 1.5 1997/06/02 22:04:21 johnl Exp $"
#
# Commands to be done daily
# Nothing -- most things are done on weekdays only
exit 0
If you start the scripts you are executing through cron with:
#!/bin/bash -l
They should pick up your ~/.bash_profile environment variables
Expanding on #carestad example, which I find easier, is to run the script with cron and have the environment in the script.
In crontab -e file:
SHELL=/bin/bash
*/1 * * * * $HOME/cron_job.sh
In cron_job.sh file:
#!/bin/bash
source $HOME/.bash_profile
some_other_cmd
Any command after the source of .bash_profile will have your environment as if you logged in.
Whatever you set in crontab will be available in the cronjobs, both directly and using the variables in the scripts.
Use them in the definition of the cronjob
You can configure crontab so that it sets variables that then the can cronjob use:
$ crontab -l
myvar="hi man"
* * * * * echo "$myvar. date is $(date)" >> /tmp/hello
Now the file /tmp/hello shows things like:
$ cat /tmp/hello
hi man. date is Thu May 12 12:10:01 CEST 2016
hi man. date is Thu May 12 12:11:01 CEST 2016
Use them in the script run by cronjob
You can configure crontab so that it sets variables that then the scripts can use:
$ crontab -l
myvar="hi man"
* * * * * /bin/bash /tmp/myscript.sh
And say script /tmp/myscript.sh is like this:
echo "Now is $(date). myvar=$myvar" >> /tmp/myoutput.res
It generates a file /tmp/myoutput.res showing:
$ cat /tmp/myoutput.res
Now is Thu May 12 12:07:01 CEST 2016. myvar=hi man
Now is Thu May 12 12:08:01 CEST 2016. myvar=hi man
...
For me I had to set the environment variable for a php application. I resolved it by adding the following code to my crontab.
$ sudo crontab -e
crontab:
ENVIRONMENT_VAR=production
* * * * * /home/deploy/my_app/cron/cron.doSomethingWonderful.php
and inside doSomethingWonderful.php I could get the environment value with:
<?php
echo $_SERVER['ENVIRONMENT_VAR']; # => "production"
I hope this helps!
Instead of
0 * * * * sh /my/script.sh
Use bash -l -c
0 * * * * bash -l -c 'sh /my/script.sh'
You can also prepend your command with env to inject Environment variables like so:
0 * * * * env VARIABLE=VALUE /usr/bin/mycommand
Expanding on #Robert Brisita has just expand , also if you don't want to set up all the variables of the profile in the script, you can select the variables to export on the top of the script
In crontab -e file:
SHELL=/bin/bash
*/1 * * * * /Path/to/script/script.sh
In script.sh
#!/bin/bash
export JAVA_HOME=/path/to/jdk
some-other-command
I'm using Oh-my-zsh in my macbook so I've tried many things to get the crontab task runs but finally, my solution was prepending the .zshrc before the command to run.
*/30 * * * * . $HOME/.zshrc; node /path/for/my_script.js
This task runs every 30 minutes and uses .zshrc profile to execute my node command.
Don't forget to use the dot before the $HOME var.
I tried most of the provided solutions, but nothing worked at first. It turns out, though, that it wasn't the solutions that failed to work. Apparently, my ~/.bashrc file starts with the following block of code:
case $- in
*i*) ;;
*) return;;
esac
This basically is a case statement that checks the current set of options in the current shell to determine that the shell is running interactively.
If the shell happens to be running interactively, then it moves on to sourcing the ~/.bashrc file.
However, in a shell invoked by cron, the $- variable doesn't contain the i value which indicates interactivity.
Therefore, the ~/.bashrc file never gets sourced fully. As a result, the environment variables never got set.
If this happens to be your issue, feel free to comment out the block of code as follows and try again:
# case $- in
# *i*) ;;
# *) return;;
# esac
I hope this turns out useful
Unfortunately, crontabs have a very limited environment variables scope, thus you need to export them every time the corntab runs.
An easy approach would be the following example, suppose you've your env vars in a file called env, then:
* * * * * . ./env && /path/to_your/command
this part . ./env will export them and then they're used within the same scope of your command
Another way - inspired by this this answer - to "inject" variables is the following (fcron example):
%daily 00 12 \
set -a; \
. /path/to/file/containing/vars; \
set +a; \
/path/to/script/using/vars
From help set:
-a Mark variables which are modified or created for export.
Using + rather than - causes these flags to be turned off.
So everything in between set - and set + gets exported to env and is then available for other scripts, etc. Without using set the variables get sourced but live in set only.
Aside from that it's also useful to pass variables when a program requires a non-root account to run but you'd need some variables inside that other user's environment. Below is an example passing in nullmailer vars to format the e-mail header:
su -s /bin/bash -c "set -a; \
. /path/to/nullmailer-vars; \
set +a; \
/usr/sbin/logcheck" logcheck
All the above solutions work fine.
It will create issues when there are any special characters in your environment variable.
I have found the solution:
eval $(printenv | awk -F= '{print "export " "\""$1"\"""=""\""$2"\"" }' >> /etc/profile)
For me I had to specify path in my NodeJS file.
// did not work!!!!!
require('dotenv').config()
instead
// DID WORK!!
require('dotenv').config({ path: '/full/custom/path/to/your/.env' })
I found this issue while looking at a similar problem that matched the title, but I am stuck with the environment file syntax that systemd or docker use:
FOO=bar
BAZ=qux
This won't work for Vishal's excellent answer because they aren't bash scripts (note the lack of export).
The solution I've used is to read each line into xargs and export them before running the command:
0 5 * * * export $(xargs < $HOME/.env); /path/to/command/to/run
Set Globally env
sudo sh -c "echo MY_GLOBAL_ENV_TO_MY_CURRENT_DIR=$(pwd)" >> /etc/environment"
Add scheduled job to start a script
crontab -e
*/5 * * * * sh -c "$MY_GLOBAL_ENV_TO_MY_CURRENT_DIR/start.sh"
=)
what worked for me (debian based):
create a file with all the needed env var :
#!/bin/bash
env | grep VAR1= > /etc/environment
env | grep VAR2= >> /etc/environment
env | grep VAR3= >> /etc/environment
then build the crontab content, by calling the env file before calling the script that needs it, therefore start the cron service
(crontab -l ; echo '* * * * * . /etc/environment; /usr/local/bin/python /mycode.py >> /var/log/cron-1.log 2>&1') | crontab
service cron start
nb : for python use case, be sure to call the whole python path, else wrong python could be invocated, generating non-sense syntax error
I'm using a basic shell script to log the results of top, netstat, ps and free every minute.
This is the script:
/scripts/logtop:
TERM=vt100
export TERM
time=$(date)
min=${time:14:2}
top -b -n 1 > /var/log/systemCheckLogs/$min
netstat -an >> /var/log/systemCheckLogs/$min
ps aux >> /var/log/systemCheckLogs/$min
free >> /var/log/systemCheckLogs/$min
echo "Message Content: $min" | mail -s "Ran System Check script" email#domain.com
exit 0
When I run this script directly it works fine. It creates the files and puts them in /var/log/systemCheckLogs/ and then sends me an email.
I can't, however, get it to work when trying to get cron to do it every minute.
I tried putting it in /var/spool/cron/root like so:
* * * * * /scripts/logtop > /dev/null 2>&1
and it never executes
I also tried putting it in /var/spool/cron/myservername and also like so:
* * * * * /scripts/logtop > /dev/null 2>&1
it'll run every minute, but nothing gets created in systemCheckLogs.
Is there a reason it works when I run it but not when cron runs it?
Also, here's what the permissions look like:
-rwxrwxrwx 1 root root 326 Jul 21 01:53 logtop
drwxr-xr-x 2 root root 4096 Jul 21 01:51 systemCheckLogs
Normally crontabs are kept in "/var/spool/cron/crontabs/". Also, normally, you update it with the crontab command as this HUPs crond after you're done and it'll make sure the file gets in the correct place.
Are you using the crontab command to create the cron entry? crontab to import a file directly. crontab -e to edit the current crontab with $EDITOR.
All jobs run by cron need the interpreter listed at the top, so cron knows how to run them.
I can't tell if you just omitted that line or if it is not in your script.
For example,
#!/bin/bash
echo "Test cron jon"
When running from /var/spool/cron/root, it may be failing because cron is not configured to run for root. On linux, root cron jobs are typically run from /etc/crontab rather than from /var/spool/cron.
When running from /var/spool/cron/myservername, you probably have a permissions problem. Don't redirect the error to /dev/null -- capture them and examine.
Something else to be aware of, cron doesn't initialize the full run environment, which can sometimes mean you can run it just fine from a fully logged-in shell, but it doesn't behave the same from cron.
In the case of above, you don't have a "#!/bin/shell" up top in your script. If root is configured to use something like a regular bourne shell or cshell, the syntax you use to populate your variables will not work. This would explain why it would run, but not populate your files. So if you need it to be ksh, "#!/bin/ksh". It's generally best not to trust the environment to keep these things sane. If you need your profile run the do a ". ~/.profile" up front as well. Or a quick and dirty way to get your relatively full env is to do it from su as such "* * * * * su - root -c "/path/to/script" > /dev/null 2>&1
Just some things I've picked up over the years. You're definitely expecting a ksh based on your syntax, so you might want to be sure it's using it.
Thanks for the tips... used a little bit of each answer to get to the bottom of this.
I did have the interpreter at the top (wasn't shown here), but may have been wrong.
Am using #!/bin/bash now and that works.
Also had to tinker with the permissions of the directory the log files are being dumped in to get things working.