Where can I set environment variables that crontab will use? - linux

I have a crontab running every hour. The user running it has environment variabless in the .bash_profile that work when the user runs the job from the terminal, however, obviously these don't get picked up by crontab when it runs.
I've tried setting them in .profile and .bashrc but they still don't seem to get picked up. Does anyone know where I can put environment vars that crontab can pick up?

You can define environment variables in the crontab itself when running crontab -e from the command line.
LANG=nb_NO.UTF-8
LC_ALL=nb_NO.UTF-8
# m h dom mon dow command
* * * * * sleep 5s && echo "yo"
This feature is only available to certain implementations of cron. Ubuntu and Debian currently use vixie-cron which allows these to be declared in the crontab file (also GNU mcron).
Archlinux and RedHat use cronie which does not allow environment variables to be declared and will throw syntax errors in the cron.log. Workaround can be done per-entry:
# m h dom mon dow command
* * * * * export LC_ALL=nb_NO.UTF-8; sleep 5s && echo "yo"

I got one more solution for this problem:
0 5 * * * . $HOME/.profile; /path/to/command/to/run
In this case it will pick all the environment variable defined in your $HOME/.profile file.
Of course $HOME is also not set, you have to replace it with the full path of your $HOME.

Setting vars in /etc/environment also worked for me in Ubuntu. As of 12.04, variables in /etc/environment are loaded for cron.

Have 'cron' run a shell script that sets the environment before running the command.
Always.
# #(#)$Id: crontab,v 4.2 2007/09/17 02:41:00 jleffler Exp $
# Crontab file for Home Directory for Jonathan Leffler (JL)
#-----------------------------------------------------------------------------
#Min Hour Day Month Weekday Command
#-----------------------------------------------------------------------------
0 * * * * /usr/bin/ksh /work1/jleffler/bin/Cron/hourly
1 1 * * * /usr/bin/ksh /work1/jleffler/bin/Cron/daily
23 1 * * 1-5 /usr/bin/ksh /work1/jleffler/bin/Cron/weekday
2 3 * * 0 /usr/bin/ksh /work1/jleffler/bin/Cron/weekly
21 3 1 * * /usr/bin/ksh /work1/jleffler/bin/Cron/monthly
The scripts in ~/bin/Cron are all links to a single script, 'runcron', which looks like:
: "$Id: runcron.sh,v 2.1 2001/02/27 00:53:22 jleffler Exp $"
#
# Commands to be performed by Cron (no debugging options)
# Set environment -- not done by cron (usually switches HOME)
. $HOME/.cronfile
base=`basename $0`
cmd=${REAL_HOME:-/real/home}/bin/$base
if [ ! -x $cmd ]
then cmd=${HOME}/bin/$base
fi
exec $cmd ${#:+"$#"}
(Written using an older coding standard - nowadays, I'd use a shebang '#!' at the start.)
The '~/.cronfile' is a variation on my profile for use by cron - rigorously non-interactive and no echoing for the sake of being noisy. You could arrange to execute the .profile and so on instead. (The REAL_HOME stuff is an artefact of my environment - you can pretend it is the same as $HOME.)
So, this code reads the appropriate environment and then executes the non-Cron version of the command from my home directory. So, for example, my 'weekday' command looks like:
: "#(#)$Id: weekday.sh,v 1.10 2007/09/17 02:42:03 jleffler Exp $"
#
# Commands to be done each weekday
# Update ICSCOPE
n.updics
The 'daily' command is simpler:
: "#(#)$Id: daily.sh,v 1.5 1997/06/02 22:04:21 johnl Exp $"
#
# Commands to be done daily
# Nothing -- most things are done on weekdays only
exit 0

If you start the scripts you are executing through cron with:
#!/bin/bash -l
They should pick up your ~/.bash_profile environment variables

Expanding on #carestad example, which I find easier, is to run the script with cron and have the environment in the script.
In crontab -e file:
SHELL=/bin/bash
*/1 * * * * $HOME/cron_job.sh
In cron_job.sh file:
#!/bin/bash
source $HOME/.bash_profile
some_other_cmd
Any command after the source of .bash_profile will have your environment as if you logged in.

Whatever you set in crontab will be available in the cronjobs, both directly and using the variables in the scripts.
Use them in the definition of the cronjob
You can configure crontab so that it sets variables that then the can cronjob use:
$ crontab -l
myvar="hi man"
* * * * * echo "$myvar. date is $(date)" >> /tmp/hello
Now the file /tmp/hello shows things like:
$ cat /tmp/hello
hi man. date is Thu May 12 12:10:01 CEST 2016
hi man. date is Thu May 12 12:11:01 CEST 2016
Use them in the script run by cronjob
You can configure crontab so that it sets variables that then the scripts can use:
$ crontab -l
myvar="hi man"
* * * * * /bin/bash /tmp/myscript.sh
And say script /tmp/myscript.sh is like this:
echo "Now is $(date). myvar=$myvar" >> /tmp/myoutput.res
It generates a file /tmp/myoutput.res showing:
$ cat /tmp/myoutput.res
Now is Thu May 12 12:07:01 CEST 2016. myvar=hi man
Now is Thu May 12 12:08:01 CEST 2016. myvar=hi man
...

For me I had to set the environment variable for a php application. I resolved it by adding the following code to my crontab.
$ sudo crontab -e
crontab:
ENVIRONMENT_VAR=production
* * * * * /home/deploy/my_app/cron/cron.doSomethingWonderful.php
and inside doSomethingWonderful.php I could get the environment value with:
<?php
echo $_SERVER['ENVIRONMENT_VAR']; # => "production"
I hope this helps!

Instead of
0 * * * * sh /my/script.sh
Use bash -l -c
0 * * * * bash -l -c 'sh /my/script.sh'

You can also prepend your command with env to inject Environment variables like so:
0 * * * * env VARIABLE=VALUE /usr/bin/mycommand

Expanding on #Robert Brisita has just expand , also if you don't want to set up all the variables of the profile in the script, you can select the variables to export on the top of the script
In crontab -e file:
SHELL=/bin/bash
*/1 * * * * /Path/to/script/script.sh
In script.sh
#!/bin/bash
export JAVA_HOME=/path/to/jdk
some-other-command

I'm using Oh-my-zsh in my macbook so I've tried many things to get the crontab task runs but finally, my solution was prepending the .zshrc before the command to run.
*/30 * * * * . $HOME/.zshrc; node /path/for/my_script.js
This task runs every 30 minutes and uses .zshrc profile to execute my node command.
Don't forget to use the dot before the $HOME var.

I tried most of the provided solutions, but nothing worked at first. It turns out, though, that it wasn't the solutions that failed to work. Apparently, my ~/.bashrc file starts with the following block of code:
case $- in
*i*) ;;
*) return;;
esac
This basically is a case statement that checks the current set of options in the current shell to determine that the shell is running interactively.
If the shell happens to be running interactively, then it moves on to sourcing the ~/.bashrc file.
However, in a shell invoked by cron, the $- variable doesn't contain the i value which indicates interactivity.
Therefore, the ~/.bashrc file never gets sourced fully. As a result, the environment variables never got set.
If this happens to be your issue, feel free to comment out the block of code as follows and try again:
# case $- in
# *i*) ;;
# *) return;;
# esac
I hope this turns out useful

Unfortunately, crontabs have a very limited environment variables scope, thus you need to export them every time the corntab runs.
An easy approach would be the following example, suppose you've your env vars in a file called env, then:
* * * * * . ./env && /path/to_your/command
this part . ./env will export them and then they're used within the same scope of your command

Another way - inspired by this this answer - to "inject" variables is the following (fcron example):
%daily 00 12 \
set -a; \
. /path/to/file/containing/vars; \
set +a; \
/path/to/script/using/vars
From help set:
-a Mark variables which are modified or created for export.
Using + rather than - causes these flags to be turned off.
So everything in between set - and set + gets exported to env and is then available for other scripts, etc. Without using set the variables get sourced but live in set only.
Aside from that it's also useful to pass variables when a program requires a non-root account to run but you'd need some variables inside that other user's environment. Below is an example passing in nullmailer vars to format the e-mail header:
su -s /bin/bash -c "set -a; \
. /path/to/nullmailer-vars; \
set +a; \
/usr/sbin/logcheck" logcheck

All the above solutions work fine.
It will create issues when there are any special characters in your environment variable.
I have found the solution:
eval $(printenv | awk -F= '{print "export " "\""$1"\"""=""\""$2"\"" }' >> /etc/profile)

For me I had to specify path in my NodeJS file.
// did not work!!!!!
require('dotenv').config()
instead
// DID WORK!!
require('dotenv').config({ path: '/full/custom/path/to/your/.env' })

I found this issue while looking at a similar problem that matched the title, but I am stuck with the environment file syntax that systemd or docker use:
FOO=bar
BAZ=qux
This won't work for Vishal's excellent answer because they aren't bash scripts (note the lack of export).
The solution I've used is to read each line into xargs and export them before running the command:
0 5 * * * export $(xargs < $HOME/.env); /path/to/command/to/run

Set Globally env
sudo sh -c "echo MY_GLOBAL_ENV_TO_MY_CURRENT_DIR=$(pwd)" >> /etc/environment"
Add scheduled job to start a script
crontab -e
*/5 * * * * sh -c "$MY_GLOBAL_ENV_TO_MY_CURRENT_DIR/start.sh"
=)

what worked for me (debian based):
create a file with all the needed env var :
#!/bin/bash
env | grep VAR1= > /etc/environment
env | grep VAR2= >> /etc/environment
env | grep VAR3= >> /etc/environment
then build the crontab content, by calling the env file before calling the script that needs it, therefore start the cron service
(crontab -l ; echo '* * * * * . /etc/environment; /usr/local/bin/python /mycode.py >> /var/log/cron-1.log 2>&1') | crontab
service cron start
nb : for python use case, be sure to call the whole python path, else wrong python could be invocated, generating non-sense syntax error

Related

source bashrc doesn't work in cron

All we know that cron ignores variables defined in “.bashrc” and “.bash_profile”, so we have to define it inside cron. I constantly do the same what was written inside the similar question https://unix.stackexchange.com/questions/67940/cron-ignores-variables-defined-in-bashrc-and-bash-profile but still global variables inside .bashrc still not working. I found way to execute it - by defining sh script with "set +a" bashrc script. But "source" still doesn't work.
SHELL=/bin/bash
BASH_ENV=/root/.bashrc
PATH=:/opt/spark/spark-2.2.0-bin-hadoop2.7/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin
SPARK_HOME=/opt/spark/spark-2.2.0-bin-hadoop2.7
MAILTO=root HOME=/
# m h dom mon dow command
* * * * * /bin/bash -c 'source $HOME/.bashrc; echo "SPARK_HOME: '$SPARK_HOME'"; echo "JAVA_HOME: '$JAVA_HOME'"' > /var/log/file.log 2>&1
# DO NOT DELETE LAST LINE
return log file
SPARK_HOME: /opt/spark/spark-2.2.0-bin-hadoop2.7
JAVA_HOME:
also tried to execute this in interactive mode as it was written by mklement0
source .bashrc in a script not working
* * * * * /bin/bash -i source /root/.bashrc; echo $JAVA_HOME > /var/log/file.log 2>&1
As you can see SPARK_HOME is defined inside crontab whilst JAVA_HOME only in .bashrc. P.S in bashrc java home is defined "export JAVA_HOME=/usr/jdk1.8.0_131"
All different cron jobs work fine when I change JAVA_HOME to SPARK_HOME, tried different crontab jobs but the answer is the same as it was.
ubuntu kernel version is
uname -a
Linux 6101c32b9243 4.9.62-21.56.amzn1.x86_64 #1 SMP Thu Nov 16 05:37:08 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Your last attempt runs bash and sources the file, then exits Bash and runs the echo in the calling shell, which isn't Bash, and of course doesn't know or care that the now-defunct Bash process loaded some settings and then forgot them when it exited. (Or, well, this is what would happen if you fixed the simple quoting errors to keep the source command and its argument as a single string.)
The superficial fix is easy;
* * * * * bash -c 'source $HOME/.bashrc; echo "$JAVA_HOME"' >/var/log/file.log 2>&1
The proper fix, however, is probably to encapsulate all of this in a separate script and keep your crontab really simple.
(Are you really sure your user is allowed to overwrite the log file? Replacing the log once a minute seems rather misdirected, although it's good enough for a quick test to see if the job is running at all.)

AWS Linux crontab job isnt executing script

First of all I've tried multiple solutions I know there are several posts with similar problems none of those were a solution for me.
I have a Clojure application which is started using:
lein run -m tsdb-delete.core
The plan is to execute this every day at midnight I want to avoid using Clojure based cron libraries and this is a very light weight application.
I created the following script (start.sh):
/usr/bin/lein run -m tsdb-delete.core
which calls this script at run time (delete.sh):
#!/bin/bash
echo "Deleting:" $1
OUTPUT="$(sudo /opt/opentsdb/build/tsdb scan --delete 30d-ago 7d-ago sum $1)"
echo "${OUTPUT}"
If I call './start.sh' manually it all works as expected and I see console output.
start.sh is located at /home/ec2-user/tsdb-delete/start.sh and delete.sh is located at /home/ec2-user/tsdb-delete/delete.sh
I have added the following to me crontab using crontab -e
PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/ec2-user/bin
SHELL=/bin/bash
*/5 * * * * /home/ec2-user/tsdb-delete/start.sh > /var/tmp/tsdb-delete.out
* * * * * env > /tmp/env.output
* * * * * env > /tmp/env.output is used for debugging purposes, the contents of env.output are as follows:
SHELL=/bin/bash
USER=ec2-user
PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/ec2-user/bin
PWD=/home/ec2-user
LANG=en_GB.UTF-8
SHLVL=1
HOME=/home/ec2-user
LOGNAME=ec2-user
_=/bin/env
and if I run env in the terminal myself I get the following:
HOSTNAME=ip-xx-xx-xx-xx
LESS_TERMCAP_md=
LESS_TERMCAP_me=
TERM=xterm-256color
SHELL=/bin/bash
HISTSIZE=1000
SSH_CLIENT=xxxxxxxxx
LESS_TERMCAP_ue=
SSH_TTY=/dev/pts/0
USER=ec2-user
LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:*.tar=38;5;9:*.tgz=38;5;9:*.arc=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lha=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.tzo=38;5;9:*.t7z=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lrz=38;5;9:*.lz=38;5;9:*.lzo=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.bz=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.war=38;5;9:*.ear=38;5;9:*.sar=38;5;9:*.rar=38;5;9:*.alz=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.cab=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.axv=38;5;13:*.anx=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.axa=38;5;45:*.oga=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:
LESS_TERMCAP_us=
MAIL=/var/spool/mail/ec2-user
PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/ec2-user/bin
PWD=/var/tmp
LANG=en_GB.UTF-8
HISTCONTROL=ignoredups
SHLVL=1
HOME=/home/ec2-user
LOGNAME=ec2-user
SSH_CONNECTION= xxxxxxx
LESSOPEN=||/usr/bin/lesspipe.sh %s
LESS_TERMCAP_se=
_=/bin/env
OLDPWD=/home/ec2-user
The key environment attributes seem to match, and in var/cron/log I see the following:
Oct 28 11:45:01 ip-xx-xx-xx-xx CROND[23591]: (ec2-user) CMD (/home/ec2-user/tsdb-delete/start.sh > /var/tmp/tsdb-delete.out)
Oct 28 11:45:01 ip-xx-xx-xx-xx CROND[23592]: (ec2-user) CMD (env > /tmp/env.output)
and in /var/spool/mail/ I don't see any error messages being thrown and the file /var/tmp/tsdb-delete.out is not created.
Any ideas?
sudo requires a tty, which doesn't exist while running a cron. (1,2)
Here's a better solution; place this in /etc/cron.d/tsdb-delete:
*/5 * * * * root /home/ec2-user/tsdb-delete/start.sh > /var/tmp/tsdb-delete.out
This requires having the execute bit set on start.sh. Also note the /etc/cron entries take a sixth argument which is the user.
It's also bad form to use something like sudo in a cron, and generally using a user cron (crontab -e) is not terribly friendly for configuration management. The above fixes these problems. Still, I'd recommend moving the script to a safer location (since it's running via root), and since it's root you can easily send output to /var/log/ (and append it):
*/5 * * * * root /opt/tsdb-delete/start.sh >> /var/log/tsdb-delete.out

Linux bash shell script output is different from cronjob vs manually running the script

I wrote a linux bash shell script which works fine except the output when I run it manually is different than when I run it from a cronjob.
The particular command is lftp:
lftp -e "lcd $outgoingpathlocal;mput -O $incomingpathremote *.CSV;exit" -u $FTPUSERNAME,$FTPPASSWORD $FTPSERVER >> ${SCRIPTLOGFILE} 2>&1
When I run the script manually, the ${SCRIPTLOGFILE} contains a lot of info such as how many files/bytes/etc transferred. But when I run the same script from a cronjob there is no output unless there was an error (such as could not connect). I have tried various terminal output configurations but none work for this lftp command. Suggestions?
It's worth reading this:
crontab PATH and USER
In particular, cron won't set the same environment variables you're used to an interactive shell.
You might want to wrap your entire cron job up in a script, and then you can, for example, temporarily add some code like export >> scriptenvironment.txt and see what the difference is between the cron invoked script and the interactively invoked script.
Try man 5 crontab for details.
Once you know what envrionment variables you need for your script to run, you can set them in the crontab as necessary, or source at the start of your own script.
EXAMPLE CRON FILE
# use /bin/sh to run commands, overriding the default set by cron
SHELL=/bin/sh
# mail any output to `paul', no matter whose crontab this is
MAILTO=paul
#
# run five minutes after midnight, every day
5 0 * * * $HOME/bin/daily.job >> $HOME/tmp/out 2>&1
# run at 2:15pm on the first of every month -- output mailed to paul
15 14 1 * * $HOME/bin/monthly
# run at 10 pm on weekdays, annoy Joe
0 22 * * 1-5 mail -s "It's 10pm" joe%Joe,%%Where are your kids?%
23 0-23/2 * * * echo "run 23 minutes after midn, 2am, 4am ..., everyday"
5 4 * * sun echo "run at 5 after 4 every sunday"

Run a script with arguments using crontab

I know this may have been answered earlier in various posts, but I've not been able to make this run myself.
I have a bash script (service.sh) that I would like to run every minute. It needs an argument to be passed (start in this case).
Using another script (test.sh) I am scheduling the cron expression for the above script:
echo "* * * * * /opt/service.sh start" > /opt/cronForSecops
crontab /opt/cronForSecops
I can see by using crontab -l that this is being set correctly as:
* * * * * /opt/service.sh start
However, the service.sh does not run, and I see no logs/files being created (which the service.sh file is supposed to do, when I run it normally).
Can anybody please guide me on where I am going wrong?
I was having this same issue where I using the following crontab:
0 23 * * * sudo -u myname /home/myname/bin/buildme.sh -f >> /home/myname/log.txt
And inside the bash script I was using this to get the -f option:
while getopts ":f" opt; do
case $opt in
f)
force_full=1
;;
\?)
echo "Invalid option: -$OPTARG" >&2
;;
esac
done
So I noticed that the option wasn't being honored when I ran this through cron for some reason. Well, adding /bin/bash to the cronjob fixed it right up. The new crontab is:
0 23 * * * sudo -u myname /bin/bash /home/myname/bin/buildme.sh -f >> /home/myname/log.txt
Hope it helps!
Try creating a simple wrapper script called /opt/start-service.sh with this content:
#!/bin/sh
/opt/service.sh start
and make sure it's executable then use
* * * * * /opt/start-service.sh
as the crontab entry

Ubuntu 10.04 LTS Cron jobs not working

I'm trying to use a cronjob to run a ruby script (Using Rails3 runner) with the following Cronjobs:
#!/bin/bash
0-59 * * * * echo 'script test'
# Begin Whenever generated tasks for: test1
* * * * * /bin/bash -l -c '/home/administrator/test1/script/rails runner /home/administrator/test1/app/create_flag.rb >> /home/administrator/test1/test.log 2>&1'
# End Whenever generated tasks for: test1
test1 is the name of the Rails3 project folder.
the "echo 'script test'" was added as a test, but neither seems to be executing. I'm currently using Ubuntu 10.04 LTS.
Have I written the cronjob incorrectly?
Crontab file is not a shell script. So you don't need #!/bin/bash at the beginning of the file. Plus, spaces there are suspicious. Try something like this:
SHELL=/bin/bash
MAILTO=administrator#localhost
BASH_ENV=/home/administrator/.bash_profile
* * * * * /home/administrator/test1/script/rails runner /home/administrator/test1/app/create_flag.rb >> /home/administrator/test1/test.log 2>&1'
Plus, make sure you call crontab -e as administrator to edit the crontab file.
You need to specify the user which runs the commands (you can see the format here. Also the echo will output 'script test' to what? If you want a test try doing a touch on a file, so you can physically see the action of the cron job.
Cron does not use your user environment, so it will not have the same path set that you have. This means that you should use absolute paths for commands.

Resources