I am trying to use a docker container based on an Alpine image to run a scheduled cron job, following this tutorial, but after printing the statement in my startup script, the container just exits, without running my other script.
My docker-compose service is configured as follows:
cron:
image: alpine:3.11
command: /usr/local/startup.sh && crond -f -l 8
volumes:
- ./cron_tasks_folder/1min:/etc/periodic/1min/:ro
- ./cron_tasks_folder/15min:/etc/periodic/15min/:ro
- ./cron_tasks_folder/hourly:/etc/periodic/hourly/:ro
- ./scripts/startup.sh:/usr/local/startup.sh:ro
So it runs an initial script called startup.sh and then starts the cron daemon. The startup.sh script contains the following:
#!/bin/sh
echo "Starting startup.sh.."
echo "* * * * * run-parts /etc/periodic/1min" >> /etc/crontabs/root
crontab -l
sleep 300
I dropped a sleep command in there just so I could launch an interactive shell on the container and make sure everything inside it looks good. The script creates another folder for 1min scripts. I have added a test script in there, and I can verify it's there:
/etc/periodic/1min # ls -a
. .. testScript
The script is executable:
/etc/periodic/1min # ls -l testScript
-rwxr-xr-x 1 root root 31 Jul 30 01:51 testScript
And testScript is just an echo statement to make sure it's working first:
echo "The donkey is in charge"
And looking at the root file in etc/crontabs, I see the following (I've re-run the container several times, and each time it's creating a new 1min folder, which is unnecessary, but I think not the problem here):
# do daily/weekly/monthly maintenance
# min hour day month weekday command
*/15 * * * * run-parts /etc/periodic/15min
0 * * * * run-parts /etc/periodic/hourly
0 2 * * * run-parts /etc/periodic/daily
0 3 * * 6 run-parts /etc/periodic/weekly
0 5 1 * * run-parts /etc/periodic/monthly
* * * * * run-parts /etc/periodic/1min
* * * * * run-parts /etc/periodic/1min
* * * * * run-parts /etc/periodic/1min
* * * * * run-parts /etc/periodic/1min
* * * * * run-parts /etc/periodic/1min
The echo statement in testScript is never printed to my terminal, and the container exits with exit code 0 shortly after starting. I want to print this statement every minute... what am I missing?
In the docker compose file you have
command: /usr/local/startup.sh && crond -f -l 8
The intention is to run as a shell command, but it's not at all clear from the question that's what's going to happen; that depends on your ENTRYPOINT. Since it's defined with [] brackets, not additional shell will be provided. The command value will be passed as arguments to the ENTRYPOINT.
Assuming that will become a shell command, && in the shell runs the left hand side, and if that succeeds, then runs the right hand side. So startup.sh needs to complete before crond is executed. startup.sh ends with
sleep 300
crond is invoked only after that 300 seconds.
In either case, crond is either not invoked at all, or sleep has not been completing. The comments show that an error starting crond was discovered.
Using an entrypoint such as this is standard practice to configure the environment before, or provide runtime parameters when, invoking the main executable. To do it right, you should make sure to use exec to run the main executable so that it receives the signals that would otherwise go to the bash shell running the entrypoint script.
So at the end of startup.sh:
exec crond -f -l 8
Will replace the shell running startup.sh with crond, so that crond receives all signals (at this point the shell is gone). It's subtle but important!
In general, keep the invocation of the application as simple as possible. Case in point, your execution process was split between entrypoint, command, and startup script, with no clear interface between them. You wouldn't have gotten hung up on the invocation if you had put crond directly into the Dockerfile and left it at that. Sometimes arguments must be provided at runtime, but environment variables - which have names, not just positions - are often preferred. This keeps invocations simple and debugging straightforward. But, when that doesn't work, a shell script entrypoint is a great solution - just make sure to exec your final process!
I have a very simple script that works from the command line.
#!/bin/bash
reboot
When I put a call to execute the script into root users crontab -e using the following format it does not run. It does run the first two commands, just that last one is giving me grief. I have no MTA installed as I do not need it.
*/10 * * * * service jwtpay restart
0 3 * * * bash /root/backup/mongo.backup.s3.sh kickass /root/backup >/dev/null 2>&1
0 */3 * * * bash /root/reboot.sh >/dev/null 2>&1
What am I missing?
Maybe the script is not executable... Since you use root's crontab why call the binary via a script and not the binary itself? Use the full path to the binary. It may vary on your system. Find out where it is with which reboot.
0 */3 * * * /sbin/reboot
Don't forget to restart the cron daemon, after changeing the crontab.
This is my perl script, it's just a test.
#!/usr/local/bin/perl
open (MYFILE, '>>data.txt');
print MYFILE "Worked!\n";
close (MYFILE);
I saved it as test.pl in cgi-bin/
When I run a shell command,
root#srv ./test.pl
it works fine and it adds "Worked!" in the data.txt file successfully.
But when I add this in cronjob to run every minute it doesn't work
code:
*/1 * * * * root cd /home/testing/public_html/cgi-bin;./test.pl
I've also tried
*/1 * * * * cd /home/testing/public_html/cgi-bin;./test.pl
I'm thinking it's my environment for cron that's not setup correctly, but I don't know how to fix it.
I've used this in crontab to execute the env
*/1 * * * * env > /home/tmp2/env.cron
And Here is the result:
SHELL=/bin/sh
USER=root
PATH=/usr/local/jdk/bin:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/X11R6/bin:/root/bin
PWD=/root
SHLVL=1
HOME=/root
LOGNAME=root
_=/bin/env
And then I exported the env of the shell i'm using to another file env.shell
results:
HOSTNAME=srv.testing.com
SELINUX_ROLE_REQUESTED=
TERM=xterm
SHELL=/bin/bash
HISTSIZE=1000
SSH_CLIENT=xxx.xx.xxx.xx 58048 22
SELINUX_USE_CURRENT_RANGE=
QTDIR=/usr/lib64/qt-3.3
QTINC=/usr/lib64/qt-3.3/include
SSH_TTY=/dev/pts/0
USER=root
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=0$
MAIL=/var/spool/mail/root
PATH=/usr/local/jdk/bin:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/X11R6/bin:/root/bin
PWD=/home/tmp2
JAVA_HOME=/usr/local/jdk
EDITOR=pico
LANG=en_US.UTF-8
SELINUX_LEVEL_REQUESTED=
HISTCONTROL=ignoredups
SHLVL=1
HOME=/root
LS_OPTIONS=--color=tty -F -a -b -T 0
LOGNAME=root
VISUAL=pico
QTLIB=/usr/lib64/qt-3.3/lib
CVS_RSH=ssh
CLASSPATH=.:/usr/local/jdk/lib/classes.zip
SSH_CONNECTION=xxx.xx.xxx.xx 58048 xx.xx.xx.xx 22
LESSOPEN=|/usr/bin/lesspipe.sh %s
G_BROKEN_FILENAMES=1
_=/bin/env
OLDPWD=/home/testing/public_html/cgi-bin
I copied the shell path
PATH=/usr/local/jdk/bin:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/X11R6/bin:/root/bin
And added it to /etc/crontab and I restart cron but it still didn't work. So, I changed it back to normal, and then I added it to crontab using crontab -e and it still didn't help.
I'm not sure what is going on,
Thank you for your time
I finally solved it!
I changed the path of crontab -e
PATH=/usr/local/jdk/bin:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/X11R6/bin:/root/bin
That's the same path in my shell you can get it if you do this
root # env > env.shell
root # nano env.shell
then copy the path from there, and paste it in crontab using crontab -e
Then, the last step was to add it to crontab -e, but after I added the path I had the cronjob in /etc/cron.d/sysstat so that's why it wasn't working.
Now it works and the main problem was the PATH.
Thank you for your time everyone.
Try
* * * * * root cd "/home/testing/public_html/cgi-bin"; ./test.pl
I am trying to figure out how to run a SSH command via cron for linux. The command I want to run is:
svn update /path/to/working/dir
Something like:
*/1 * * * * root ssh svn update /path/to/working/dir
Anyone know what I would need to do with the cron line?
EDIT: I don't need it to be SSH, just need to run svn update on the same server as cron to the working directory.
EDIT 2: What I was looking for was:
*/1 * * * * svn update /path/to/your/working/copy
I worded it incorrectly though, asking too specific about SSH, so I awarded the answer that talks about cron via SSH specifically, but if anyone wants to know how to do it locally, you don't need SSH.
You must compare the environment variables. I've wrote a simple bash script to do that for me:
#!/bin/bash
env
echo $PATH
type -a svn
cd /home/<username>
svn info
exit 0
And save it in /home//crontest.sh
Then you execute the code by hand and write the result into a file:
/home/<username>/crontest.sh > /home/<username>/hand_env
Then in your crontab:
* * * * * /home/<username/crontest.sh > /home/<username>/cron_env
After that you have two files, you will probably see that there are some differences between the environment variables.
The solution to the svn and cron problem is to set the environment variables in bash script of svn update to those obtained by hand it should look something like this (parts 2>/dev/null and exit 0 ARE VERY IMPORTANT):
#!/bin/bash
export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X$
export MAIL=/var/mail/root
export _=/usr/bin/env
export PWD=/home/tv
export LANG=en_US.UTF-8
export HOME=/home/tv
export SHLVL=2
export LOGNAME=root
#export LESSOPEN=| /usr/bin/lesspipe %s
#export LESSCLOSE=/usr/bin/lesspipe %s %s
export SHELL=/bin/bash
svn update https://localhost/svn /var/www/<dir> 2>/dev/null
exit 0
Write this script in for eg. /etc/init.d/skrypty/cron.sh
Then you just put in your crontab (i've done it as root)
* * * * * /etc/init.d/skrypty/cron.sh >/dev/null 2>&1
You need to pass the host name (and also the username, unless you want to log in as root) to SSH:
*/1 * * * * root ssh user#hostname svn update /path/to/working/dir
Also see ssh --help for more information.
Note that you'd need to input your password with SSH, unless you've setup your SSH to be able to log in without password
I have faced the same issue where I had edited crontab as root user. I have working copy under my home (/home/myname/project) directory and I did edit crontab as myname user and it worked.
0 22 * * * bash /home/myname/svn.sh
svn.sh has the following lines
#!/bin/sh
svn up /home/myname/project
I have a crontab running every hour. The user running it has environment variabless in the .bash_profile that work when the user runs the job from the terminal, however, obviously these don't get picked up by crontab when it runs.
I've tried setting them in .profile and .bashrc but they still don't seem to get picked up. Does anyone know where I can put environment vars that crontab can pick up?
You can define environment variables in the crontab itself when running crontab -e from the command line.
LANG=nb_NO.UTF-8
LC_ALL=nb_NO.UTF-8
# m h dom mon dow command
* * * * * sleep 5s && echo "yo"
This feature is only available to certain implementations of cron. Ubuntu and Debian currently use vixie-cron which allows these to be declared in the crontab file (also GNU mcron).
Archlinux and RedHat use cronie which does not allow environment variables to be declared and will throw syntax errors in the cron.log. Workaround can be done per-entry:
# m h dom mon dow command
* * * * * export LC_ALL=nb_NO.UTF-8; sleep 5s && echo "yo"
I got one more solution for this problem:
0 5 * * * . $HOME/.profile; /path/to/command/to/run
In this case it will pick all the environment variable defined in your $HOME/.profile file.
Of course $HOME is also not set, you have to replace it with the full path of your $HOME.
Setting vars in /etc/environment also worked for me in Ubuntu. As of 12.04, variables in /etc/environment are loaded for cron.
Have 'cron' run a shell script that sets the environment before running the command.
Always.
# #(#)$Id: crontab,v 4.2 2007/09/17 02:41:00 jleffler Exp $
# Crontab file for Home Directory for Jonathan Leffler (JL)
#-----------------------------------------------------------------------------
#Min Hour Day Month Weekday Command
#-----------------------------------------------------------------------------
0 * * * * /usr/bin/ksh /work1/jleffler/bin/Cron/hourly
1 1 * * * /usr/bin/ksh /work1/jleffler/bin/Cron/daily
23 1 * * 1-5 /usr/bin/ksh /work1/jleffler/bin/Cron/weekday
2 3 * * 0 /usr/bin/ksh /work1/jleffler/bin/Cron/weekly
21 3 1 * * /usr/bin/ksh /work1/jleffler/bin/Cron/monthly
The scripts in ~/bin/Cron are all links to a single script, 'runcron', which looks like:
: "$Id: runcron.sh,v 2.1 2001/02/27 00:53:22 jleffler Exp $"
#
# Commands to be performed by Cron (no debugging options)
# Set environment -- not done by cron (usually switches HOME)
. $HOME/.cronfile
base=`basename $0`
cmd=${REAL_HOME:-/real/home}/bin/$base
if [ ! -x $cmd ]
then cmd=${HOME}/bin/$base
fi
exec $cmd ${#:+"$#"}
(Written using an older coding standard - nowadays, I'd use a shebang '#!' at the start.)
The '~/.cronfile' is a variation on my profile for use by cron - rigorously non-interactive and no echoing for the sake of being noisy. You could arrange to execute the .profile and so on instead. (The REAL_HOME stuff is an artefact of my environment - you can pretend it is the same as $HOME.)
So, this code reads the appropriate environment and then executes the non-Cron version of the command from my home directory. So, for example, my 'weekday' command looks like:
: "#(#)$Id: weekday.sh,v 1.10 2007/09/17 02:42:03 jleffler Exp $"
#
# Commands to be done each weekday
# Update ICSCOPE
n.updics
The 'daily' command is simpler:
: "#(#)$Id: daily.sh,v 1.5 1997/06/02 22:04:21 johnl Exp $"
#
# Commands to be done daily
# Nothing -- most things are done on weekdays only
exit 0
If you start the scripts you are executing through cron with:
#!/bin/bash -l
They should pick up your ~/.bash_profile environment variables
Expanding on #carestad example, which I find easier, is to run the script with cron and have the environment in the script.
In crontab -e file:
SHELL=/bin/bash
*/1 * * * * $HOME/cron_job.sh
In cron_job.sh file:
#!/bin/bash
source $HOME/.bash_profile
some_other_cmd
Any command after the source of .bash_profile will have your environment as if you logged in.
Whatever you set in crontab will be available in the cronjobs, both directly and using the variables in the scripts.
Use them in the definition of the cronjob
You can configure crontab so that it sets variables that then the can cronjob use:
$ crontab -l
myvar="hi man"
* * * * * echo "$myvar. date is $(date)" >> /tmp/hello
Now the file /tmp/hello shows things like:
$ cat /tmp/hello
hi man. date is Thu May 12 12:10:01 CEST 2016
hi man. date is Thu May 12 12:11:01 CEST 2016
Use them in the script run by cronjob
You can configure crontab so that it sets variables that then the scripts can use:
$ crontab -l
myvar="hi man"
* * * * * /bin/bash /tmp/myscript.sh
And say script /tmp/myscript.sh is like this:
echo "Now is $(date). myvar=$myvar" >> /tmp/myoutput.res
It generates a file /tmp/myoutput.res showing:
$ cat /tmp/myoutput.res
Now is Thu May 12 12:07:01 CEST 2016. myvar=hi man
Now is Thu May 12 12:08:01 CEST 2016. myvar=hi man
...
For me I had to set the environment variable for a php application. I resolved it by adding the following code to my crontab.
$ sudo crontab -e
crontab:
ENVIRONMENT_VAR=production
* * * * * /home/deploy/my_app/cron/cron.doSomethingWonderful.php
and inside doSomethingWonderful.php I could get the environment value with:
<?php
echo $_SERVER['ENVIRONMENT_VAR']; # => "production"
I hope this helps!
Instead of
0 * * * * sh /my/script.sh
Use bash -l -c
0 * * * * bash -l -c 'sh /my/script.sh'
You can also prepend your command with env to inject Environment variables like so:
0 * * * * env VARIABLE=VALUE /usr/bin/mycommand
Expanding on #Robert Brisita has just expand , also if you don't want to set up all the variables of the profile in the script, you can select the variables to export on the top of the script
In crontab -e file:
SHELL=/bin/bash
*/1 * * * * /Path/to/script/script.sh
In script.sh
#!/bin/bash
export JAVA_HOME=/path/to/jdk
some-other-command
I'm using Oh-my-zsh in my macbook so I've tried many things to get the crontab task runs but finally, my solution was prepending the .zshrc before the command to run.
*/30 * * * * . $HOME/.zshrc; node /path/for/my_script.js
This task runs every 30 minutes and uses .zshrc profile to execute my node command.
Don't forget to use the dot before the $HOME var.
I tried most of the provided solutions, but nothing worked at first. It turns out, though, that it wasn't the solutions that failed to work. Apparently, my ~/.bashrc file starts with the following block of code:
case $- in
*i*) ;;
*) return;;
esac
This basically is a case statement that checks the current set of options in the current shell to determine that the shell is running interactively.
If the shell happens to be running interactively, then it moves on to sourcing the ~/.bashrc file.
However, in a shell invoked by cron, the $- variable doesn't contain the i value which indicates interactivity.
Therefore, the ~/.bashrc file never gets sourced fully. As a result, the environment variables never got set.
If this happens to be your issue, feel free to comment out the block of code as follows and try again:
# case $- in
# *i*) ;;
# *) return;;
# esac
I hope this turns out useful
Unfortunately, crontabs have a very limited environment variables scope, thus you need to export them every time the corntab runs.
An easy approach would be the following example, suppose you've your env vars in a file called env, then:
* * * * * . ./env && /path/to_your/command
this part . ./env will export them and then they're used within the same scope of your command
Another way - inspired by this this answer - to "inject" variables is the following (fcron example):
%daily 00 12 \
set -a; \
. /path/to/file/containing/vars; \
set +a; \
/path/to/script/using/vars
From help set:
-a Mark variables which are modified or created for export.
Using + rather than - causes these flags to be turned off.
So everything in between set - and set + gets exported to env and is then available for other scripts, etc. Without using set the variables get sourced but live in set only.
Aside from that it's also useful to pass variables when a program requires a non-root account to run but you'd need some variables inside that other user's environment. Below is an example passing in nullmailer vars to format the e-mail header:
su -s /bin/bash -c "set -a; \
. /path/to/nullmailer-vars; \
set +a; \
/usr/sbin/logcheck" logcheck
All the above solutions work fine.
It will create issues when there are any special characters in your environment variable.
I have found the solution:
eval $(printenv | awk -F= '{print "export " "\""$1"\"""=""\""$2"\"" }' >> /etc/profile)
For me I had to specify path in my NodeJS file.
// did not work!!!!!
require('dotenv').config()
instead
// DID WORK!!
require('dotenv').config({ path: '/full/custom/path/to/your/.env' })
I found this issue while looking at a similar problem that matched the title, but I am stuck with the environment file syntax that systemd or docker use:
FOO=bar
BAZ=qux
This won't work for Vishal's excellent answer because they aren't bash scripts (note the lack of export).
The solution I've used is to read each line into xargs and export them before running the command:
0 5 * * * export $(xargs < $HOME/.env); /path/to/command/to/run
Set Globally env
sudo sh -c "echo MY_GLOBAL_ENV_TO_MY_CURRENT_DIR=$(pwd)" >> /etc/environment"
Add scheduled job to start a script
crontab -e
*/5 * * * * sh -c "$MY_GLOBAL_ENV_TO_MY_CURRENT_DIR/start.sh"
=)
what worked for me (debian based):
create a file with all the needed env var :
#!/bin/bash
env | grep VAR1= > /etc/environment
env | grep VAR2= >> /etc/environment
env | grep VAR3= >> /etc/environment
then build the crontab content, by calling the env file before calling the script that needs it, therefore start the cron service
(crontab -l ; echo '* * * * * . /etc/environment; /usr/local/bin/python /mycode.py >> /var/log/cron-1.log 2>&1') | crontab
service cron start
nb : for python use case, be sure to call the whole python path, else wrong python could be invocated, generating non-sense syntax error