This question already has answers here:
Is there a special restriction on commands executed by cron? [duplicate]
(2 answers)
Closed 4 years ago.
I can use this command
mysqldump -u"root" myDB| gzip > mydb_`date +%d-%m-%Y`.sql.gz
but when run in crontab
* * * * * mysqldump -u"root" myDB| gzip > mydb_`date +%d-%m-%Y`.sql.gz
( this error cause by function date, when i remove it , crontab run good )
on ubuntu, it happen this error in log file.
ubuntu CRON[xxxx] (user) CMD(mysqldump -u"root" myDB| gzip > mydb_`date+)
ubuntu CRON[xxxx] (CRON) error ( grandchild #5353 failed with exit status 2)
ubuntu CRON[xxxx] (CRON) info (no MTA installed, discarding output)
% signs in a crontab command are converted to newlines, and all data after the first % is sent to the command's stdin. Replace each % with \%.
(And you only had 4 time fields: * * * *; you need 5 (you later fixed the question).)
From the man 5 crontab:
The ``sixth'' field (the rest of the line) specifies the command to be
run.
The entire command portion of the line, up to a newline or %
character, will be executed by /bin/sh or by the shell specified
in the SHELL variable of the crontab file.
Percent-signs (%) in the
command, unless escaped with backslash (), will be changed into
newline characters, and all data after the first % will be sent to the
command as standard input. There is no way to split a single command
line onto multiple lines, like the shell's trailing "\".
Related
I am a lot confused about this behavior.
Each time I run this script from terminal, it works fine, but it fails once is executed from a crontab.
Inside the script you can find each step description.
The target is to print date and time with the peso variable to a file.
I have changed line 16 countless times. Same thing for line #4.
Edit for clarity: THIS IS JUST A SMALL PART FROM THE WHOLE SCRIPT.
It runs nice every minute. It does everything, except the peso issue.
Please HELP!!!
1 # Here I compute one decimal value (like z=0.123) with two integers (sd and mean)
2 peso=$( echo "scale=4; ${z}*${sd}/100 + ${mean}/100" | bc -l)
3 echo "peso_mtrx="$peso # This is for checking: shows 40.123 (example), so it is OK
4 peso+=";" # Add a semicolon to check its behaviour
5 echo "peso= "$peso # show it: OK
6 peso1=$(date "+%D %T") # Now I capture date and time
7 echo "fecha= "$peso1 # shows it, so it is OK
8 peso1+=";" # add a semicolon to date
9 peso1+=$peso # concatenate the peso variable
10 echo $(printf '%s' "$peso1") # shows it, so it is ok up to here
11 echo $(printf '%s' "$peso1") >> ~/projects/Files/normal.csv # WRITE TO FILE
12 # whenever I run this script from terminal, all variables showed right and even print all data into file.
13 # File stores a new line like: 02/03/21 08:24:40;40.1709;
14 # BUT... when it is executed from a CRON job... everything except peso are stored.
15 # File stores a line like: 02/03/21 08:24:40;; peso variable just vanishes.
16 # is it something related to subshells? how to solve this rigmarole?
As I was suspicious, the whole thing was related to subshell issues.
I just did something inside crontab.
Once I execute crontab -e, I initially had something like:
*/1 * * * * /absolute/path/to/project.sh
So doing some reading I ended up doing this:
SHELL=/bin/bash
*/1 * * * * exec bash -l /absolute/path/to/project.sh
I beg to an expert to enlighten us about this solution. As far as I do understand, it is related to create a login shell inside cron using the information stored in .bash_profile.
It did enable the environment variables to be reachable.
I am executing a PHP script via the command line which, for a specific user, runs fine when executed on the commandline, but when the exact same command is put into the same user's crontab, a PHP iconv error is returned.
The commandline is utilising the Yii framework and the Zend Lucene library, but I'm not sure if that's pertinent.
I've made all executable and script paths absolute in the crontab line and can verify that it works when executed directly on the commandline.
I wrapped the actual PHP invocation in a one-line shell script, as I read elsewhere here that this solved a similar problem for someone, but no joy.
The command successfully executed on the commandline is:
/bin/sh /var/www/yii-projects/projectname/protected/scripts/buildIndex.sh >> /var/lucene/lucene.log
The content of the buildIndex.sh script is:
/usr/bin/php /var/www/yii-projects/projectname/protected/scripts/cron.php lucene buildIndex
And the crontab line is:
*/10 * * * * /bin/sh /var/www/yii-projects/projectname/protected/scripts/buildIndex.sh >> /var/lucene/lucene.log
The error shown in the log when the crontab executes is:
PHP Error[8]: iconv(): Detected an illegal character in input string
in file /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Analysis/Analyzer/Common/Text.php at line 58
0 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Analysis/Analyzer/Common/Text.php(58): iconv()
1 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Analysis/Analyzer.php(125): Zend_Search_Lucene_Analysis_Analyzer_Common_Text_CaseInsensitive->reset()
2 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Index/SegmentWriter/DocumentWriter.php(98): Zend_Search_Lucene_Analysis_Analyzer_Common_Text_CaseInsensitive->setInput()
3 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Index/Writer.php(244): Zend_Search_Lucene_Index_SegmentWriter_DocumentWriter->addDocument()
4 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene.php(1410): Zend_Search_Lucene_Index_Writer->addDocument()
5 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Proxy.php(500): Zend_Search_Lucene->addDocument()
6 /var/www/yii-projects/projectname/protected/commands/LuceneCommand.php(97): Zend_Search_Lucene_Proxy->addDocument()
7 unknown(0): LuceneCommand->actionBuildIndex()
8 /var/www/yii-projects/yii-1.1.12.b600af/framework/console/CConsoleCommand.php(173): ReflectionMethod->invokeArgs()
9 /var/www/yii-projects/yii-1.1.12.b600af/framework/console/CConsoleCommandRunner.php(68): LuceneCommand->run()
10 /var/www/yii-projects/yii-1.1.12.b600af/framework/console/CConsoleApplication.php(92): CConsoleCommandRunner->run()
11 /var/www/yii-projects/yii-1.1.12.b600af/framework/base/CApplication.php(162): CConsoleApplication->processRequest()
12 /var/www/yii-projects/projectname/protected/scripts/cron.php(14): CConsoleApplication->run()
I cannot think of any reason why there is any difference, given the measures taken, and the fact that the user is the same in both cases.
Please help!
Thanks
Edit - I should also confirm that the underlying data that is being indexed is not changing - I've executed both scenarios alternately many times and get the above results consistently.
Try with the -f switch and directly from crontab:
/usr/bin/php -f /var/www/yii-projects/projectname/protected/scripts/cron.php lucene buildIndex
Also are you sure that the text of the command you are passing is in UTF8? Could there be some other symbol there? Maybe a BOM? You can check this with a HEX editor - open your shell script, omit all the letters and see what's left. Usually a BOM in UTF8 is EF BB BF but it may not be a bom at all. Just check.
Necessary shell environment variables are not available to crontab job, so added this to irishhp users's crontab:
PATH=/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
LANG=en_US.UTF-8
which resolved.
How can I store variables in my crontab? I realize it's not shell but say I want to have some constants like a path to my app or something?
In Vixie cron, which is possibly the most common, you can do this almost exactly like a shell script.
VARIABLE=value
PATH=/bin:/path/to/doathing
0 0 * * * doathing.sh $VARIABLE
The man page says:
An active line in a crontab will be either an environment setting or a cron command. An environment setting is of the form,
name = value
where the spaces around the equal-sign (=) are optional, and any subsequent non-leading spaces in value will be part of the value assigned
to name. The value string may be placed in quotes (single or double, but matching) to preserve leading or trailing blanks. The name
string may also be placed in quote (single or double, but matching) to preserve leading, trailing or inner blanks.
You can tell if you have Vixie cron by checking the man page for crontab; the author will be Paul Vixie. Different crons may or may not support this (BusyBox's cron for example, does not), in which case your best option is to wrap your command in a shell script and run that script from cron instead. In fact, this is a good thing to do for anything complicated.
To keep my crontab clean, I would just call a shell script and do the fun stuff in the script.
I think the important fact to point out here is (as stated in an earlier comment by Pierre D Mar 25, 2015 at 18:58) that variable declarations are not expand/interpolated and so can not embed other variable values.
Variables are only expanded/interpolated in the commands themselves.
So:
var1 = bar
var2 = foo${var1}
42 17 * * * /path/to/command ${var2}
Results in: /path/to/command foo${var1}
While:
var1 = bar
var2 = foo
42 17 * * * /path/to/command ${var2}${var1}
Results in: /path/to/command foobar
So in my case the following works fine, no wrapping in shell scripts required:
SHELL=/bin/bash
timestamp=date +20%y_%m_%d_%H_%M_%S
logdir=/my/log/dir
0 2 * * * /my/command/path/mycmd param >> ${logdir}/myfile_$(${timestamp}).log
verses something like this which does not work:
logfile = /my/log/dir/myfile_${timestamp}.log
since the later is not expanded, but is rather interpreted as is including "${" and "}" as part of the string.
Just a working example of using variables in the crontab file and their substitution in the strings:
CURRENT_TIME=date +%Y.%m.%d_%H:%M:%S.%3N
CURRENT_DATE=date +%Y_%m_%d
SIMPLE_VAR=the_simple_var
LOG_DIR=/var/log/cron
* * * * * /bin/echo "simple variable test! ${SIMPLE_VAR}__test!" >> "${LOG_DIR}/test.log"
* * * * * /bin/echo "complex variable test! $(${CURRENT_TIME})__test!" >> "${LOG_DIR}/test.log"
Tested on this Docker image (paste the above crontab to the crontab.txt):
FROM debian:10-slim
# Install docker (Yep, this is a docker in docker):
RUN curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh
# Install CRON:
RUN apt-get update && apt-get install -y --no-install-recommends cron
# Add a crontab_config.txt task:
COPY crontab.txt /var/crontab.txt
RUN crontab /var/crontab.txt
ENTRYPOINT ["cron", "-f"]
Add this to the crontab to run any commands inside another docker containers:
/usr/bin/docker exec container_name ls -la
If you have a few environment variables you want to set for a particular job, just inline those into the sh snippet.
42 17 * * * myvariable='value' path/to/command
In case it's not obvious, the sh syntax var=value command sets var to value for the duration of command. You can have several of these if you need more than one.
42 17 * * * firstname='Fred` lastname='Muggs' path/to/command
If you have nontrivial variables you want to access from several places, probably put them in a separate file, and source (.) that file from your shell startup script and your cron jobs.
Let's say you have a file $HOME/bin/myenv with settings like
myvar=$(output of some complex command)
another='another
complex
variable'
then you can add
. $HOME/bin/myenv
to your .profile (or .zshrc or .bash_profile or what have you; but .profileis portable, and also used bysh`) and
42 17 * * * . $HOME/bin/myenv; path/to/command
in your crontab.
Notice the lone dot before the space before the file name; this is the dot command (also known as source in e.g. Bash) which reads the file into the current shell instance as if you had typed in the things in the file here.
Tangentially, the $HOME/ part is strictly speaking redundant; cron jobs will always run in the invoking user's home directory.
Obviously, if you want a variable to be true in your entire crontab, set it at the top, before the scheduled jobs.
The following works fine from command line
/usr/bin/mysqldump -uUser -pPass Db_name > /var/www/db_backup/db.`date +%Y%m%d%H%M`.sql
but when I try to do that in cron, I get the error:
bad ` sign
errors in crontab file, can't install
I saw someone else on the net solve the same problem by escaping the percent signs, but that didn't help and I tried it with just date inside backquotes with no format specifiers and still got the errors.
I've also seen date's argument enclosed in single or double quotes, but that doesn't help either.
Granted I could just throw it into a script and execute that I suppose - but what fun is that?
Any ideas? I'm using RHEL 5.
Try it with $() instead of backticks. And you probably do need to escape the percent signs since cron converts them to newlines otherwise.
* 0 * * * /usr/bin/mysqldump -uUser -pPass Db_name > /var/www/db_backup/db.$(date +\%Y\%m\%d\%H\%M).sql
Also, you should store the password in an option file with secure permissions (eg. 600 or 640) instead of passing it on the command line.
Put your one line script (as shown) into a proper script file and invoke that from cron:
$ cat /usr/local/bin/db-backup
#!/bin/sh
/usr/bin/mysqldump -uUser -pPass Db_name > \
/var/www/db_backup/db.`date +%Y%m%d%H%M`.sql
$ # use RHEL commands to add db-backup to your crontab
This question already has an answer here:
How is % (percent sign) special in crontab?
(1 answer)
Closed 4 years ago.
I have a crontab that looks like
0 0 * * * pg_dump DB_NAME > /path/to/dumps/`date +%Y%m%d`.dmp
which works fine when I run it manually, but not when cron runs it. After digging through the logs, I see
Dec 12 00:00:01 localhost crond[17638]: (postgres) CMD (pg_dump DB_NAME > /path/to/dumps/`date +)
It looks like a problem with percent signs, but the man page doesn't even contain the percent character at all, so I thought they were alright.
You have to escape percent signs with a backslash:
0 0 * * * pg_dump DB_NAME > /path/to/dumps/`date +\%Y\%m\%d`.dmp
From man 5 crontab:
The ‘‘sixth’’ field (the rest of the line) specifies the command to
be
run. The entire command portion of the line, up to a
newline or %
character, will be executed by /bin/sh or by the shell specified in
the
SHELL variable of the crontab file. Percent-signs (%) in the
command,
unless escaped with backslash (\), will be changed into newline
characters, and all data after the first % will be sent to the command
as
standard input. There is no way to split a single command line
onto
multiple lines, like the shell’s trailing "\".
There's another characteristic problem that can affect programs run by cron as compared with the command line (other than the interpretation of '%' signs described by Robert Gamble).
That difference is in the environment. If the program run relies on special environment variables, then it will work when you run it from the command line, with the environment you normally use, and it will likely work if you run it with at because that captures the environment when you create the job. But cron does no special environment setting.
I habitually, therefore, configure cron to run scripts by absolute pathname, and that script does the environment setting that I need (adds my $HOME/bin directory to PATH, for example). I even have a standardized infrastructure for this - a shell script that sets the environment and runs other programs.
# #(#)$Id: crontab,v 4.2 2007/09/17 02:41:00 jleffler Exp $
# Crontab file for Home Directory for Jonathan Leffler (JL)
#-----------------------------------------------------------------------------
#Min Hour Day Month Weekday Command
#-----------------------------------------------------------------------------
0 * * * * /usr/bin/ksh /work1/jleffler/bin/Cron/hourly
1 1 * * * /usr/bin/ksh /work1/jleffler/bin/Cron/daily
23 1 * * 1-5 /usr/bin/ksh /work1/jleffler/bin/Cron/weekday
2 3 * * 0 /usr/bin/ksh /work1/jleffler/bin/Cron/weekly
21 3 1 * * /usr/bin/ksh /work1/jleffler/bin/Cron/monthly
The script in /work1/jleffler/bin/Cron sets the environment and then runs the script of the same name in /work1/jleffler/bin to do the real work. The names in the Cron sub-directory are actually all links to the same script.