The following works fine from command line
/usr/bin/mysqldump -uUser -pPass Db_name > /var/www/db_backup/db.`date +%Y%m%d%H%M`.sql
but when I try to do that in cron, I get the error:
bad ` sign
errors in crontab file, can't install
I saw someone else on the net solve the same problem by escaping the percent signs, but that didn't help and I tried it with just date inside backquotes with no format specifiers and still got the errors.
I've also seen date's argument enclosed in single or double quotes, but that doesn't help either.
Granted I could just throw it into a script and execute that I suppose - but what fun is that?
Any ideas? I'm using RHEL 5.
Try it with $() instead of backticks. And you probably do need to escape the percent signs since cron converts them to newlines otherwise.
* 0 * * * /usr/bin/mysqldump -uUser -pPass Db_name > /var/www/db_backup/db.$(date +\%Y\%m\%d\%H\%M).sql
Also, you should store the password in an option file with secure permissions (eg. 600 or 640) instead of passing it on the command line.
Put your one line script (as shown) into a proper script file and invoke that from cron:
$ cat /usr/local/bin/db-backup
#!/bin/sh
/usr/bin/mysqldump -uUser -pPass Db_name > \
/var/www/db_backup/db.`date +%Y%m%d%H%M`.sql
$ # use RHEL commands to add db-backup to your crontab
Related
A simple script to list sites on my Webinoly Ubuntu 18 server works in the CLI but fails in cron. Webinoly is a site management script and has a command to list the sites it is managing:
site -list
The output from this command in the CLI looks like this:
- catalyst.dk39ecbk3.com
- siteexample3.com
- webinoly.dk39ecbk3.com
The script I'm having trouble with (below) should remove the control characters, the " - " from the beginning of each line, and the blank lines using sed:
#!/bin/bash
# create array of sites using webinoly 'site' command
# webinoly site -list command returns lines that start with hyphens and spaces, along with control chars, which need to be removed
# SED s/[\x01-\x1F\x7F]//g removes control characters
# SED s/.{7}// removes first seven chars and s/.{5}$// removes last 5 chars
# SED /^\s*$/d removes blank lines
# removing characters http://www.theunixschool.com/2014/08/sed-examples-remove-delete-chars-from-line-file.html
# removing empty lines https://stackoverflow.com/questions/16414410/delete-empty-lines-using-sed
SITELIST=($(/usr/bin/site -list | sed -r "s/[\x01-\x1F\x7F]//g;s/.{7}//;s/.{5}$//;/^\s*$/d"))
#print site list
for SITE in ${SITELIST[#]}; do
echo "$SITE"
done
Here's the desired output, which I see in the CLI:
root#server1 ~/scripts # ./gdrive-backup-test.sh
catalyst.dk39ecbk3.com
siteexample3.com
webinoly.dk39ecbk3.com
The trouble happens when the script runs in cron. Here's the cron file:
root#server1 ~/scripts # crontab -l
SHELL=/bin/bash
MAILTO=myemail#gmail.com
15 3 * * 7 certbot renew --post-hook "service nginx restart"
47 01 * * * /root/scripts/gdrive-backup-test.sh > /root/scripts/output-gdrive-backup.txt
and here's the output-gdrive-backup.txt file generated by the cron command:
root#server1 ~/scripts # cat output-gdrive-backup.txt
lyst.dk39ecbk3
example3
noly.dk39ecbk3
The first three characters of each line are missing, as are the last four (the .com).
I've researched and made sure to force use of bash in the cron file as well as at the beginning of the script.
With the following input:
$ cat site
- catalyst.dk39ecbk3.com
- siteexample3.com
- webinoly.dk39ecbk3.com
- webinoly.dk39ecbk3.com
you can use the following sed command to reach your output:
$ cat site | sed -e "s/^\s*-\s*//g;/^\s*$/d"
catalyst.dk39ecbk3.com
siteexample3.com
webinoly.dk39ecbk3.com
webinoly.dk39ecbk3.com
Replace the cat site by the command you want to filter the output from.
The answer turned out to be failure to specify TERM in my cron file. That solved the main issue I was having. This was a strange issue -- hard to research and figure out.
There were a few others -- one of them was that the path for one of the commands wasn't part of the path cron uses, but was included in the user root in the CLI. For more info on the TERM issue, see "tput: No value for $TERM and no -T specified " error logged by CRON process.
I want to analysis some data in one linux server,then send the it as Email text to my Email account , But when i execute this shell scripts in shell command, It works well, Weird is that when i put all the procedure into crontab job, The Email text will turns to an attached file, Can someone help?
#* * * * * sh -x /opt/bin/exec.sh >> /opt/bin/mailerror 2>&1
/* exec.sh */
#/bin/sh
cd /opt/bin
./analysis.sh > test
mail -s "Today's Weather" example#example.com < test
But when i execute exec.sh in shell command line directly, The Email will get text, Can someone explain it for me, grate thanks.
Ran into the same problem myself, only I'm piping text output into mailx - Heirloom mailx 12.4 7/29/08
When running the script on the command line the email came out as normal email with a text body.
However, when I ran the exact same script via crontab the body of the email came as an attachment - ATT00001.BIN (Outlook), application/octet-stream (mutt) or "noname" (Gmail).
Took some research to figure this out, but here goes:
Problem
Mailx will, if it encounters unknown / control characters in text input, convert it into an attachment with application/octet-stream mime-type set.
From the man page:
for any file that contains formatting characters other than newlines and horizontal tabulators
So you need to remove those control characters, which can be done with i.e. tr
echo "$Output" | /usr/bin/tr -cd '\11\12\15\40-\176' | mail ...
However since I had Norwegian UTF8 characters: æøå - the list expand, and you don't really want to maintain such a list, and I need the norwegian characters.
And inspecting the attachment I found I had only \r, \n the "regular" ASCII characters in range 32-176 - all printable and 184 and 195 --> UTF8
Sollution
Explicitly set the locale in your script:
LANG="en_US.UTF8" ; export LANG
Run export in your shell - or setenv if you run csh or tcsh to determine what your locale is set to.
Explanation
Mailx - when run in your shell - with LANG set to .UTF8, will correctly identify the UTF8 chars and continue.
When run in crontab LANG is not set, and default to LANG=C, since by default crontab will run only a restricted set of environment variables (system dependant).
mailx (or other programs) will then not recognize UTF8 characters and determine that the input containes unknown control characters.
My issue was UTF8 characters, yours could be other control characters in your input. Run it through hexdump or od -c, but since it works OK in a regular shell I'm suspecting LANG issues.
References:
linux mail < file.log has Content-Type: application/octet-stream (a noname attachment in Gmail)
http://alvinalexander.com/blog/post/linux-unix/how-remove-non-printable-ascii-characters-file-unix
I had this same issue and none of the above fixed the problem. Moving the extra return in the file fixed the issue for me:
cat logfile | tr -d \\r | mailx -s'the logfile' to-me#.....
Thanks to this forum:
https://forums.opensuse.org/showthread.php/445955-mailx-creates-unwanted-attachment
Make sure you change this in your script
#/bin/sh
to be replaced by
#!/bin/sh
Coming to the problem
Your script assumes that it is being run from a particular directory (note that almost every path is a relative path, not an absolute path). cron happens to be running it from another directory.
The Fix for text appearing on email
mydir=$(dirname "$0") && cd "${mydir}" || exit 1
./opt/bin/analysis.sh > test
mail -s "Today's Weather" example#example.com < /opt/bin/test
Explanation
$0 is the (possibly relative) filename of the shell script being executed. Given a filename, the dirname command returns the directory containing the filename.
So, that line changes directories to the directory containing the script or exits with an error code if either dirname or cd fails.
OR try to have full path like
./opt/bin/analysis.sh > test
mail -s "Today's Weather" example#example.com < /opt/bin/test
Note: The same problem is discussed earlier here
FOLLOW UP:
Try to remove
sh -x /opt/bin/exec.sh >> /opt/bin/mailerror 2>&1
and instead use
sh /opt/bin/exec.sh 2>&1 >> /opt/bin/mailerror
FOLLOW UP
You have to restart cron for changes to take effect if you do not use the crontab command to edit the file.
crontab -l > oldcrontab
cp oldcrontab newcrontab
echo "$newline" >> newcrontab
crontab < newcrontab
In my case, the cron was not a shell script but a PHP script (so I couldn't put the export LANG thing):
0 9 * * * apache php /test/myscript.php | mail -s "CRON - myscript" foo#bar.com
Solution:
In order to fix the same issue (content is mailed as attachment instead of body), I add LANG=fr_FR.UTF-8 at the beginning of the cron file:
MAILTO=vme1.etc-crond-backoffice-conf
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
LANG=fr_FR.UTF-8
0 9 * * * apache php /test/myscript.php | mail -s "CRON - myscript" foo#bar.com
NB: puting LANG=fr_FR.UTF-8 in the /etc/environment file and restarting cron service worked too.
Reference:
Set LANG in crontab https://www.logikdev.com/2010/02/02/locale-settings-for-your-cron-job/
I am executing a PHP script via the command line which, for a specific user, runs fine when executed on the commandline, but when the exact same command is put into the same user's crontab, a PHP iconv error is returned.
The commandline is utilising the Yii framework and the Zend Lucene library, but I'm not sure if that's pertinent.
I've made all executable and script paths absolute in the crontab line and can verify that it works when executed directly on the commandline.
I wrapped the actual PHP invocation in a one-line shell script, as I read elsewhere here that this solved a similar problem for someone, but no joy.
The command successfully executed on the commandline is:
/bin/sh /var/www/yii-projects/projectname/protected/scripts/buildIndex.sh >> /var/lucene/lucene.log
The content of the buildIndex.sh script is:
/usr/bin/php /var/www/yii-projects/projectname/protected/scripts/cron.php lucene buildIndex
And the crontab line is:
*/10 * * * * /bin/sh /var/www/yii-projects/projectname/protected/scripts/buildIndex.sh >> /var/lucene/lucene.log
The error shown in the log when the crontab executes is:
PHP Error[8]: iconv(): Detected an illegal character in input string
in file /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Analysis/Analyzer/Common/Text.php at line 58
0 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Analysis/Analyzer/Common/Text.php(58): iconv()
1 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Analysis/Analyzer.php(125): Zend_Search_Lucene_Analysis_Analyzer_Common_Text_CaseInsensitive->reset()
2 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Index/SegmentWriter/DocumentWriter.php(98): Zend_Search_Lucene_Analysis_Analyzer_Common_Text_CaseInsensitive->setInput()
3 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Index/Writer.php(244): Zend_Search_Lucene_Index_SegmentWriter_DocumentWriter->addDocument()
4 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene.php(1410): Zend_Search_Lucene_Index_Writer->addDocument()
5 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Proxy.php(500): Zend_Search_Lucene->addDocument()
6 /var/www/yii-projects/projectname/protected/commands/LuceneCommand.php(97): Zend_Search_Lucene_Proxy->addDocument()
7 unknown(0): LuceneCommand->actionBuildIndex()
8 /var/www/yii-projects/yii-1.1.12.b600af/framework/console/CConsoleCommand.php(173): ReflectionMethod->invokeArgs()
9 /var/www/yii-projects/yii-1.1.12.b600af/framework/console/CConsoleCommandRunner.php(68): LuceneCommand->run()
10 /var/www/yii-projects/yii-1.1.12.b600af/framework/console/CConsoleApplication.php(92): CConsoleCommandRunner->run()
11 /var/www/yii-projects/yii-1.1.12.b600af/framework/base/CApplication.php(162): CConsoleApplication->processRequest()
12 /var/www/yii-projects/projectname/protected/scripts/cron.php(14): CConsoleApplication->run()
I cannot think of any reason why there is any difference, given the measures taken, and the fact that the user is the same in both cases.
Please help!
Thanks
Edit - I should also confirm that the underlying data that is being indexed is not changing - I've executed both scenarios alternately many times and get the above results consistently.
Try with the -f switch and directly from crontab:
/usr/bin/php -f /var/www/yii-projects/projectname/protected/scripts/cron.php lucene buildIndex
Also are you sure that the text of the command you are passing is in UTF8? Could there be some other symbol there? Maybe a BOM? You can check this with a HEX editor - open your shell script, omit all the letters and see what's left. Usually a BOM in UTF8 is EF BB BF but it may not be a bom at all. Just check.
Necessary shell environment variables are not available to crontab job, so added this to irishhp users's crontab:
PATH=/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
LANG=en_US.UTF-8
which resolved.
I've created a Cron task at my webhost to daily backup my database and I would like it to append the current date to the filename.
My Cron job looks like this
mysqldump -u username -pPassword db_name > www/db_backup/db_backup+date%d%m%y.sql
But the file I get is this: db_backup+date no file extension or date.
I've also tried this command
mysqldump -u username -pPassword db_name > www/db_backup/db_backup_'date +%d%m%y'.sql
but that doesn't even give an file output.
What is the right syntax for getting the date appended to my file??
* * * * * echo "hello" > /tmp/helloFile_$(date +\%Y\%m\%d\%H\%M\%S).txt
You just need to escape the percent signs.
Other date formats:
http://www.cyberciti.biz/faq/linux-unix-formatting-dates-for-display/
You should use `` instead of '' around the command you want to execute to generate the current date for your filename.
You must escape the format and use evaluation
mysqldump -u username -pPassword db_name > www/db_backup/db_backup_`date +\%d\%m\%y`.sql
I need to create a new log file every time command is executed. So every day I should have a log like this /home/me/Logs/power_20151230.log
The crontab line that I use is this:
00 8 * * * /home/me/power.py ON >> /home/me/Logs/power\_`date +20\%y\%m\%d`
Note the underscore character must be escaped too.
How can I store variables in my crontab? I realize it's not shell but say I want to have some constants like a path to my app or something?
In Vixie cron, which is possibly the most common, you can do this almost exactly like a shell script.
VARIABLE=value
PATH=/bin:/path/to/doathing
0 0 * * * doathing.sh $VARIABLE
The man page says:
An active line in a crontab will be either an environment setting or a cron command. An environment setting is of the form,
name = value
where the spaces around the equal-sign (=) are optional, and any subsequent non-leading spaces in value will be part of the value assigned
to name. The value string may be placed in quotes (single or double, but matching) to preserve leading or trailing blanks. The name
string may also be placed in quote (single or double, but matching) to preserve leading, trailing or inner blanks.
You can tell if you have Vixie cron by checking the man page for crontab; the author will be Paul Vixie. Different crons may or may not support this (BusyBox's cron for example, does not), in which case your best option is to wrap your command in a shell script and run that script from cron instead. In fact, this is a good thing to do for anything complicated.
To keep my crontab clean, I would just call a shell script and do the fun stuff in the script.
I think the important fact to point out here is (as stated in an earlier comment by Pierre D Mar 25, 2015 at 18:58) that variable declarations are not expand/interpolated and so can not embed other variable values.
Variables are only expanded/interpolated in the commands themselves.
So:
var1 = bar
var2 = foo${var1}
42 17 * * * /path/to/command ${var2}
Results in: /path/to/command foo${var1}
While:
var1 = bar
var2 = foo
42 17 * * * /path/to/command ${var2}${var1}
Results in: /path/to/command foobar
So in my case the following works fine, no wrapping in shell scripts required:
SHELL=/bin/bash
timestamp=date +20%y_%m_%d_%H_%M_%S
logdir=/my/log/dir
0 2 * * * /my/command/path/mycmd param >> ${logdir}/myfile_$(${timestamp}).log
verses something like this which does not work:
logfile = /my/log/dir/myfile_${timestamp}.log
since the later is not expanded, but is rather interpreted as is including "${" and "}" as part of the string.
Just a working example of using variables in the crontab file and their substitution in the strings:
CURRENT_TIME=date +%Y.%m.%d_%H:%M:%S.%3N
CURRENT_DATE=date +%Y_%m_%d
SIMPLE_VAR=the_simple_var
LOG_DIR=/var/log/cron
* * * * * /bin/echo "simple variable test! ${SIMPLE_VAR}__test!" >> "${LOG_DIR}/test.log"
* * * * * /bin/echo "complex variable test! $(${CURRENT_TIME})__test!" >> "${LOG_DIR}/test.log"
Tested on this Docker image (paste the above crontab to the crontab.txt):
FROM debian:10-slim
# Install docker (Yep, this is a docker in docker):
RUN curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh
# Install CRON:
RUN apt-get update && apt-get install -y --no-install-recommends cron
# Add a crontab_config.txt task:
COPY crontab.txt /var/crontab.txt
RUN crontab /var/crontab.txt
ENTRYPOINT ["cron", "-f"]
Add this to the crontab to run any commands inside another docker containers:
/usr/bin/docker exec container_name ls -la
If you have a few environment variables you want to set for a particular job, just inline those into the sh snippet.
42 17 * * * myvariable='value' path/to/command
In case it's not obvious, the sh syntax var=value command sets var to value for the duration of command. You can have several of these if you need more than one.
42 17 * * * firstname='Fred` lastname='Muggs' path/to/command
If you have nontrivial variables you want to access from several places, probably put them in a separate file, and source (.) that file from your shell startup script and your cron jobs.
Let's say you have a file $HOME/bin/myenv with settings like
myvar=$(output of some complex command)
another='another
complex
variable'
then you can add
. $HOME/bin/myenv
to your .profile (or .zshrc or .bash_profile or what have you; but .profileis portable, and also used bysh`) and
42 17 * * * . $HOME/bin/myenv; path/to/command
in your crontab.
Notice the lone dot before the space before the file name; this is the dot command (also known as source in e.g. Bash) which reads the file into the current shell instance as if you had typed in the things in the file here.
Tangentially, the $HOME/ part is strictly speaking redundant; cron jobs will always run in the invoking user's home directory.
Obviously, if you want a variable to be true in your entire crontab, set it at the top, before the scheduled jobs.