The shell script:
echo '/bin/date -d "1 days ago" +"%x"'
When I run the shell script directly from terminal the output is:
06/07/2013\n
But while I run script in the crontab, the output is:
06/07/13\n
I hope the output should be the same with the upper one. If you know the reason, can you share with me.
The %x flag means "locale's date representation". chrony has different locale settings compared to your user, to verify this try to execute the locale command under the cron user (ie adding * * * * * root locale > /root/cron-locale to /etc/crontab) ; you will see it is:
LANG=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=
while running locale for your user account would probably return something like:
LANG=en_US.utf8
LC_CTYPE="en_US.utf8"
LC_NUMERIC="en_US.utf8"
LC_TIME="en_US.utf8"
LC_COLLATE="en_US.utf8"
LC_MONETARY="en_US.utf8"
LC_MESSAGES="en_US.utf8"
LC_PAPER="en_US.utf8"
LC_NAME="en_US.utf8"
LC_ADDRESS="en_US.utf8"
LC_TELEPHONE="en_US.utf8"
LC_MEASUREMENT="en_US.utf8"
LC_IDENTIFICATION="en_US.utf8"
LC_ALL=
To have a consistent date representation, use: %Y , which means 4-chars year
Related
I am a lot confused about this behavior.
Each time I run this script from terminal, it works fine, but it fails once is executed from a crontab.
Inside the script you can find each step description.
The target is to print date and time with the peso variable to a file.
I have changed line 16 countless times. Same thing for line #4.
Edit for clarity: THIS IS JUST A SMALL PART FROM THE WHOLE SCRIPT.
It runs nice every minute. It does everything, except the peso issue.
Please HELP!!!
1 # Here I compute one decimal value (like z=0.123) with two integers (sd and mean)
2 peso=$( echo "scale=4; ${z}*${sd}/100 + ${mean}/100" | bc -l)
3 echo "peso_mtrx="$peso # This is for checking: shows 40.123 (example), so it is OK
4 peso+=";" # Add a semicolon to check its behaviour
5 echo "peso= "$peso # show it: OK
6 peso1=$(date "+%D %T") # Now I capture date and time
7 echo "fecha= "$peso1 # shows it, so it is OK
8 peso1+=";" # add a semicolon to date
9 peso1+=$peso # concatenate the peso variable
10 echo $(printf '%s' "$peso1") # shows it, so it is ok up to here
11 echo $(printf '%s' "$peso1") >> ~/projects/Files/normal.csv # WRITE TO FILE
12 # whenever I run this script from terminal, all variables showed right and even print all data into file.
13 # File stores a new line like: 02/03/21 08:24:40;40.1709;
14 # BUT... when it is executed from a CRON job... everything except peso are stored.
15 # File stores a line like: 02/03/21 08:24:40;; peso variable just vanishes.
16 # is it something related to subshells? how to solve this rigmarole?
As I was suspicious, the whole thing was related to subshell issues.
I just did something inside crontab.
Once I execute crontab -e, I initially had something like:
*/1 * * * * /absolute/path/to/project.sh
So doing some reading I ended up doing this:
SHELL=/bin/bash
*/1 * * * * exec bash -l /absolute/path/to/project.sh
I beg to an expert to enlighten us about this solution. As far as I do understand, it is related to create a login shell inside cron using the information stored in .bash_profile.
It did enable the environment variables to be reachable.
I want to analysis some data in one linux server,then send the it as Email text to my Email account , But when i execute this shell scripts in shell command, It works well, Weird is that when i put all the procedure into crontab job, The Email text will turns to an attached file, Can someone help?
#* * * * * sh -x /opt/bin/exec.sh >> /opt/bin/mailerror 2>&1
/* exec.sh */
#/bin/sh
cd /opt/bin
./analysis.sh > test
mail -s "Today's Weather" example#example.com < test
But when i execute exec.sh in shell command line directly, The Email will get text, Can someone explain it for me, grate thanks.
Ran into the same problem myself, only I'm piping text output into mailx - Heirloom mailx 12.4 7/29/08
When running the script on the command line the email came out as normal email with a text body.
However, when I ran the exact same script via crontab the body of the email came as an attachment - ATT00001.BIN (Outlook), application/octet-stream (mutt) or "noname" (Gmail).
Took some research to figure this out, but here goes:
Problem
Mailx will, if it encounters unknown / control characters in text input, convert it into an attachment with application/octet-stream mime-type set.
From the man page:
for any file that contains formatting characters other than newlines and horizontal tabulators
So you need to remove those control characters, which can be done with i.e. tr
echo "$Output" | /usr/bin/tr -cd '\11\12\15\40-\176' | mail ...
However since I had Norwegian UTF8 characters: æøå - the list expand, and you don't really want to maintain such a list, and I need the norwegian characters.
And inspecting the attachment I found I had only \r, \n the "regular" ASCII characters in range 32-176 - all printable and 184 and 195 --> UTF8
Sollution
Explicitly set the locale in your script:
LANG="en_US.UTF8" ; export LANG
Run export in your shell - or setenv if you run csh or tcsh to determine what your locale is set to.
Explanation
Mailx - when run in your shell - with LANG set to .UTF8, will correctly identify the UTF8 chars and continue.
When run in crontab LANG is not set, and default to LANG=C, since by default crontab will run only a restricted set of environment variables (system dependant).
mailx (or other programs) will then not recognize UTF8 characters and determine that the input containes unknown control characters.
My issue was UTF8 characters, yours could be other control characters in your input. Run it through hexdump or od -c, but since it works OK in a regular shell I'm suspecting LANG issues.
References:
linux mail < file.log has Content-Type: application/octet-stream (a noname attachment in Gmail)
http://alvinalexander.com/blog/post/linux-unix/how-remove-non-printable-ascii-characters-file-unix
I had this same issue and none of the above fixed the problem. Moving the extra return in the file fixed the issue for me:
cat logfile | tr -d \\r | mailx -s'the logfile' to-me#.....
Thanks to this forum:
https://forums.opensuse.org/showthread.php/445955-mailx-creates-unwanted-attachment
Make sure you change this in your script
#/bin/sh
to be replaced by
#!/bin/sh
Coming to the problem
Your script assumes that it is being run from a particular directory (note that almost every path is a relative path, not an absolute path). cron happens to be running it from another directory.
The Fix for text appearing on email
mydir=$(dirname "$0") && cd "${mydir}" || exit 1
./opt/bin/analysis.sh > test
mail -s "Today's Weather" example#example.com < /opt/bin/test
Explanation
$0 is the (possibly relative) filename of the shell script being executed. Given a filename, the dirname command returns the directory containing the filename.
So, that line changes directories to the directory containing the script or exits with an error code if either dirname or cd fails.
OR try to have full path like
./opt/bin/analysis.sh > test
mail -s "Today's Weather" example#example.com < /opt/bin/test
Note: The same problem is discussed earlier here
FOLLOW UP:
Try to remove
sh -x /opt/bin/exec.sh >> /opt/bin/mailerror 2>&1
and instead use
sh /opt/bin/exec.sh 2>&1 >> /opt/bin/mailerror
FOLLOW UP
You have to restart cron for changes to take effect if you do not use the crontab command to edit the file.
crontab -l > oldcrontab
cp oldcrontab newcrontab
echo "$newline" >> newcrontab
crontab < newcrontab
In my case, the cron was not a shell script but a PHP script (so I couldn't put the export LANG thing):
0 9 * * * apache php /test/myscript.php | mail -s "CRON - myscript" foo#bar.com
Solution:
In order to fix the same issue (content is mailed as attachment instead of body), I add LANG=fr_FR.UTF-8 at the beginning of the cron file:
MAILTO=vme1.etc-crond-backoffice-conf
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
LANG=fr_FR.UTF-8
0 9 * * * apache php /test/myscript.php | mail -s "CRON - myscript" foo#bar.com
NB: puting LANG=fr_FR.UTF-8 in the /etc/environment file and restarting cron service worked too.
Reference:
Set LANG in crontab https://www.logikdev.com/2010/02/02/locale-settings-for-your-cron-job/
I've created a Cron task at my webhost to daily backup my database and I would like it to append the current date to the filename.
My Cron job looks like this
mysqldump -u username -pPassword db_name > www/db_backup/db_backup+date%d%m%y.sql
But the file I get is this: db_backup+date no file extension or date.
I've also tried this command
mysqldump -u username -pPassword db_name > www/db_backup/db_backup_'date +%d%m%y'.sql
but that doesn't even give an file output.
What is the right syntax for getting the date appended to my file??
* * * * * echo "hello" > /tmp/helloFile_$(date +\%Y\%m\%d\%H\%M\%S).txt
You just need to escape the percent signs.
Other date formats:
http://www.cyberciti.biz/faq/linux-unix-formatting-dates-for-display/
You should use `` instead of '' around the command you want to execute to generate the current date for your filename.
You must escape the format and use evaluation
mysqldump -u username -pPassword db_name > www/db_backup/db_backup_`date +\%d\%m\%y`.sql
I need to create a new log file every time command is executed. So every day I should have a log like this /home/me/Logs/power_20151230.log
The crontab line that I use is this:
00 8 * * * /home/me/power.py ON >> /home/me/Logs/power\_`date +20\%y\%m\%d`
Note the underscore character must be escaped too.
How can I store variables in my crontab? I realize it's not shell but say I want to have some constants like a path to my app or something?
In Vixie cron, which is possibly the most common, you can do this almost exactly like a shell script.
VARIABLE=value
PATH=/bin:/path/to/doathing
0 0 * * * doathing.sh $VARIABLE
The man page says:
An active line in a crontab will be either an environment setting or a cron command. An environment setting is of the form,
name = value
where the spaces around the equal-sign (=) are optional, and any subsequent non-leading spaces in value will be part of the value assigned
to name. The value string may be placed in quotes (single or double, but matching) to preserve leading or trailing blanks. The name
string may also be placed in quote (single or double, but matching) to preserve leading, trailing or inner blanks.
You can tell if you have Vixie cron by checking the man page for crontab; the author will be Paul Vixie. Different crons may or may not support this (BusyBox's cron for example, does not), in which case your best option is to wrap your command in a shell script and run that script from cron instead. In fact, this is a good thing to do for anything complicated.
To keep my crontab clean, I would just call a shell script and do the fun stuff in the script.
I think the important fact to point out here is (as stated in an earlier comment by Pierre D Mar 25, 2015 at 18:58) that variable declarations are not expand/interpolated and so can not embed other variable values.
Variables are only expanded/interpolated in the commands themselves.
So:
var1 = bar
var2 = foo${var1}
42 17 * * * /path/to/command ${var2}
Results in: /path/to/command foo${var1}
While:
var1 = bar
var2 = foo
42 17 * * * /path/to/command ${var2}${var1}
Results in: /path/to/command foobar
So in my case the following works fine, no wrapping in shell scripts required:
SHELL=/bin/bash
timestamp=date +20%y_%m_%d_%H_%M_%S
logdir=/my/log/dir
0 2 * * * /my/command/path/mycmd param >> ${logdir}/myfile_$(${timestamp}).log
verses something like this which does not work:
logfile = /my/log/dir/myfile_${timestamp}.log
since the later is not expanded, but is rather interpreted as is including "${" and "}" as part of the string.
Just a working example of using variables in the crontab file and their substitution in the strings:
CURRENT_TIME=date +%Y.%m.%d_%H:%M:%S.%3N
CURRENT_DATE=date +%Y_%m_%d
SIMPLE_VAR=the_simple_var
LOG_DIR=/var/log/cron
* * * * * /bin/echo "simple variable test! ${SIMPLE_VAR}__test!" >> "${LOG_DIR}/test.log"
* * * * * /bin/echo "complex variable test! $(${CURRENT_TIME})__test!" >> "${LOG_DIR}/test.log"
Tested on this Docker image (paste the above crontab to the crontab.txt):
FROM debian:10-slim
# Install docker (Yep, this is a docker in docker):
RUN curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh
# Install CRON:
RUN apt-get update && apt-get install -y --no-install-recommends cron
# Add a crontab_config.txt task:
COPY crontab.txt /var/crontab.txt
RUN crontab /var/crontab.txt
ENTRYPOINT ["cron", "-f"]
Add this to the crontab to run any commands inside another docker containers:
/usr/bin/docker exec container_name ls -la
If you have a few environment variables you want to set for a particular job, just inline those into the sh snippet.
42 17 * * * myvariable='value' path/to/command
In case it's not obvious, the sh syntax var=value command sets var to value for the duration of command. You can have several of these if you need more than one.
42 17 * * * firstname='Fred` lastname='Muggs' path/to/command
If you have nontrivial variables you want to access from several places, probably put them in a separate file, and source (.) that file from your shell startup script and your cron jobs.
Let's say you have a file $HOME/bin/myenv with settings like
myvar=$(output of some complex command)
another='another
complex
variable'
then you can add
. $HOME/bin/myenv
to your .profile (or .zshrc or .bash_profile or what have you; but .profileis portable, and also used bysh`) and
42 17 * * * . $HOME/bin/myenv; path/to/command
in your crontab.
Notice the lone dot before the space before the file name; this is the dot command (also known as source in e.g. Bash) which reads the file into the current shell instance as if you had typed in the things in the file here.
Tangentially, the $HOME/ part is strictly speaking redundant; cron jobs will always run in the invoking user's home directory.
Obviously, if you want a variable to be true in your entire crontab, set it at the top, before the scheduled jobs.
The following works fine from command line
/usr/bin/mysqldump -uUser -pPass Db_name > /var/www/db_backup/db.`date +%Y%m%d%H%M`.sql
but when I try to do that in cron, I get the error:
bad ` sign
errors in crontab file, can't install
I saw someone else on the net solve the same problem by escaping the percent signs, but that didn't help and I tried it with just date inside backquotes with no format specifiers and still got the errors.
I've also seen date's argument enclosed in single or double quotes, but that doesn't help either.
Granted I could just throw it into a script and execute that I suppose - but what fun is that?
Any ideas? I'm using RHEL 5.
Try it with $() instead of backticks. And you probably do need to escape the percent signs since cron converts them to newlines otherwise.
* 0 * * * /usr/bin/mysqldump -uUser -pPass Db_name > /var/www/db_backup/db.$(date +\%Y\%m\%d\%H\%M).sql
Also, you should store the password in an option file with secure permissions (eg. 600 or 640) instead of passing it on the command line.
Put your one line script (as shown) into a proper script file and invoke that from cron:
$ cat /usr/local/bin/db-backup
#!/bin/sh
/usr/bin/mysqldump -uUser -pPass Db_name > \
/var/www/db_backup/db.`date +%Y%m%d%H%M`.sql
$ # use RHEL commands to add db-backup to your crontab