Use crontab job send mail, The email text turns to an attached file which named ATT00001.bin - linux

I want to analysis some data in one linux server,then send the it as Email text to my Email account , But when i execute this shell scripts in shell command, It works well, Weird is that when i put all the procedure into crontab job, The Email text will turns to an attached file, Can someone help?
#* * * * * sh -x /opt/bin/exec.sh >> /opt/bin/mailerror 2>&1
/* exec.sh */
#/bin/sh
cd /opt/bin
./analysis.sh > test
mail -s "Today's Weather" example#example.com < test
But when i execute exec.sh in shell command line directly, The Email will get text, Can someone explain it for me, grate thanks.

Ran into the same problem myself, only I'm piping text output into mailx - Heirloom mailx 12.4 7/29/08
When running the script on the command line the email came out as normal email with a text body.
However, when I ran the exact same script via crontab the body of the email came as an attachment - ATT00001.BIN (Outlook), application/octet-stream (mutt) or "noname" (Gmail).
Took some research to figure this out, but here goes:
Problem
Mailx will, if it encounters unknown / control characters in text input, convert it into an attachment with application/octet-stream mime-type set.
From the man page:
for any file that contains formatting characters other than newlines and horizontal tabulators
So you need to remove those control characters, which can be done with i.e. tr
echo "$Output" | /usr/bin/tr -cd '\11\12\15\40-\176' | mail ...
However since I had Norwegian UTF8 characters: æøå - the list expand, and you don't really want to maintain such a list, and I need the norwegian characters.
And inspecting the attachment I found I had only \r, \n the "regular" ASCII characters in range 32-176 - all printable and 184 and 195 --> UTF8
Sollution
Explicitly set the locale in your script:
LANG="en_US.UTF8" ; export LANG
Run export in your shell - or setenv if you run csh or tcsh to determine what your locale is set to.
Explanation
Mailx - when run in your shell - with LANG set to .UTF8, will correctly identify the UTF8 chars and continue.
When run in crontab LANG is not set, and default to LANG=C, since by default crontab will run only a restricted set of environment variables (system dependant).
mailx (or other programs) will then not recognize UTF8 characters and determine that the input containes unknown control characters.
My issue was UTF8 characters, yours could be other control characters in your input. Run it through hexdump or od -c, but since it works OK in a regular shell I'm suspecting LANG issues.
References:
linux mail < file.log has Content-Type: application/octet-stream (a noname attachment in Gmail)
http://alvinalexander.com/blog/post/linux-unix/how-remove-non-printable-ascii-characters-file-unix

I had this same issue and none of the above fixed the problem. Moving the extra return in the file fixed the issue for me:
cat logfile | tr -d \\r | mailx -s'the logfile' to-me#.....
Thanks to this forum:
https://forums.opensuse.org/showthread.php/445955-mailx-creates-unwanted-attachment

Make sure you change this in your script
#/bin/sh
to be replaced by
#!/bin/sh
Coming to the problem
Your script assumes that it is being run from a particular directory (note that almost every path is a relative path, not an absolute path). cron happens to be running it from another directory.
The Fix for text appearing on email
mydir=$(dirname "$0") && cd "${mydir}" || exit 1
./opt/bin/analysis.sh > test
mail -s "Today's Weather" example#example.com < /opt/bin/test
Explanation
$0 is the (possibly relative) filename of the shell script being executed. Given a filename, the dirname command returns the directory containing the filename.
So, that line changes directories to the directory containing the script or exits with an error code if either dirname or cd fails.
OR try to have full path like
./opt/bin/analysis.sh > test
mail -s "Today's Weather" example#example.com < /opt/bin/test
Note: The same problem is discussed earlier here
FOLLOW UP:
Try to remove
sh -x /opt/bin/exec.sh >> /opt/bin/mailerror 2>&1
and instead use
sh /opt/bin/exec.sh 2>&1 >> /opt/bin/mailerror
FOLLOW UP
You have to restart cron for changes to take effect if you do not use the crontab command to edit the file.
crontab -l > oldcrontab
cp oldcrontab newcrontab
echo "$newline" >> newcrontab
crontab < newcrontab

In my case, the cron was not a shell script but a PHP script (so I couldn't put the export LANG thing):
0 9 * * * apache php /test/myscript.php | mail -s "CRON - myscript" foo#bar.com
Solution:
In order to fix the same issue (content is mailed as attachment instead of body), I add LANG=fr_FR.UTF-8 at the beginning of the cron file:
MAILTO=vme1.etc-crond-backoffice-conf
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
LANG=fr_FR.UTF-8
0 9 * * * apache php /test/myscript.php | mail -s "CRON - myscript" foo#bar.com
NB: puting LANG=fr_FR.UTF-8 in the /etc/environment file and restarting cron service worked too.
Reference:
Set LANG in crontab https://www.logikdev.com/2010/02/02/locale-settings-for-your-cron-job/

Related

adding the .txt file's content in the body of the message in the mail of mailx command in linux

I have written a script where i am storing the name of the top 10 processes which are using the greatest ram of the cpu with a command:
ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head >/etc/tmp/file.txt
Now I am attaching this file in the mailx command to send a mail with its body as content with the command:
echo -e "Warning, server memory is running low!\n\nFree memory: $free MB\n$file" | mailx -a "ALERT" -s "$subject" -r "$from" -c "$also_to" "$to"
this "subject" , "from", "also_to", "to" are some variables and have actual mailid defined in the shell script.
when I get receive the mail I get the content in a very rough order like the picture below that i have added.that mail's pic
I want this content to of the file to be in proper format or if I could just send this txt file as an attachment, I made a whole study for sending the file as an attachment using the mailx command but to no avail.
You are misusing the -a option. You are supposed to use it to add a mail header (Header: value), not the single word "ALERT" like you did.
As to how to send the file as an attachment, I usually use the -A option with no trouble (at worst, I vaguely remember that sometimes I had to specify the file encoding so that accented characters are correctly displayed, but you don't need this with your ASCII file)

sed output in bash script works in CLI, output different in cron

A simple script to list sites on my Webinoly Ubuntu 18 server works in the CLI but fails in cron. Webinoly is a site management script and has a command to list the sites it is managing:
site -list
The output from this command in the CLI looks like this:
- catalyst.dk39ecbk3.com
- siteexample3.com
- webinoly.dk39ecbk3.com
The script I'm having trouble with (below) should remove the control characters, the " - " from the beginning of each line, and the blank lines using sed:
#!/bin/bash
# create array of sites using webinoly 'site' command
# webinoly site -list command returns lines that start with hyphens and spaces, along with control chars, which need to be removed
# SED s/[\x01-\x1F\x7F]//g removes control characters
# SED s/.{7}// removes first seven chars and s/.{5}$// removes last 5 chars
# SED /^\s*$/d removes blank lines
# removing characters http://www.theunixschool.com/2014/08/sed-examples-remove-delete-chars-from-line-file.html
# removing empty lines https://stackoverflow.com/questions/16414410/delete-empty-lines-using-sed
SITELIST=($(/usr/bin/site -list | sed -r "s/[\x01-\x1F\x7F]//g;s/.{7}//;s/.{5}$//;/^\s*$/d"))
#print site list
for SITE in ${SITELIST[#]}; do
echo "$SITE"
done
Here's the desired output, which I see in the CLI:
root#server1 ~/scripts # ./gdrive-backup-test.sh
catalyst.dk39ecbk3.com
siteexample3.com
webinoly.dk39ecbk3.com
The trouble happens when the script runs in cron. Here's the cron file:
root#server1 ~/scripts # crontab -l
SHELL=/bin/bash
MAILTO=myemail#gmail.com
15 3 * * 7 certbot renew --post-hook "service nginx restart"
47 01 * * * /root/scripts/gdrive-backup-test.sh > /root/scripts/output-gdrive-backup.txt
and here's the output-gdrive-backup.txt file generated by the cron command:
root#server1 ~/scripts # cat output-gdrive-backup.txt
lyst.dk39ecbk3
example3
noly.dk39ecbk3
The first three characters of each line are missing, as are the last four (the .com).
I've researched and made sure to force use of bash in the cron file as well as at the beginning of the script.
With the following input:
$ cat site
- catalyst.dk39ecbk3.com
- siteexample3.com
- webinoly.dk39ecbk3.com
- webinoly.dk39ecbk3.com
you can use the following sed command to reach your output:
$ cat site | sed -e "s/^\s*-\s*//g;/^\s*$/d"
catalyst.dk39ecbk3.com
siteexample3.com
webinoly.dk39ecbk3.com
webinoly.dk39ecbk3.com
Replace the cat site by the command you want to filter the output from.
The answer turned out to be failure to specify TERM in my cron file. That solved the main issue I was having. This was a strange issue -- hard to research and figure out.
There were a few others -- one of them was that the path for one of the commands wasn't part of the path cron uses, but was included in the user root in the CLI. For more info on the TERM issue, see "tput: No value for $TERM and no -T specified " error logged by CRON process.

Bash empty variable after running cron, but runs manually

I have simple bash script that working manual at the terminal, but after cron gives an empty variable.
#!/bin/bash
gwip=`/usr/bin/nmcli dev list iface eth0 | grep IP4-SETTINGS.GATEWAY: | awk '{ print $2}'`
printf '%s\n' "$(date) =- $gwip -= " >> /var/log/looog.log
run: /bin/bash /test.bash
output in file /var/log/looog.log:
1 monday 2016 14:17:36 +0300 =- 23.18.117.254 -=
When I run through cron, variable is empty.
*/1 * * * * root /bin/bash /test.bash
output in file /var/log/looog.log:
1 monday 2016 14:19:13 +0300 =- -=
Why variable $gwip is empty? how to fix it?
Qualifying /usr/bin/nmcli isn't enough -- you're calling a bunch of other tools that need to be found from the PATH also.
Also, in general -- when debugging a cron job, arrange for its stderr to go to a file, like so:
#!/bin/bash
# log stdout and stderr to two different files
exec >>/var/log/looog.log 2>>/var/log/looog.err.log
# ...and log every command we try to execute to stderr (aka looog.err.log)
set -x
# set a PATH variable
export PATH=/bin:/usr/bin
# original code here, using modern POSIX $() syntax, vs old hard-to-nest ``
gwip=$(nmcli dev list iface eth0 | awk '/IP4-SETTINGS[.]GATEWAY:/ { print $2}')
printf '%s\n' "$(date) =- $gwip -= "
The key things here are the explicitly-set PATH (not having the value you expect set in PATH is a common issue in cron jobs), and the stderr log (which ensures that any other issues can be identified by reading its contents).
Note the use of a single redirection to looog.log up-front. This doesn't make a significant difference when you're literally only running one print statement, but if you would have extended this script to have more than one, it's more efficient to open your output file only one than to re-open it every time you have something to write.

PHP iconv error using Zend Lucene when executing script via cron, but not on commandline

I am executing a PHP script via the command line which, for a specific user, runs fine when executed on the commandline, but when the exact same command is put into the same user's crontab, a PHP iconv error is returned.
The commandline is utilising the Yii framework and the Zend Lucene library, but I'm not sure if that's pertinent.
I've made all executable and script paths absolute in the crontab line and can verify that it works when executed directly on the commandline.
I wrapped the actual PHP invocation in a one-line shell script, as I read elsewhere here that this solved a similar problem for someone, but no joy.
The command successfully executed on the commandline is:
/bin/sh /var/www/yii-projects/projectname/protected/scripts/buildIndex.sh >> /var/lucene/lucene.log
The content of the buildIndex.sh script is:
/usr/bin/php /var/www/yii-projects/projectname/protected/scripts/cron.php lucene buildIndex
And the crontab line is:
*/10 * * * * /bin/sh /var/www/yii-projects/projectname/protected/scripts/buildIndex.sh >> /var/lucene/lucene.log
The error shown in the log when the crontab executes is:
PHP Error[8]: iconv(): Detected an illegal character in input string
in file /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Analysis/Analyzer/Common/Text.php at line 58
0 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Analysis/Analyzer/Common/Text.php(58): iconv()
1 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Analysis/Analyzer.php(125): Zend_Search_Lucene_Analysis_Analyzer_Common_Text_CaseInsensitive->reset()
2 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Index/SegmentWriter/DocumentWriter.php(98): Zend_Search_Lucene_Analysis_Analyzer_Common_Text_CaseInsensitive->setInput()
3 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Index/Writer.php(244): Zend_Search_Lucene_Index_SegmentWriter_DocumentWriter->addDocument()
4 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene.php(1410): Zend_Search_Lucene_Index_Writer->addDocument()
5 /var/www/yii-projects/projectname/protected/vendors/Zend/Search/Lucene/Proxy.php(500): Zend_Search_Lucene->addDocument()
6 /var/www/yii-projects/projectname/protected/commands/LuceneCommand.php(97): Zend_Search_Lucene_Proxy->addDocument()
7 unknown(0): LuceneCommand->actionBuildIndex()
8 /var/www/yii-projects/yii-1.1.12.b600af/framework/console/CConsoleCommand.php(173): ReflectionMethod->invokeArgs()
9 /var/www/yii-projects/yii-1.1.12.b600af/framework/console/CConsoleCommandRunner.php(68): LuceneCommand->run()
10 /var/www/yii-projects/yii-1.1.12.b600af/framework/console/CConsoleApplication.php(92): CConsoleCommandRunner->run()
11 /var/www/yii-projects/yii-1.1.12.b600af/framework/base/CApplication.php(162): CConsoleApplication->processRequest()
12 /var/www/yii-projects/projectname/protected/scripts/cron.php(14): CConsoleApplication->run()
I cannot think of any reason why there is any difference, given the measures taken, and the fact that the user is the same in both cases.
Please help!
Thanks
Edit - I should also confirm that the underlying data that is being indexed is not changing - I've executed both scenarios alternately many times and get the above results consistently.
Try with the -f switch and directly from crontab:
/usr/bin/php -f /var/www/yii-projects/projectname/protected/scripts/cron.php lucene buildIndex
Also are you sure that the text of the command you are passing is in UTF8? Could there be some other symbol there? Maybe a BOM? You can check this with a HEX editor - open your shell script, omit all the letters and see what's left. Usually a BOM in UTF8 is EF BB BF but it may not be a bom at all. Just check.
Necessary shell environment variables are not available to crontab job, so added this to irishhp users's crontab:
PATH=/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
LANG=en_US.UTF-8
which resolved.

Cron error with using backquotes

The following works fine from command line
/usr/bin/mysqldump -uUser -pPass Db_name > /var/www/db_backup/db.`date +%Y%m%d%H%M`.sql
but when I try to do that in cron, I get the error:
bad ` sign
errors in crontab file, can't install
I saw someone else on the net solve the same problem by escaping the percent signs, but that didn't help and I tried it with just date inside backquotes with no format specifiers and still got the errors.
I've also seen date's argument enclosed in single or double quotes, but that doesn't help either.
Granted I could just throw it into a script and execute that I suppose - but what fun is that?
Any ideas? I'm using RHEL 5.
Try it with $() instead of backticks. And you probably do need to escape the percent signs since cron converts them to newlines otherwise.
* 0 * * * /usr/bin/mysqldump -uUser -pPass Db_name > /var/www/db_backup/db.$(date +\%Y\%m\%d\%H\%M).sql
Also, you should store the password in an option file with secure permissions (eg. 600 or 640) instead of passing it on the command line.
Put your one line script (as shown) into a proper script file and invoke that from cron:
$ cat /usr/local/bin/db-backup
#!/bin/sh
/usr/bin/mysqldump -uUser -pPass Db_name > \
/var/www/db_backup/db.`date +%Y%m%d%H%M`.sql
$ # use RHEL commands to add db-backup to your crontab

Resources