cron & wget adding new line to each document write - cron

So I am trying to create a cron job that will, on a daily basis, execute a job on a webpage.
My cron command:
0 0 * * */1 wget -t 1 - "http://urlto.job/job9" -O ->> home/password_reset_log.txt
All of this works, however I am trying to carry out the seemingly menial task of adding a new line to each entry in the text file.
Currently I get:
{"error":0,"result":"Password reset email successfully sent to 0 users, 0 emails failed to send","jDateLastRun":"10 Jan 2018, 00:00:07","jHandle":"check_password_resets","jID":"9"}{"error":0,"result":"Password reset email successfully sent to 0 users, 0 emails failed to send","jDateLastRun":"10 Jan 2018, 00:00:07","jHandle":"check_password_resets","jID":"9"}
So all the entries are on a single line. What I want is:
{"error":0,"result":"Password reset email successfully sent to 0 users, 0 emails failed to send","jDateLastRun":"10 Jan 2018, 00:00:07","jHandle":"check_password_resets","jID":"9"}
{"error":0,"result":"Password reset email successfully sent to 0 users, 0 emails failed to send","jDateLastRun":"10 Jan 2018, 00:00:07","jHandle":"check_password_resets","jID":"9"}
So all the entries start on a new line (easier to read and see which jobs ran successfully and which failed and the result of each time the job ran).
Is there a simple way to do this (perhaps adding a parameter to my cron command).

Use | xargs echo to append the newline.
Your entry would look something like this:
0 0 * * */1 wget -t 1 - "http://urlto.job/job9" -O - | xargs echo >> home/password_reset_log.txt
Of course, this will only work if your URL always gives exactly one line each time.
Here the demo for everyone to enjoy:
user#host:~$ echo -n "No newline"
No newlineuser#host:~$ echo -n "With newline" | xargs echo
With newline
user#host:~$

Related

Crontab showing different time and Shell script when run manually shows different time stamp

#!/bin/sh
string1=$(date +"%T")
string2=$(date -r merchant.xml +"%T")
StartDate=$(date -d "$string1" +"%s")
FinalDate=$(date -d "$string2" +"%s")
echo Since, $(date -d "0 $StartDate sec - $FinalDate sec" +"%H:%M") HOURS, mail has not been updated | mail -s "Merchant File Staleness" hello#gmail.com
it is my shell script name hp.sh and output is
Since, 00:55 HOURS, mail has not been updated.
The crontab is
0 * * * * /tmp/hp.sh
and output of crontab is
Since, 07:04 HOURS, mail has not been updated.
The output of both is different. I need output of my shell script using crontab every hour.
Check the TZ variable, both system and user-defined - cron will happily oblige to whatever is set there, even if it is different from system settings.

Getting logs of cronjob script execution with logs in the body of the email

I'm trying to get the cron job to send me an email directly after the script execution and get the last logs into the body of the email
21 14 * * * /opt/anaconda/bin/python /Path/to/Script/script.py >> /Path/to/logfile/log.txt 2>&1 | mail -s "cronjob OK" "first#mail.x,second#mail.x"
How could I do that? what should I add?
Thanks in advance.
Your script is not producing any output because you just redirected it to a file.
cron will send any output by email to the owner of the cron job anyway. You can specify a different address with MAILTO in some cron implementations.
MAILTO=first#mail.x,second#mail.x
21 14 * * * /opt/anaconda/bin/python /Path/to/Script/script.py 2>&1 | tee -a /Path/to/logfile/log.txt
The tee command saves a copy of standard input to the file (-a says to append instead of overwrite) and to standard output.
If you need more control over the generated message (e.g. to use a different Subject: header if Cron's is not acceptable) maybe keep the tee but put back the pipe to mail.

Crontab not giving results

I am not in the root,
I entered the following commands in the crontab:
*/1 * * * * /home/ajain/testscript.sh
The file testscript.sh has the following commands:
#!/bin/bash
echo "The script begins now"
ping -c 2 live.com
echo The script has been run on `date` >> /home/ajain/testscript.log
echo "The script ends now"
exit
The crontab is not giving the results, however, the following command is giving the result in the testscript.log file correctly, displaying the ping date.
bash testscript.sh
Why is the crontab not working?
You can fix it in two different ways.
To provide full path to the script /home/ajain/testscript.sh. Here you don't even need to add bash because you have clearly mentioned in which shell your script should run i.e. first line of your script #!/bin/bash
Add this line before executing the script
set path=$path:/home/ajain/
testscript.sh # no need to use bash in front of it
Also providing execution permission to a script is not just enough. You need to check whether the user who is going to execute the script has permission to the location of the script or not. That means whether user can do a cd /home/ajain/ or not.
Hope this will help you.
remove >> /home/ajain/script.log, just add line to the top of file /home/ajain/testscript.sh:
#!/bin/bash
exec >> /home/ajain/script.log 2>&1
echo "The script begins now"
ping -c 2 live.com
echo "The script has been run on `date`"
echo "The script ends now"
exit
remove >> /home/ajain/script.log from crontab, just using:
*/1 * * * * /home/ajain/testscript.sh
You need do define script output.
Because of cron man page:
When executing commands, any output is mailed to the owner of the
crontab (or to the user named in the MAILTO environment variable in
the crontab, if such exists). The children copies of cron running
these processes have their name coerced to uppercase, as will be seen
in the syslog and ps output.
Fore everything else than echo The script has been run on 'date' >> /home/ajain/testscript.log you should check your/root's mail, or the syslog (eg. /var/log/syslog).
I could recommend make something like this:
*/1 * * * * /home/ajain/testscript.sh >> /home/ajain/script.log
#Edit
My crontab file on my user:
$crontab -l
# test
*/1 * * * * /home/3sky/tests/test.sh >> /home/3sky/tests/log.log
Script look like that:
#!/bin/bash
echo "The script begins now"
ping -c 2 live.com
echo The script has been run on `date`
echo "The script ends now"
exit
Permission on files:
test.sh - 0755/-rwxr-xr-x
log.log - 0644/-rw-r--r--
Output in log file:
$tail -20 log.log
2 packets transmitted, 0 received, 100% packet loss, time 10999ms
The script has been run on Thu Jun 21 12:18:12 CEST 2018
The script ends now
The script begins now
PING live.com (204.79.197.212) 56(84) bytes of data.
--- live.com ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 10999ms
The script has been run on Thu Jun 21 12:19:12 CEST 2018
The script ends now
The script begins now
PING live.com (204.79.197.212) 56(84) bytes of data.
--- live.com ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 10999ms
The script has been run on Thu Jun 21 12:20:12 CEST 2018
The script ends now

cron job not mailing output of urlwatch

I have one cron job like this:
* * * * * urlwatch | mail -s "job changes" pc_xxx#msn.com
It mails every minute as expected. However when I alter the test html page on my local server it doesn't email the differences, just continues to mail a blank mail with the title 'job changes'.
When I paste the job to a prompt:
pc#dellbox:/$urlwatch | mail -s "job changes" pc_xxx#msn.com
and run before/after changing the html, it emails the differences in the 2nd email as expected.
(pc is the owner of urls.txt and the cronjob was created by pc via crontab -e)
Why does the cron version not email the urlwatch output?
This is driving me batty...
Any/all help gratefully received.
ps couldn't create urlwatch as a new tag - need 1500 rep :(
Update:
If I split the command into two bits like this:
urlwatch > ~/.urlwatch/output.txt
mail -s "output" pc_xxx#msn.com < ~/.urlwatch/output.txt
This works.
If I join the two statements with a pipe like this:
urlwatch > ~/.urlwatch/output.txt | mail -s "output" pc_xxx#msn.com < ~/.urlwatch/output.txt
I get a prompt IMMEDIATELY that says
Null message body; hope that's ok
I notice urlwatch takes 2 - 3 seconds to complete, and I understand shell commands wait for preceding commands to finish (unless you're using &?). Dunno if this is significant.
Also, I'm using sSMTP...
Use full path when calling urlwatch:
$ which urlwatch
/usr/bin/urlwatch
Then your cron have to be::
* * * * * /usr/bin/urlwatch | mail -s "job changes" pc_xxx#msn.com
Put .urlwatch directories under /root/ as the program will look there to store its data
#askchipbug: your case looks very specific, but here are the things to do that may help:
-check /var/log/mail.err or /var/log/mail.log to find hints
-crontab user/permission
-some server strips content for not a good reason
-i use mailx it works fine for me and here is my cron job in crontab -e:
* * * * * urlwatch --urls=/var/www/html/urls.txt --hooks=/home/foo/hooks.py | mailx -v -r "bar#gmail.com" -s "This is the subject line" -S smtp="smtp.gmail.com:587" -S smtp-use-starttls -S smtp-auth=login -S smtp-auth-user="foo2#gmail.com" -S smtp-auth-password="bardaf16charsfoo" -S ssl-verify=ignore receipient#hotmail.com
i found the info for heirloom-mailx client installation from here: http://www.binarytides.com/linux-mail-with-smtp/
hope it helps.

Running bash script in cron

I have the following bash script:
!/bin/bash
# script to send simple email
#Wait 5 seconds
#sleep 05
# email subject
SUBJECT="SUBJECT"
# Email To ?
EMAIL="EMAIL"
# Email text/message
MYIP=$(curl http://ipecho.net/plain)
LASTIP=$(head -n 1 /home/ubuntu/myip.txt)
echo "Last IP:" $LASTIP
echo "Current IP:" $MYIP
#Check if IPs are the same
if [[ "$MYIP" == "$LASTIP" ]]; then
echo "IP hasn't changed. Do nothing."
else
sendemail -f $EMAIL -t $EMAIL -u $SUBJECT -m $MYIP -s smtp.gmail.com -o tls=yes -xu username -xp password
echo $MYIP > myip.txt
fi
When I try to run it in the command line, it works perfectly. The problem starts when I include it in "crontab -e" like this: "* * * * * /home/ubuntu/myip.sh".
Then it does not work. Seem to be that the sendmail is not functioning properly.
When I do a: tail -f /var/log/syslog
Sep 18 21:48:02 gpuserver sendmail[18665]: r8J1m1gO018665: from=ubuntu, size=314, class=0, nrcpts=1, msgid=<201309190148.r8J1m1gO018665#gpuserver>, relay=ubuntu#localhost
Sep 18 21:48:02 gpuserver sendmail[18665]: r8J1m1gO018665: to=ubuntu, ctladdr=ubuntu (1000/1000), delay=00:00:01, xdelay=00:00:00, mailer=relay, pri=30314, relay=[127.0.0.1] [127.0.0.1], dsn=4.0.0, stat=Deferred: Connection refused by [127.0.0.1]
Any ideas?
When you run a script from the command line you are utilizing your current environment. Cron doesn't have that info ... So what you should do from the command line is:
env > me
Then edit the script and include ./me at the start of the script. That way your cron task will run with the same environment.
When a cron prints something to stdout or stderr, cron itself tries to send a report by e-mail. Since you probably don't have system-wide e-mail configured, this is probably where the error message in syslog is coming from.
To prevent cron from sending any mail, make sure your script and any commands it spawns don't output anything to stdout or stderr. An easy way to do this is to add "&>/dev/null" to your cron line.
You can also add 'MAILTO=""' at the beginning of your crontab.
This all documented in the man pages for cron/crontab (man -k cron).

Resources