How to send email for failed jobs in shell scripts - linux

I have a shell script. To this script I am passing arguments from a file. This file contains tables names
The script is working fine. I am able execute the command for all the tables in the file.
shell script
#!/bin/bash
[ $# -ne 1 ] && { echo "Usage : $0 input file "; exit 1; }
input_file=$1
TIMESTAMP=`date "+%Y-%m-%d"`
touch /home/$USER/logs/${TIMESTAMP}.success_log
touch /home/$USER/logs/${TIMESTAMP}.fail_log
success_logs=/home/$USER/logs/${TIMESTAMP}.success_log
failed_logs=/home/$USER/logs/${TIMESTAMP}.fail_log
#Function to get the status of the job creation
function log_status
{
status=$1
message=$2
if [ "$status" -ne 0 ]; then
echo "`date +\"%Y-%m-%d %H:%M:%S\"` [ERROR] $message [Status] $status : failed" | tee -a "${failed_logs}"
#echo "Please find the attached log file for more details"
#exit 1
else
echo "`date +\"%Y-%m-%d %H:%M:%S\"` [INFO] $message [Status] $status : success" | tee -a "${success_logs}"
fi
}
while read table ;do
sqoop job --exec $table > /home/$USER/logging/"${table}_log" 2>&1
g_STATUS=$?
log_status $g_STATUS "Sqoop job ${table}"
cp /home/$USER/logging/"${table}_log" /home/$USER/debug/`date "+%Y-%m-%d"`/logs/
done < ${input_file}
Now I want to send emails to my email address for failed jobs.
Requirements
1) Send email for each failed job i.e If `status log` has failed job for one particular table then I want email sent out saying job for that table has failed.
2) Or Send out one email for all the jobs that have failed for one input file.
Which is the best method to approach. I would like the 2nd option atleast it will reduce the no of emails to go through.
But better if I know both methods to do
edited function log_status
#Function to get the status of the job creation
function log_status
{
status=$1
message=$2
if [ "$status" -ne 0 ]; then
echo "`date +\"%Y-%m-%d %H:%M:%S\"` [ERROR] $message [Status] $status : failed" | tee -a "${failed_logs}"
mail -a mail.txt -s "This is the failed job log" user#example.com < /home/$USER/logs/${TIMESTAMP}.fail_log
#exit 1
else
echo "`date +\"%Y-%m-%d %H:%M:%S\"` [INFO] $message [Status] $status : success" | tee -a "${success_logs}"
fi
}
If I do this will I get one email for all the failed jobs.

Also it's possible using sendmail command:
sendmail user#example.com < email.txt
Using mail command:
mail -a mail.txt -s "This is the failed job log" user#example.com
-a is a attachment, -s is a subject
And with many attachments:
$(uuencode file1.txt file2.txt) | mailx -s "Subject" user#example.com

Here is a simple snippet for sending an err.log:
mail -s "stder logs for ps" "name#example.com" < err.log
This appends a subject in the first quotes and mails it to the recipient in the second set. In this case I suppose you would have a code block export the error logs via 3> to a file which I called err.log for readability. I would place the above snippet either inside of your script if you are not worried about being too vocal about the errors or neatly outside of the script if you want to be more discrete about how often you are emailing the recipient.

Related

file check 3 times and exit shell script

I want to check for file in directory if there then push it to ssh server checing server connection if file not there then try 3 times with each 1min interval and in between if it comes ( on 2nd attend for example) then try again to connect ssh and push. else check for 3 attempts and exit
Please check my below code it is halting after 1st attempt ( during 2nd attempt I am making file available)
#!/bin/sh
echo "OK, start pushing the Userdetails to COUPA now..."
cd /usr/App/ss/outbound/usrdtl/
n=0
until [ $n -ge 3 ] || [ ! -f /usr/App/ss/outbound/usrdtl/USERS_APPROVERS_*.csv ]
do
if [ -f /usr/App/ss/outbound/usrdtl/USERS_APPROVERS_*.csv ] ;
then
pushFiles()
else
n=$[$n+1]
sleep 60
echo " trying " $n "times "
fi
done
pushFiles()
{
echo "File present Now try SSH connection"
while [ $? -eq 0 ];
do
echo $(date);
scpg3 -v /usr/App/ss/outbound/usrdtl/USERS_APPROVERS_*.csv <sshHost>:/Incoming/Users/
if [ $? -eq 0 ]; then
echo "Successfull"
echo $(date);
echo "Successfull" >> /usr/App/ss/UserApproverDetails.log
exit 1;
else
echo $(date);
echo "Failed" >> /usr/App/ss/UserApproverDetails.log
echo "trying again to push file.."
scpg3 -v /usr/App/sg/outbound/usrdtl/USERS_APPROVERS_*.csv <ssh Host>:/Incoming/Users/
echo $(date);
exit 1;
fi
done
}
I've tried to simplify this code for you. I hope it helps:
#!/bin/bash
outdir="/usr/App/ss/outbound/usrdtl"
logfile="/usr/App/ss/UserApproverDetails.log"
file_prefix="USERS_APPROVERS_"
function push_files() {
echo "File present now try SSH connection"
local attempts=1
local retries=2
date
while [[ ${attempts} -lt ${retries} ]]; do
if scp ${outdir}/${file_prefix}*.csv <sshHost>:/Incoming/Users/ ; then
echo "Successful" | tee -a ${logfile}
date
exit 0
else
echo "Failed" >> ${logfile}
fi
attempts=$((attempts+1))
do
echo "scp failed twice" | tee -a ${logfile}
exit 2
}
echo "OK, start pushing the Userdetails to COUPA now..."
cd ${outdir}
attempts=1
retries=3
while [[ ${attempts} -lt ${retries} ]]; do
echo "looking for files...attempt ${attempts}"
if test -n "$(shopt -s nullglob; echo ${outdir}/${file_prefix}*.csv)"; then
push_files()
fi
attempts=$((attempts+1))
sleep 60
done
echo "Files were never found" | tee -a ${logfile}
exit 1
Look at this code and tell me how it's not doing what you're trying to do. The most complicated part here is the nullglob stuff, which is a handy trick to see if any file in a glob matches
Also, I generally used bashisms.

after each command, if success: show a custom message, if error: show error, and send to file

A bash script executes several commands.
After each command, how to:
if success: show a custom message in the console and append it to a log file
if error: show the error message in the console, append error message to the log file and resume the entire script
Here is where I am:
log_file="log.txt"
output() {
echo $#
echo $# 2>&1 >> $log_file;
if [ $# -eq 0 ]
then
exit 1
fi
}
and then after each command:
if [ $? -eq 0 ]
then
output "Custom message"
else
output $?
fi
Which makes lots of repetitions…
You could create a "run" function to limit repetition:
run()
{
message="$1"
shift
eval "$*"
if [ $? -eq 0 ]; then
output "$message"
else
output $?
fi
}
And simply prefix every command with:
run "message" "command"
The command only needs to be quoted if it contains shell meta-expressions.
Here's a way to accomplish that; after each command to be tracked, add this line:
rc="$?"; if [ "$rc" -eq 0 ]; then echo "Custom message" 2>&1 | tee -a /folder/log; else echo "$rc" 2>&1 | tee -a /folder/log; fi
The following command will give you the exit status of your last run command.
$?
e.g lets say you executed "ls" command , after that if you execute the following
echo $?
you will see the exit status as 0. which means successful . An Non-zero exit status means failure.
You can use this in some sort of if else in your shell script and based on the exit value do whatever you need to do.

Server patching email notification linux script

I need to create an email notification linux script which can be used in the server patching activity. I have created one script for the same but getting some errors. Can anyone please check what's the issue.
Script -
#email constants
export to_addresses=xyz#abc.com
auth_1_app_path=/export/home/apps/test1/dev1;
function sendServerPatchingSuccessMail()
{
printf "HI Team,\r\n\r\nServer patching activity has been done successfully.\r\n\r\nRequest eText DEV team to validate the server patching changes.\r\n\r\nTotal time taken = $(( ($end_date-$start_date) / ( 60) )) minutes.\r\n\r\nRegards,\r\nAdmin\r\n\r\nNote: This is an auto generated mail, please do not reply. " | mail -s "server patching activity is done sucessfully." $to_addresses -c $cc_addresses
}
function sendServerPatchingStartMail()
{
printf "Hi Team,\r\n\r\nServer patching activity has started at $start_date.\r\n\r\nRequest eText DEV team to validate the server patching changes once they receive the server patching success mail.\r\n\r\nRegards,\r\nAdmin\r\n\r\nNote: This is an auto generated mail, please do not reply. " | mail -s "server patching activity has started." $to_addresses -c $cc_addresses
}
#define list of hosts as array
servers_array=( bookvm04 #DEV1
);
function doWork() {
start_date=$(date +%Y%m%d%H%M%S);
sendServerPatchingStartMail
#stop tomcat on all hosts
echo "`date "+%Y-%m-%d %H:%M:%S"` Stop tomcat stop."
#start for each host
for i in "${servers_array[#]}"
do
export ssh_to_remote_host="ssh -l bookweb -i /export/home/apps/bookplus/.ssh/id_dsa "
case $i in
#DEV1 Book Server
b3bookvm05)
app_path=$view_1_app_path;
;;
#DEV1 Book Server
b3bookvm04)
app_path=$auth_1_app_path;
;;
*) echo "`date "+%Y-%m-%d %H:%M:%S"` $i is not a valid option exiting"
exit 0;;
esac #end case
$ssh_to_remote_host $i $app_path/scripts/stopremote
if [ "$?" -ne 0 ]
then
echo "`date "+%Y-%m-%d %H:%M:%S"` node ${i} could not be stopped"
exit 0
fi
end_date=$(date +%Y%m%d%H%M%S);
echo "`date "+%Y-%m-%d %H:%M:%S"` Total time taken = $(( ($end_date- $start_date) / ( 60) )) minutes";
sendServerPatchingSuccessMail
}
Error while executing above script
-bash-4.1$ sh server_patching_alert.sh
: command not foundrt.sh: line 4:
: command not foundrt.sh: line 5:
: command not foundrt.sh: line 9:
: command not foundrt.sh: line 13:
'erver_patching_alert.sh: line 14: syntax error near unexpected token `
'erver_patching_alert.sh: line 14: `function sendServerPatchingSuccessMail()
This script is working fine.
server_array=$(hostname)
function sendServerPatchingSuccessMail()
{
# Set email constants
to_addresses=
cc_addresses=
# Send server patching completion mail
printf "mail body - server patching activity for hostname '$server_array' has been completed successfully" | mail -s "mail subject" -c $cc_addresses $to_addresses
}
sendServerPatchingSuccessMail

bash script | URL health check | mail sending option

We need to health check a URL every minute of the clock.
Algorithm of bash script is following:
if (url is not up)
send mail
wait for 5 minutes
if (still not up)
send mail again and keep on doing it
else
nothing
My logic is following:
#!/bin/bash
IP="192.168.XX.XXX"
URL="http://$IP:YYYY"
CCMAIL="abc#def.com"
TOMAIL="abc#def.com"
STATUSCODE=$(curl -o /dev/null --silent --head --write-out '%{http_code}\n' $URL)
NOWENTRY=$(date +"%d-%m-%Y %r")
NOWLOG=$(date +"%d-%m-%Y")
if [ $STATUSCODE -ne "200" ]
then
echo "$IP URL: $URL was down at $NOWENTRY" >> "path/log_$NOWLOG.txt"
#send mail in one line no timing logic out here
fi
Note: I have put this script in crontab to execute every minute of the clock.
if [ "$STATUSCODE" -ne "200" ]
then
msg="$IP URL: $URL was down at $NOWENTRY"
echo "$msg" >> "path/log_$NOWLOG.txt"
# send email
echo "$msg" | mailx -c "$CCEMAIL" -s "$URL down" "$TOEMAIL"
sleep 5m
# re-spawn this script.
exec "$0"
fi

Script that checks a URL for specific text

Looking for a bash script that will accomplish the followings:
Check a URL (ex. www.google.com)
Looks for a specific text string
if it exists, it does nothing
if it doesnt, then sends out an email to alert someone
I tried the following script, it doesnt do anything, I dont get any email or anything.
#!/bin/sh
URL="URL"
TMPFILE=`mktemp /string_watch.XXXXXX`
curl -s -o ${TMPFILE} ${URL} 2>/dev/null
if [ "$?" -ne "0" ];
then
echo "Unable to connect to ${URL}"
exit 2
fi
RES=`grep -i "StringToLookFor" ${TMPFILE}`
if [ "$?" -ne "0" ];
then
echo "String not found in ${URL}" | mail -s "Alert" your#email
exit 1
fi
echo "String found"
exit 0;
The command
mail -s "Alert" your#email
is pausing to let you enter the text of your email. If you want to send an email with the indicated subject and no text you need to do
mail -s "Alert" your#email < /dev/null

Resources