I have a script on a linux box in my environment to loop through a series of services that should always be running, and if one isn't running, send me an email.
Now, it seems to work fine except for two issues. I'd really appreciate some help and insight. Most of my background comes from Python and Powershell.
Whenever the script detects a service that's down, it exits the script, instead of looping through the rest. It then appends the services it didn't check to the body of the email, despite me not specifying an email body in the mail command.
Every so often, it throws a false error on "hostservices"; and I have no idea how to even go about figuring out why.
The script is a cron job running every 10 minutes. Full text of the script and list of services are below, as well as a screenshot of what happens when the script finds a service that's down.
Script
#!/bin/bash
while read services; do
#Run Command to get Service Status, store results as string variable
service_status=$(service $services status)
#Check if the service is NOT running.
if [[ "$service_status" != *"is running..." ]];
then
mail -s "Service $services is down on [SERVER]" [EMAIL ADDRESS]
elif [[ $service_status == *"is running..." ]];
then
:
else
mail -s "ERROR IN SCRIPT, unable to get $services status on [SERVER]" [EMAIL ADDRESS]
fi
done < /home/services.txt
services.txt
hostcontect
hostservices
ecs-ec
ecs-ep
imq
tomcat
httpd
Email Alert for Down Service
SUBJECT: "Service hostservices is down on [SERVER]"
BODY:
ecs-ec
ecs-ep
imq
tomcat
httpd
mail reads the body of the email from standard input. In your case, the input file is redirected to stdin, so it's read instead. Tell mail to read the body from elsewhere, e.g.
mail -s ... < /dev/null
Related
I have Jenkins pipeline job which goes thought all our Jenkins servers and check the connectivity (runs every few minutes).
ksh file:
#!/bin/ksh
JENKINS_URL=$1
curl --connect-timeout 10 "$JENKINS_URL" >/dev/null
status=`echo $?`
if [ "$status" == "7" ]; then
export SUBJECT="Connection refused or can not connect to URL $JENKINS_URL"
echo "$SUBJECT"|/usr/sbin/sendmail -t XXXX#gmail.com
else
echo "successfully connected $JENKINS_URL"
fi
exit 0
I would like to add another piece of code, which record all the times that server was down (it should include the name of the server and timestamp) into a file, and in case the server is up again, send an email which will notify about it, and it will be also recorded in the file.
I don't want to get extra alerts, only one alert (to file and mail) when it's down, and one when it's up again. any idea how to implement it?
The detailed answer was given by unix.stackexchange community:
https://unix.stackexchange.com/questions/562594/how-to-set-and-record-alerts-for-jenkin-server-down-and-up
Im trying to send a notification upon login via PAM, but i cant figure out how to send it to the user that is logging in.
I'm configuring my PAM to execute a script every time a user logs in. The problem is i need to send a notification if there have been any login attempts (its part of a bigger security thing im trying to add, where my laptop takes a picture with the webcam upon failed logins, and notifies me when i log in again, since my classmates like to try and guess my password for some reason).
The problem is that the line in my .sh file, which sends a user notification, sends it to root since thats the 'user' that executes the script, i want my script to send the notification to my current user (called "andreas"), but im having problems figuring this out.
Here is the line i added to the end of the PAM file system-login:
auth [default=ignore] pam_exec.so /etc/lockCam/call.sh
And here is the call.sh file:
#!/bin/sh
/etc/lockCam/notifier.sh &
The reason im calling another file is because i want it to run in the background WHILE the login process continues, that way the process doesnt slow down logging in.
Here is the script that is then executed:
#!/bin/sh
#sleep 10s
echo -e "foo" > "/etc/lockCam/test"
#This line is simply to make sure that i know that my script was executed
newLogins=`sed -n '3 p' /etc/lockCam/lockdata`
if [ $newLogins -gt 0 ]
then
su andreas -c ' notify-send --urgency=critical --expire-time=6000 "Someone tried to log in!" "$newLogins new lockCam images!" && exit'
callsInRow=`sed -n '2 p' /etc/lockCam/lockdata`
crntS=$(date "+%S")
crntS=${crntS#0}
crntM=$(date "+%M")
crntM=${crntM#0}
crntH=$(date "+%H")
crntH=${crntH#0}
((crntTime = $crntH \* 60 \* 60 + $crntM \* 60 + $crntS ))
#This whole process is absolutely stupid but i cant figure out a better way to make sure none of the integers are called "01" or something like that, which would trigger an error
echo -e "$crntTime\n$callsInRow\n0" > "/etc/lockCam/lockdata"
fi
exit 0
And this is where i THINK my error is, the line "su andreas -c...." is most likely formatted wrong or im doing something else wrong, everythin is executed upon login EXCEPT the notification doesnt show up. If i execute the script from a terminal when im already logged in there is no notification either, unless i remove the "su andreas -c" part and simply do "notify-send...", but that doesnt send out a notification when i log in, and i think thats because the notification is sent to the root user, and not "andreas".
I think your su needs to be passed the desktop users DBUS session bus address. The bus address can be easily obtained and used for X11 user sessions, but Wayland has tighter security, for Wayland the user session actually has to run up proxy to receive the messages. (Had you considered it might be easier to send an email?)
I have notify-desktop gist on github that works for X11 and should also work on Wayland (provided the proxy is running). For completeness I've appended the source code of the script to this post, it's extensively commented, I think it contains the pieces necessary to get you own code working.
#!/bin/bash
# Provides a way for a root process to perform a notify send for each
# of the local desktop users on this machine.
#
# Intended for use by cron and timer jobs. Arguments are passed straight
# to notify send. Falls back to using wall. Care must be taken to
# avoid using this script in any potential fast loops.
#
# X11 users should already have a dbus address socket at /run/user/<userid>/bus
# and this script should work without requiring any initialisation. Should
# this not be the case, X11 users could initilise a proxy as per the wayland
# instructions below.
#
# Due to stricter security requirments Wayland lacks an dbus socket
# accessable to root. Wayland users will need to run a proxy to
# provide root with the necessary socket. Each user can must add
# the following to a Wayland session startup script:
#
# notify-desktop --create-dbus-proxy
#
# That will start xdg-dbus-proxy process and make a socket available under:
# /run/user/<userid>/proxy_dbus_<desktop_sessionid>
#
# Once there is a listening socket, any root script or job can pass
# messages using the syntax of notify-send (man notify-send).
#
# Example messages
# notify-desktop -a Daily-backup -t 0 -i dialog-information.png "Backup completed without error"
# notify-desktop -a Remote-rsync -t 6000 -i dialog-warning.png "Remote host not currently on the network"
# notify-desktop -a Daily-backup -t 0 -i dialog-error.png "Error running backup, please consult journalctl"
# notify-desktop -a OS-Upgrade -t 0 -i dialog-warning.png "Update in progress, do not shutdown until further completion notice."
#
# Warnings:
# 1) There has only been limited testing on wayland
# 2) There has only been no testing for multiple GUI sessions on one desktop
#
if [ $1 == "--create-dbus-proxy" ]
then
if [ -n "$DBUS_SESSION_BUS_ADDRESS" ]
then
sessionid=$(cat /proc/self/sessionid)
xdg-dbus-proxy $DBUS_SESSION_BUS_ADDRESS /run/user/$(id -u)/proxy_dbus_$sessionid &
exit 0
else
echo "ERROR: no value for DBUS_SESSION_BUS_ADDRESS environment variable - not a wayland/X11 session?"
exit 1
fi
fi
function find_desktop_session {
for sessionid in $(loginctl list-sessions --no-legend | awk '{ print $1 }')
do
loginctl show-session -p Id -p Name -p User -p State -p Type -p Remote -p Display $sessionid |
awk -F= '
/[A-Za-z]+/ { val[$1] = $2; }
END {
if (val["Remote"] == "no" &&
val["State"] == "active" &&
(val["Type"] == "x11" || val["Type"] == "wayland")) {
print val["Name"], val["User"], val["Id"];
}
}'
done
}
count=0
while read -r -a desktop_info
do
if [ ${#desktop_info[#]} -eq 3 ]
then
desktop_user=${desktop_info[0]}
desktop_id=${desktop_info[1]}
desktop_sessionid=${desktop_info[2]}
proxy_bus_socket="/run/user/$desktop_id/proxy_dbus_$desktop_sessionid"
if [ -S $proxy_bus_socket ]
then
bus_address="$proxy_bus_socket"
else
bus_address="/run/user/$desktop_id/bus"
fi
sudo -u $desktop_user DBUS_SESSION_BUS_ADDRESS="unix:path=$bus_address" notify-send "$#"
count=$[count + 1]
fi
done <<<$(find_desktop_session)
# If no one has been notified fall back to wall
if [ $count -eq 0 ]
then
echo "$#" | wall
fi
# Don't want this to cause a job to stop
exit 0
So, I have a script which it's intended purpose is to:
Ask for the DC number and Time clock number
log in to the Time clock server for the DC stated above
After log in, it is intended to run a seperate script inside my script which updates the time clock number also stated above.
My issue is that once I trigger the script, it logs into the server as intended, prompts me for my user ID, and then I have to press "enter" when "xterm" comes up after that. After this, the update script is supposed to run, however, it doesn't, and sits at the command line.
After I exit the server, THEN it runs the update script, but fails, because the update script doesn't exist in the jump box.
My question is, after the script logs in to the server, how can I get it to trigger the script inside the Time clock server, as I am wanting it to? Thanks.
Script is below:
#!/bin/bash -x
export LANG="C"
####
####
## This script is intended to speed up the process to setup timeclocks from DC tickets
## Created by Blake Smreker | b0s00dg | bsmreker#walmart.com
####
####
#Asks for DC number
echo "What is the four digit DC number?"
read DC #User input
#Asks for Timeclock number
echo "What is the two digit Timeclock number?"
read TMC #User input
#Defines naming convention of tna server
tnaserver="cs-tna.s0${DC}.us.wal-mart.com"
#creating variable to define the update script
tcupd="/u/applic/tna/shell/tc_software_update.sh tmc${TMC}.s0${DC}.us REFURBISHED"
#Logging in to the cs-tna package at the specified DC
/usr/bin/dzdo -u osedc /bin/ssh -qo PreferredAuthentications=publickey root#$tnaserver
echo "Preforming Timeclock update on Timeclock=$TMC, at DC=${DC}"
echo ""
echo "-----------------------------------------------------------------------------------------------------------------------------------------"
$tcupd #Runs update script
echo "-----------------------------------------------------------------------------------------------------------------------------------------"
echo ""
sleep 2
echo "If prompted to engage NOC due to Timeclock not being on the network, send the ticket to DC Networking"
echo ""
echo "OR"
echo ""
echo "If the script completed successfully, and the Timeclock was updated, you can now resolve the ticket"
You must run the command inside ssh session, not after it:
echo "Preforming Timeclock update on Timeclock=$TMC, at DC=${DC}"
echo ""
echo "-----------------------------------------------------------------------------------------------------------------------------------------"
###### $tcupd #Runs update script
/usr/bin/dzdo -u osedc /bin/ssh -qo PreferredAuthentications=publickey root#$tnaserver /bin/bash -c /u/applic/tna/shell/tc_software_update.sh tmc${TMC}.s0${DC}.us REFURBISHED
echo "-----------------------------------------------------------------------------------------------------------------------------------------"
echo ""
sleep 2
echo "If prompted to engage NOC due to Timeclock not being on the network, send the ticket to DC Networking"
echo ""
echo "OR"
echo ""
echo "If the script completed successfully, and the Timeclock was updated, you can now resolve the ticket"
From man ssh you see ssh [-46AaCfGgKkMNnqsTtVvXxYy] ....... destination [command]. If [command] is not given ssh runs remote login command scripts, for example xterm. You read more here or here or just browse google.
You need to think how and which environment variable you want to pass to the remote machine and remember about properly enclosing the variables, so they get expanded on your or the remote machine.
I am doing an ftp and I want to check the status. I don't want to use '$?' as mostly it returns 0 (Success) for ftp even though internally ftp didn't go through.
I know I can check the log file and do a grep from there for "Transfer complete" (221 status). That works fine but I don't want to do it as I have many different reports doing ftp. So creating multiple log files for all of them is what I want to avoid.
Can I get the logged information in a local script variable and process it inside the script itself?
Something similar to these (I've tried both but neither worked):
Grab FTP output in BASH SCRIPT
FTP status check whether successful or not
Below is something similar to what I am trying to do:
ftp -inv ${HOST} > log_file.log <<!
user ${USER} ${PASS}
bin
cd "${TARGET}"
put ${FEEDFILE}
bye
!
Any suggestions on how can I get the entire ftp output in a script variable and then check it within the script?
To capture stdout to a variable you can use bash's command substitution, so either OUTPUT=`cmd` or OUTPUT=$(cmd).
Here's an example how to capture the output from ftp in your case:
CMDS="user ${USER} ${PASS}
bin
cd \"${TARGET}\"
put \"${FEEDFILE}\"
bye"
OUTPUT=$(echo "${CMDS}" | ftp -inv ${HOST})
I need a small alarm (HTTP req or any) when the server is going down. I checked many applications like nagios, servercheck and so on... All these application monitor only remote servers. I have only two servers to monitor. So if my server (10.172.65.124) is going down cant it send a alarm. I dont want to maintain one more server to monitor this. I am using rhel6 & centos7. Any suggestions
Here's a python script that will serve the purpose. It uses sendmail to send your email which will require running it from a linux server that has sendmail enabled. Change the url to point to the url you are monitoring. If you run this script, it'll check stackoverflow.
This uses urllib to check the status code it receives when trying to load your url. If it gets a status other than 200 from the HTTP request it expects the site to be down.
To monitor your server you should run the script on a server or desktop that is independent from your webhost, otherwise you won't be alerted when your server crashes due to a number of reasons.
#Import time to allow you to sleep the script, urllib to load the site, subprocess will allow you to run a process on the machine outside of the script (in this instance it's send mail)
import time
import urllib
from email.mime.text import MIMEText
from subprocess import Popen, PIPE
#The url being monitored.
url = "http://www.stackoverflow.com"
#The contents of the email
msg = MIMEText(url + " is not responding. Please investigate.")
msg["From"] = "me#youremail.com"
msg["To"] = "me#youremail.com"
msg["Subject"] = url + "is not responding"
#This loops while the script is running.
# It gets the status returned from the urllib call, if it's not 200 it will email the email contents above.
while True:
status = urllib.urlopen(url).getcode()
if status <> 200:
#This is what sends the email. If you don't have sendmail then update this.
p = Popen(["/usr/sbin/sendmail", "-t", "-oi"], stdin=PIPE)
p.communicate(msg.as_string())
#The number of seconds the loop will pause for before checking again. I set it to 60.
time.sleep(60)
I would recommend create simple script to ping the machine (they can monitor each other)and if ping timesout send an email.
something like this
#!/bin/bash
SERVERIP=IP ADDRESS
NOTIFYEMAIL=test#example.com
ping -c 3 $SERVERIP > /dev/null 2>&1
if [ $? -ne 0 ]
then
# Use your favorite mailer here:
mailx -s "Server $SERVERIP is down" -t "$NOTIFYEMAIL" < /dev/null
fi
As given above script you can configure normal bash script to monitor server http request or any other service request so if it would not get reply then you will get mail.
There is normal application for monitoring web service which is free on limited no of site per user which you can also use this.
http://uptimerobot.com/
Below script will check for the operation state of interface, If you need add some interfaces do alert as per your wish
#!/bin/bash
while true
do
if [ $(cat /sys/class/net/eth0/operstate) != "up" ]; then
sleep 1
#send mail for logging
fi
done