I have Jenkins pipeline job which goes thought all our Jenkins servers and check the connectivity (runs every few minutes).
ksh file:
#!/bin/ksh
JENKINS_URL=$1
curl --connect-timeout 10 "$JENKINS_URL" >/dev/null
status=`echo $?`
if [ "$status" == "7" ]; then
export SUBJECT="Connection refused or can not connect to URL $JENKINS_URL"
echo "$SUBJECT"|/usr/sbin/sendmail -t XXXX#gmail.com
else
echo "successfully connected $JENKINS_URL"
fi
exit 0
I would like to add another piece of code, which record all the times that server was down (it should include the name of the server and timestamp) into a file, and in case the server is up again, send an email which will notify about it, and it will be also recorded in the file.
I don't want to get extra alerts, only one alert (to file and mail) when it's down, and one when it's up again. any idea how to implement it?
The detailed answer was given by unix.stackexchange community:
https://unix.stackexchange.com/questions/562594/how-to-set-and-record-alerts-for-jenkin-server-down-and-up
Related
We are using linux Servers (CentOS) and one backup server
We only login to prod server if any there is any issue, else we check connection from backup server
We do no have any kind of monitoring tool
I have created a simple bash script on backup server as below
#!/bin/bash
date
cat /tmp/servers.txt | while read output
do
ping -c 1 "$output" > /dev/null
if [ $? -eq 0 ] ; then
echo "Server $output is UP"
else
echo "Server $output is Down"
fi
done
How do we get output of this script after log into this server automatically.
~/.bash_profile runs every time you log in, so that seems like the place you would want to put this code.
Im trying to send a notification upon login via PAM, but i cant figure out how to send it to the user that is logging in.
I'm configuring my PAM to execute a script every time a user logs in. The problem is i need to send a notification if there have been any login attempts (its part of a bigger security thing im trying to add, where my laptop takes a picture with the webcam upon failed logins, and notifies me when i log in again, since my classmates like to try and guess my password for some reason).
The problem is that the line in my .sh file, which sends a user notification, sends it to root since thats the 'user' that executes the script, i want my script to send the notification to my current user (called "andreas"), but im having problems figuring this out.
Here is the line i added to the end of the PAM file system-login:
auth [default=ignore] pam_exec.so /etc/lockCam/call.sh
And here is the call.sh file:
#!/bin/sh
/etc/lockCam/notifier.sh &
The reason im calling another file is because i want it to run in the background WHILE the login process continues, that way the process doesnt slow down logging in.
Here is the script that is then executed:
#!/bin/sh
#sleep 10s
echo -e "foo" > "/etc/lockCam/test"
#This line is simply to make sure that i know that my script was executed
newLogins=`sed -n '3 p' /etc/lockCam/lockdata`
if [ $newLogins -gt 0 ]
then
su andreas -c ' notify-send --urgency=critical --expire-time=6000 "Someone tried to log in!" "$newLogins new lockCam images!" && exit'
callsInRow=`sed -n '2 p' /etc/lockCam/lockdata`
crntS=$(date "+%S")
crntS=${crntS#0}
crntM=$(date "+%M")
crntM=${crntM#0}
crntH=$(date "+%H")
crntH=${crntH#0}
((crntTime = $crntH \* 60 \* 60 + $crntM \* 60 + $crntS ))
#This whole process is absolutely stupid but i cant figure out a better way to make sure none of the integers are called "01" or something like that, which would trigger an error
echo -e "$crntTime\n$callsInRow\n0" > "/etc/lockCam/lockdata"
fi
exit 0
And this is where i THINK my error is, the line "su andreas -c...." is most likely formatted wrong or im doing something else wrong, everythin is executed upon login EXCEPT the notification doesnt show up. If i execute the script from a terminal when im already logged in there is no notification either, unless i remove the "su andreas -c" part and simply do "notify-send...", but that doesnt send out a notification when i log in, and i think thats because the notification is sent to the root user, and not "andreas".
I think your su needs to be passed the desktop users DBUS session bus address. The bus address can be easily obtained and used for X11 user sessions, but Wayland has tighter security, for Wayland the user session actually has to run up proxy to receive the messages. (Had you considered it might be easier to send an email?)
I have notify-desktop gist on github that works for X11 and should also work on Wayland (provided the proxy is running). For completeness I've appended the source code of the script to this post, it's extensively commented, I think it contains the pieces necessary to get you own code working.
#!/bin/bash
# Provides a way for a root process to perform a notify send for each
# of the local desktop users on this machine.
#
# Intended for use by cron and timer jobs. Arguments are passed straight
# to notify send. Falls back to using wall. Care must be taken to
# avoid using this script in any potential fast loops.
#
# X11 users should already have a dbus address socket at /run/user/<userid>/bus
# and this script should work without requiring any initialisation. Should
# this not be the case, X11 users could initilise a proxy as per the wayland
# instructions below.
#
# Due to stricter security requirments Wayland lacks an dbus socket
# accessable to root. Wayland users will need to run a proxy to
# provide root with the necessary socket. Each user can must add
# the following to a Wayland session startup script:
#
# notify-desktop --create-dbus-proxy
#
# That will start xdg-dbus-proxy process and make a socket available under:
# /run/user/<userid>/proxy_dbus_<desktop_sessionid>
#
# Once there is a listening socket, any root script or job can pass
# messages using the syntax of notify-send (man notify-send).
#
# Example messages
# notify-desktop -a Daily-backup -t 0 -i dialog-information.png "Backup completed without error"
# notify-desktop -a Remote-rsync -t 6000 -i dialog-warning.png "Remote host not currently on the network"
# notify-desktop -a Daily-backup -t 0 -i dialog-error.png "Error running backup, please consult journalctl"
# notify-desktop -a OS-Upgrade -t 0 -i dialog-warning.png "Update in progress, do not shutdown until further completion notice."
#
# Warnings:
# 1) There has only been limited testing on wayland
# 2) There has only been no testing for multiple GUI sessions on one desktop
#
if [ $1 == "--create-dbus-proxy" ]
then
if [ -n "$DBUS_SESSION_BUS_ADDRESS" ]
then
sessionid=$(cat /proc/self/sessionid)
xdg-dbus-proxy $DBUS_SESSION_BUS_ADDRESS /run/user/$(id -u)/proxy_dbus_$sessionid &
exit 0
else
echo "ERROR: no value for DBUS_SESSION_BUS_ADDRESS environment variable - not a wayland/X11 session?"
exit 1
fi
fi
function find_desktop_session {
for sessionid in $(loginctl list-sessions --no-legend | awk '{ print $1 }')
do
loginctl show-session -p Id -p Name -p User -p State -p Type -p Remote -p Display $sessionid |
awk -F= '
/[A-Za-z]+/ { val[$1] = $2; }
END {
if (val["Remote"] == "no" &&
val["State"] == "active" &&
(val["Type"] == "x11" || val["Type"] == "wayland")) {
print val["Name"], val["User"], val["Id"];
}
}'
done
}
count=0
while read -r -a desktop_info
do
if [ ${#desktop_info[#]} -eq 3 ]
then
desktop_user=${desktop_info[0]}
desktop_id=${desktop_info[1]}
desktop_sessionid=${desktop_info[2]}
proxy_bus_socket="/run/user/$desktop_id/proxy_dbus_$desktop_sessionid"
if [ -S $proxy_bus_socket ]
then
bus_address="$proxy_bus_socket"
else
bus_address="/run/user/$desktop_id/bus"
fi
sudo -u $desktop_user DBUS_SESSION_BUS_ADDRESS="unix:path=$bus_address" notify-send "$#"
count=$[count + 1]
fi
done <<<$(find_desktop_session)
# If no one has been notified fall back to wall
if [ $count -eq 0 ]
then
echo "$#" | wall
fi
# Don't want this to cause a job to stop
exit 0
I am creating a shell script which checks the status of a service running on another machine and if didn't get any response than performs some operation at the local system. I am using if clause in the script for this task.
Sometimes due to the network connection, it falsely assumes that the remote server is not responding and performs the tasks mentioned inside if clause. I want to set up a retry so that it checks if condition more than once when it didn't find any response in the first attempt.
is there any way to setup retry like thing in a shell script for this purpose?
Below is a sample code
RSI1_STATUS=$(psql -U username -h serverip -d postgres -t -c "select version();" )
if [ -z "$RSI1_STATUS" ] #Condition will be true if remote server is not active
then
touch /tmp/postgresql.trigger
fi
now I want to check if condition more than once if it is true in the first attempt.
You could add retry loop with a number of retries using a while loop:
retries=5
while ! check_network_connection && ((--retries)); do
sleep 1 # or probe the network, etc.
done
if [[ $retries -eq 0 ]]; then
echo "Error: Connection retries exhausted."
else
# connection succeeded.
fi
Whether you want to sleep or do something else depends on your usage and the application.
Note: The "network connection" might have succeeded after checking in the loop. So if retries is 0, it doesn't necessarily mean that the connection is still down.
So, I have a script which it's intended purpose is to:
Ask for the DC number and Time clock number
log in to the Time clock server for the DC stated above
After log in, it is intended to run a seperate script inside my script which updates the time clock number also stated above.
My issue is that once I trigger the script, it logs into the server as intended, prompts me for my user ID, and then I have to press "enter" when "xterm" comes up after that. After this, the update script is supposed to run, however, it doesn't, and sits at the command line.
After I exit the server, THEN it runs the update script, but fails, because the update script doesn't exist in the jump box.
My question is, after the script logs in to the server, how can I get it to trigger the script inside the Time clock server, as I am wanting it to? Thanks.
Script is below:
#!/bin/bash -x
export LANG="C"
####
####
## This script is intended to speed up the process to setup timeclocks from DC tickets
## Created by Blake Smreker | b0s00dg | bsmreker#walmart.com
####
####
#Asks for DC number
echo "What is the four digit DC number?"
read DC #User input
#Asks for Timeclock number
echo "What is the two digit Timeclock number?"
read TMC #User input
#Defines naming convention of tna server
tnaserver="cs-tna.s0${DC}.us.wal-mart.com"
#creating variable to define the update script
tcupd="/u/applic/tna/shell/tc_software_update.sh tmc${TMC}.s0${DC}.us REFURBISHED"
#Logging in to the cs-tna package at the specified DC
/usr/bin/dzdo -u osedc /bin/ssh -qo PreferredAuthentications=publickey root#$tnaserver
echo "Preforming Timeclock update on Timeclock=$TMC, at DC=${DC}"
echo ""
echo "-----------------------------------------------------------------------------------------------------------------------------------------"
$tcupd #Runs update script
echo "-----------------------------------------------------------------------------------------------------------------------------------------"
echo ""
sleep 2
echo "If prompted to engage NOC due to Timeclock not being on the network, send the ticket to DC Networking"
echo ""
echo "OR"
echo ""
echo "If the script completed successfully, and the Timeclock was updated, you can now resolve the ticket"
You must run the command inside ssh session, not after it:
echo "Preforming Timeclock update on Timeclock=$TMC, at DC=${DC}"
echo ""
echo "-----------------------------------------------------------------------------------------------------------------------------------------"
###### $tcupd #Runs update script
/usr/bin/dzdo -u osedc /bin/ssh -qo PreferredAuthentications=publickey root#$tnaserver /bin/bash -c /u/applic/tna/shell/tc_software_update.sh tmc${TMC}.s0${DC}.us REFURBISHED
echo "-----------------------------------------------------------------------------------------------------------------------------------------"
echo ""
sleep 2
echo "If prompted to engage NOC due to Timeclock not being on the network, send the ticket to DC Networking"
echo ""
echo "OR"
echo ""
echo "If the script completed successfully, and the Timeclock was updated, you can now resolve the ticket"
From man ssh you see ssh [-46AaCfGgKkMNnqsTtVvXxYy] ....... destination [command]. If [command] is not given ssh runs remote login command scripts, for example xterm. You read more here or here or just browse google.
You need to think how and which environment variable you want to pass to the remote machine and remember about properly enclosing the variables, so they get expanded on your or the remote machine.
I have a script on a linux box in my environment to loop through a series of services that should always be running, and if one isn't running, send me an email.
Now, it seems to work fine except for two issues. I'd really appreciate some help and insight. Most of my background comes from Python and Powershell.
Whenever the script detects a service that's down, it exits the script, instead of looping through the rest. It then appends the services it didn't check to the body of the email, despite me not specifying an email body in the mail command.
Every so often, it throws a false error on "hostservices"; and I have no idea how to even go about figuring out why.
The script is a cron job running every 10 minutes. Full text of the script and list of services are below, as well as a screenshot of what happens when the script finds a service that's down.
Script
#!/bin/bash
while read services; do
#Run Command to get Service Status, store results as string variable
service_status=$(service $services status)
#Check if the service is NOT running.
if [[ "$service_status" != *"is running..." ]];
then
mail -s "Service $services is down on [SERVER]" [EMAIL ADDRESS]
elif [[ $service_status == *"is running..." ]];
then
:
else
mail -s "ERROR IN SCRIPT, unable to get $services status on [SERVER]" [EMAIL ADDRESS]
fi
done < /home/services.txt
services.txt
hostcontect
hostservices
ecs-ec
ecs-ep
imq
tomcat
httpd
Email Alert for Down Service
SUBJECT: "Service hostservices is down on [SERVER]"
BODY:
ecs-ec
ecs-ep
imq
tomcat
httpd
mail reads the body of the email from standard input. In your case, the input file is redirected to stdin, so it's read instead. Tell mail to read the body from elsewhere, e.g.
mail -s ... < /dev/null