Bash script and cron anomaly - linux

I have a bash script that I am run to check to see if one of my programs has hung, and if it has kill it. The script works fine if ran from the command line, but if I schedule it with cron it does something very strange.
Basically the script (below) gets the PID of my program and gets its created date/time from its entry in the /proc/ directory. It then gets the current date/time from the system and converts these two values into seconds since 1970 with the "date" command, before finally subtracting the two. This usually ends up with a total of 2100 seconds or something like that, which equates to 35 minutes.
#!/bin/bash
THEDATE=$(date +%s)
MYPID=$(ps aux|grep -v grep|egrep "MyProgram.exe"|awk '{print $2}')
if (( ${#MYPID} > 0 )); then
STARTTIME=$(ls -ld /proc/$MYPID|date +%s -d"$(awk '{print $6, $7}')")
TOTALMINS=$(( ($THEDATE - $STARTTIME) / 60 ))
if (( $TOTALMINS >= 30 )); then
kill -9 $MYPID
logger -t "[KillLongRunningProcesses] Killed my program which had been running for $TOTALMINS minutes"
fi
fi
When ran from the command line, the two date variables (THEDATE and STARTTIME) both get the correct values. But when run by cron the STARTTIME is wrong. It has the correct date, but seems to ignore the time part and sets it to midnight, ie "2009-12-14 00:00:00" is obtained instead of "2009-12-14 13:23:00", which throws off all the calculations.
Any ideas? Thanks.

First off never parse the output of ls, read THIS to understand why. Next, your script can be greatly improved by using pgrep rather than using awk to parse the PID from a grep on 'ps aux'. Also, your script breaks horribly in the case where you have more than one PID returned. And finally, when writing shell scripts try not to use CAPITALS for variable names; that convention is reserved for variables that you export into your environment.
The following script attempts to solve the problems mentioned above. It is as efficient as I could make it and it will handle the case where you have multiple PIDs. It also checks to make sure that the PID still exists before we kill it because it's possible that when we kill the parent it may take out the child processes.
#!/bin/bash
prog_name="MyProgram.exe"
the_date=$(date +%s)
my_pids=( $(pgrep "$prog_name") )
for ((i=0; i < ${#my_pids[#]}; i++)); do
if [[ -d /proc/${my_pids[i]} ]]; then
start_time=$(stat --printf=%Y /proc/${my_pids[i]})
total_mins=$(( (the_date - start_time) / 60 ))
if (( $total_mins >= 30 )); then
kill -9 ${my_pids[i]}
logger -t "Your custom message here"
fi
fi
done

That's why you can't rely on parsing ls. You should use stat if your system has it.
stat --printf=%Y /proc/$MYPID
If not, perhaps your find can do it for you:
find /proc -maxdepth 1 -name $MYPID -printf "%T#"

For what it's worth I used to get caught out with the PATH and other environment variables not being set automatically.
Worth checking. If it's not that then comment and I'll delete my answer.

I've figured out how to do it. If I use the "--time-style=long-iso" argument with ls it returns it in the format I need. Apparently cron has different preferences for some command than the default user.
I take your point Dennis that "ls" is unreliable for parsing, but the configuration for the computer I'm using is never going to change, so now that it works I will leave it as is. In future I will likely do things your way.

Related

Rollover shell script

Assuming a shell script(commands.sh) with few commands.
I need to write a script which sends the output of commands executed by commands.sh to a file f1.csv
if file size exceeds 1MB then the output flowing should go to file f2.csv
if the file size exceeds 1 mb again here,the output flowing should go to file f3.csv
if f3.csv exceeds the size 1mb,then the older f1 should be deleted and again new file f1 should be created,
output flowing should be to written to f1. This process should go on .
I can write the crontab file, just the shell script is a bit tricky
I have been experimenting:
#!/usr/bin/env bash
PREFIX="f"
# Maximum size after which you want a new file in bytes
MAX_SIZE=1048576
LAST_FILE=`ls "$prefix"*.csv | tail -1`
# Check if file exists and if it does not, create it.
if [[ -z "$LAST_FILE" ]]
then
LAST_FILE=$PREFIX"1.csv"
touch $LAST_FILE
fi
LAST_FILE_NO=`echo $LAST_FILE | sed s/$PREFIX/''/ | sed s/.csv/''/`
LAST_FILE_SIZE=`stat -c %s $LAST_FILE`
if [ `stat -c %s $LAST_FILE` -lt 200 ]
then
`/bin/sh ./sam.sh >> $LAST_FILE`
else
UPCOMING_FILE_NO=$((LAST_FILE_NO+1))
`/bin/sh ./sam.sh >> $PREFIX$UPCOMING_FILE_NO.csv`
fi
help is appreciated guys.
EDIT: Have got the secondary shell script to work too...
Now if anyone could help me with resetting after 3 files are done and starting from f1.
thanks
It sounds like you'd be better off using logrotate, depending on how your script is running. If you are running 'commands.sh' on a cron, you can have logrotate rotate out the logs. There is a good guide on logrotate here:
http://linuxers.org/howto/howto-use-logrotate-manage-log-files
If your commands.sh isn't going to be on a cron, meaning it's not a regular time interval that triggers it, you could manually set up a log rotation at the beginning of your script. I once had to do something similar. I found this guide really useful:
http://wazem.blogspot.com/2013/11/simple-bash-log-rotate-function.html

Shell script: fire a command if system time is equal to given time

I am very new to shell scripting.
The script should fire a command if the given time as command line argument is equal to the system time, else it should wait(poll) for the given time.
I started with very basics, but i am stuck here only:-(
#!/bin/sh
now=$(date +%k%M)
cur="055" # should be given as command line arg
if ($now == $cur)
then
echo "Fire command here"
else
echo "poll for time"
fi
When i execute the script:
./script.sh: line 5: 055: command not found
poll for time
Thanks for the help.
I think the above is just a small syntax error, instead of:
if ($now == $cur)
You may want to do this:
if [ $now -eq $cur ] #very basic comparison here, you may want to have a
#look at the bash comparisons
Update
Could you change the variable to,
$cur=$(date +%H%M)
And in case the input is not provided by you, you should remove the space in front
of the $now
now=$(echo $now | sed 's/\s//g') #this removes the spaces in the input
You can run a program # a particular time with :
crontab -e
and
0 14 * * * command
to run command # 14 PM (by example)
begin="`date '+%s'`"
final=... #should be converted to seconds since epoch...
sleep $((final - begin))
exec_your_command
The problem you described seems to be exactly what crontab is designed to handle, from wikipedia "Cron is driven by a crontab (cron table) file, a configuration file that specifies shell commands to run periodically on a given schedule."
Here is a quick reference, it is reletively barebones, but should be enough to determine if it meets your needs.
Use following code
if [ $now == $cur ]

How to do multiple if, else statements in this case in bash script?

Right now I am running a few programs with bash scripts (with Cygwin).
Basically what I am doing is after the program is starting, a loop is ran that checks that the program is still running.
I was doing:
while true
do
if [ "$(ps -W | grep -w name | gawk '{print $8,$9}' | gawk -F \\ '{print $4}')" == 'program' ];then
sleep 1
else
"start program" (whatever is needed here)
fi
done
But I started to realize having such a script multiple times is just causing unnecessary system resources to be used.
I tried doing an if then, elif, but it never goes past the first if.
I need it to go "alright the first if is negative, try the next, go to the end, start over".
Here is the copy of my script and I forgot to say I was using Cygwin but that really doesn't change anything cause it seems to still use normal bash scripting just maybe different paths to start files. http://pastebin.com/s8ZdPQMn and yes the h. is not there I just can't seem to edit the pastebin.
My overall plan is check that first SRPro is running, check the next, etc, only triggering if it's detected one is not running.
EDIT: I solved it. Not exactly sure why but in my original single file per program, gawk printing $4 at the end gave me what I wanted, but for some reason when doing it this way, it turned to $5. So changing $4 to $5 made the script work.
EDIT: One really strange issue though is, it will work minutes on end, then all the sudden get confused at times, and start 7 copies of one program or something. Also it can be random on which it starts.
You might find the wait command (try help wait from a bash prompt) useful. It's unclear exactly what you want, but as an example, here's a basic respawn function:
$ respawn () {
> while true
> do
> "${#}" &
> wait ${!}
> echo "respawning ..."
> done
> }
$ respawn some_program arg1 arg2 etc

How to create a system generated alert tone in linux , without using any external audio file?

I have a linux shell script , which i need to run at every 30 min interval, so i'm using a infinite for loop like this:
for ((i=0;;i++))
do
clear;
./myscript.sh;
sleep 30m;
done
i want to add a line of code before clear , so that before the script executes everytime,system generates an alert tone or sound , which will make the user aware that 30 min interval has passed by and the script is executing again.Any way to do that?
I currently dont have any external .wav file in my system which can be played as an alert. kindly suggest.
For a simple tone you can use beep command. You can find this command in debian repositories, the package name is beep.
It is a simple script and you can use it in commandline.
you can emit a BEL character (char code 7, ^G) to /dev/console. Beware that only root is allowed to do so.
With proper escape codes, you can even control the length and frequency of a beep.
For example, the code below constitutes, run in a cron job, a cuckoo clock :-)
#!/bin/sh
h=$(((`date +%k`+11)%12+1))
echo -en "\033[11;200]" >/dev/console
for ((i=0; i<h; i++)); do
echo -en "\033[10;600]\a"
usleep 200000
echo -en "\033[10;490]\a"
usleep 500000
done >/dev/console
echo -en "\033[10;750]\033[11;100]" >/dev/console

Trace of executed programs called by a Bash script

A script is misbehaving. I need to know who calls that script, and who calls the calling script, and so on, only by modifying the misbehaving script.
This is similar to a stack-trace, but I am not interested in a call stack of function calls within a single bash script.
Instead, I need the chain of executed programs/scripts that is initiated by my script.
A simple script I wrote some days ago...
# FILE : sctrace.sh
# LICENSE : GPL v2.0 (only)
# PURPOSE : print the recursive callers' list for a script
# (sort of a process backtrace)
# USAGE : [in a script] source sctrace.sh
#
# TESTED ON :
# - Linux, x86 32-bit, Bash 3.2.39(1)-release
# REFERENCES:
# [1]: http://tldp.org/LDP/abs/html/internalvariables.html#PROCCID
# [2]: http://linux.die.net/man/5/proc
# [3]: http://linux.about.com/library/cmd/blcmdl1_tac.htm
#! /bin/bash
TRACE=""
CP=$$ # PID of the script itself [1]
while true # safe because "all starts with init..."
do
CMDLINE=$(cat /proc/$CP/cmdline)
PP=$(grep PPid /proc/$CP/status | awk '{ print $2; }') # [2]
TRACE="$TRACE [$CP]:$CMDLINE\n"
if [ "$CP" == "1" ]; then # we reach 'init' [PID 1] => backtrace end
break
fi
CP=$PP
done
echo "Backtrace of '$0'"
echo -en "$TRACE" | tac | grep -n ":" # using tac to "print in reverse" [3]
... and a simple test.
I hope you like it.
You can use Bash Debugger http://bashdb.sourceforge.net/
Or, as mentioned in the previous comments, the caller bash built-in. See: http://wiki.bash-hackers.org/commands/builtin/caller
i=0; while caller $i ;do ((i++)) ;done
Or as a bash function:
dump_stack(){
local i=0
local line_no
local function_name
local file_name
while caller $i ;do ((i++)) ;done | while read line_no function_name file_name;do echo -e "\t$file_name:$line_no\t$function_name" ;done >&2
}
Another way to do it is to change PS4 and enable xtrace:
PS4='+$(date "+%F %T") ${FUNCNAME[0]}() $BASH_SOURCE:${BASH_LINENO[0]}+ '
set -o xtrace # Comment this line to disable tracing.
~$ help caller
caller: caller [EXPR]
Returns the context of the current subroutine call.
Without EXPR, returns "$line $filename". With EXPR,
returns "$line $subroutine $filename"; this extra information
can be used to provide a stack trace.
The value of EXPR indicates how many call frames to go back before the
current one; the top frame is frame 0.
Since you say you can edit the script itself, simply put a:
ps -ef >/tmp/bash_stack_trace.$$
in it, where the problem is occurring.
This will create a number of files in your tmp directory that show the entire process list at the time it happened.
You can then work out which process called which other process by examining this output. This can either be done manually, or automated with something like awk, since the output is regular - you just use those PID and PPID columns to work out the relationships between all the processes you're interested in.
You'll need to keep an eye on the files, since you'll get one per process so they may have to be managed. Since this is something that should only be done during debugging, most of the time that line will be commented out (preceded by #), so the files won't be created.
To clean them up, you can simply do:
rm /tmp/bash_stack_trace.*
UPDATE:
The code below should work. Now I have a newer answer with a newer code version that allows a message inserted in the stacktrace.
IIRC I just couldn't find this answer to update it as well at the time. But now decided code is better kept in git so latest version of the above should be in this gist.
original code-corrected answer below:
There was another answer about this somewhere but here is a function to use for getting stack trace in the sense used for example in the java programming language. You call the function and it puts the stack trace into the variable $STACK. It show the code points that led to get_stack being called. This is mostly useful for complicated execution where single shell sources multiple script snippets and nesting.
function get_stack () {
STACK=""
# to avoid noise we start with 1 to skip get_stack caller
local i
local stack_size=${#FUNCNAME[#]}
for (( i=1; i<$stack_size ; i++ )); do
local func="${FUNCNAME[$i]}"
[ x$func = x ] && func=MAIN
local linen="${BASH_LINENO[(( i - 1 ))]}"
local src="${BASH_SOURCE[$i]}"
[ x"$src" = x ] && src=non_file_source
STACK+=$'\n'" "$func" "$src" "$linen
done
}
adding pstree -p -u `whoami` >>output in your script will probably get you the information you need.
The simplest script which returns a stack trace with all callers:
i=0; while caller $i ;do ((i++)) ;done
You could try something like
strace -f -e execve script.sh

Resources