I have a server with an open port which receives between 50 and 1000 messages per second. By message I mean that a single line of text is sent.
Essentially we want to record these messages in a file which will be processed every hour (or x minutes).
I have created a bash script (see below) which runs in the background and it works except when I kill the socat process (so I can take the file for processing and it can start a new file) we get part of a message, plus I am sure we are losing messages during the split second that socat is down.
DELAY="3600"
while true
do
NEXT_STOP=`date +%s --date "$DELAY second"`
(
while [ "$(date +%s)" -lt "$NEXT_STOP" ]
do
killall socat
socat -u TCP-LISTEN:6116,reuseaddr,keepalive,rcvbuf=131071,reuseaddr OPEN:/var/ais/out_v4.txt,creat,append
done
) & sleep $DELAY ; killall socat
mv /var/ais/out_v4.txt "/var/ais/_socat_received/"$(date +"%Y-%m-%d-%T")"__out_v4.txt"
done
Is there a way to:
Get socat to rotate its output file without killing the process
Or can we purge the content of the file whilst SOCAT is writing to it. e.g. cut the first 10000 lines into another file, so the output file remains a manageable size?
Many thanks in advance
For anyone interested the final solution looks like the following, the key difference to Nicholas solution below is that I needed to grep the PID of the socat process rather than use $?:
#!/bin/bash
DELAY=600
SOCAT_PID=$(/bin/ps -eo pid,args | grep "socat -u TCP-LISTEN:12456" | grep -v grep | awk '{ print $1 }')
while `kill -0 $SOCAT_PID`
do
touch /var/ais/out.txt
NEXT_STOP=`date +%s --date "$DELAY second"`
while `kill -0 $SOCAT_PID` && [ "$(date +%s)" -lt "$NEXT_STOP" ]
do
head -q - >> /var/ais/out.txt
done
mv /var/ais/out.txt "/var/ais/_socat_received/"$(date +"%Y-%m-%d-%T")"__out.txt"
done
In addition adding the start script within an infinite while loop so that when the client disconnects we restart socat and wait for the next connection attempt:
while true
do
socat -u TCP-LISTEN:12456,keepalive,reuseaddr,rcvbuf=131071 STDOUT | /var/ais/socat_write.sh
done
Instead of reinventing the wheel, you could use rotatelogs or multilog, both of which read log messages on std input and write them to log files with very flexible rotation config.
Even one step higher, the functionality you described is very similar to what rsyslogd and the like do.
Not so obvious! socat doesn't have any options for changing what it does with the connection halfway through. That means you'll have to be a little bit sneaky. Use socat with the output as STDOUT, and pipe to this script:
#!/bin/bash
rv=0
while [ $rv -lt 1 ]
do
NEXT_STOP=`date +%s --date "$DELAY second"`
while [ "$(date +%s)" -lt "$NEXT_STOP" ] && [ $rv -lt 1 ]
do
head -q - >> /var/ais/out_v4.txt
rv=$?
done
mv /var/ais/out_v4.txt "/var/ais/_socat_received/"$(date +"%Y-%m-%d-%T")"__out_v4.txt"
done
Totally untested, but looks reasonable?
socat -u TCP-LISTEN:6116,reuseaddr,keepalive,rcvbuf=131071,reuseaddr STDOUT | ./thescript.sh
Related
Im trying to make a little side project script to sit and monitor all of the /proc/ directories, for the most part I have the concept running and it works(to a degree). What im aiming for here is to scan through all the files and cat their status files and pull out the appropriate info, and then I would like to run this process in an infinite loop to give me live updates of when something is running on and dropping off of the scheduler. Right now every time you run the script, it will print 50+ blank lines and every single time it hits the proper regex it will print it correctly, but Im aiming for it to not roll down the screen the way it does. Any help at all would be appreciated.
regex="[0-9]"
temp=""
for f in /proc/*; do
if [[ -d $f && $f =~ /proc/$regex ]]; then
output=$(cat $f/status | grep "^State") #> /dev/null
process_id=$(cut -b 7- <<< $f)
state=$(cut -b 10-19 <<< $output)
tabs 4
if [[ $state =~ "(running)" ]]; then
echo -e "$process_id:$state\n" | sort >> temp
fi
fi
done
cat temp
rm temp````
To get the PID and status of running all processes, try:
awk -F':[[:space:]]*' '/State:/{s=$2} /Pid:/{p=$2} ENDFILE{if (s~/running/) print p,s; p="X"; s="X"}' OFS=: /proc/*/status
To get this output updated every second:
while sleep 1; do awk -F':[[:space:]]*' '/State:/{s=$2} /Pid:/{p=$2} ENDFILE{if (s~/running/) print p,s; p="X"; s="X"}' OFS=: /proc/*/status; done
I use the following script to check whether wget has finished downloading. To check for this, I'm looking for its PID, and when it is not found the computer shutdowns. This works fine for a single instance of wget, however, I'd like the script to look for all already running wget programs.
#!/bin/bash
while kill -0 $(pidof wget) 2> /dev/null; do
for i in '-' '/' '|' '\'
do
echo -ne "\b$i"
sleep 0.1
done
done
poweroff
EDIT: I'd would be great if the script would check if at least one instance of wget is running and only then check whether wget has finished and shutdown the computer.
In addition to the other answers, you can satisfy your check for at least one wget pid by initially reading the result of pidof wget into an array, for example:
pids=($(pidof wget))
if ((${#pids[#]} > 0)); then
# do your loop
fi
This also brings up a way to routinely monitor the remaining pids as each wget operation completes, for example,
edit
npids=${#pids[#]} ## save original number of pids
while (( ${#pids[#]} -gt 0 )); do ## while pids remain
for ((i = 0; i < npids; i++)); do ## loop, checking remaining pids
kill -0 ${pids[i]} || pids[$i]= ## if not unset in array
done
## do your sleep and spin
done
poweroff
There are probably many more ways to do it. This is just one that came to mind.
I don't think kill is a right Idea,
may be some thing on the lines like this
while [ 1 ]
do
live_wgets=0
for pid in `ps -ef | grep wget| awk '{print $2}'` ; # Adjust the grep
do
live_wgets=$((live_wgets+1))
done
if test $live_wgets -eq 0; then # shutdown
sudo poweroff; # or whatever that suits
fi
sleep 5; # wait for sometime
done
You can adapt your script in the following way:
#!/bin/bash
spin[0]="-"
spin[1]="\\"
spin[2]="|"
spin[3]="/"
DOWNLOAD=`ps -ef | grep wget | grep -v grep`
while [ -n "$DOWNLOAD" ]; do
for i in "${spin[#]}"
do
DOWNLOAD=`ps -ef | grep wget | grep -v grep`
echo -ne "\b$i"
sleep 0.1
done
done
sudo poweroff
However I would recommend using cron instead of an active waiting approach or even use wait
How to wait in bash for several subprocesses to finish and return exit code !=0 when any subprocess ends with code !=0?
I am using a bash script that calls multiple processes which have to start up in a particular order, and certain actions have to be completed (they then print out certain messages to the logs) before the next one can be started. The bash script has the following code which works really well for most cases:
tail -Fn +1 "$log_file" | while read line; do
if echo "$line" | grep -qEi "$search_text"; then
echo "[INFO] $process_name process started up successfully"
pkill -9 -P $$ tail
return 0
elif echo "$line" | grep -qEi '^error\b'; then
echo "[INFO] ERROR or Exception is thrown listed below. $process_name process startup aborted"
echo " ($line) "
echo "[INFO] Please check $process_name process log file=$log_file for problems"
pkill -9 -P $$ tail
return 1
fi
done
However, when we set the processes to print logging in DEBUG mode, they print so much logging that this script cannot keep up, and it takes about 15 minutes after the process is complete for the bash script to catch up. Is there a way of optimizing this, like changing 'while read line' to 'while read 100 lines', or something like that?
How about not forking up to two grep processes per log line?
tail -Fn +1 "$log_file" | grep -Ei "$search_text|^error\b" | while read line; do
So one long running grep process shall do preprocessing if you will.
Edit: As noted in the comments, it is safer to add --line-buffered to the grep invocation.
Some tips relevant for this script:
Checking that the service is doing its job is a much better check for daemon startup than looking at the log output
You can use grep ... <<<"$line" to execute fewer echos.
You can use tail -f | grep -q ... to avoid the while loop by stopping as soon as there's a matching line.
If you can avoid -i on grep it might be significantly faster to process the input.
Thou shalt not kill -9.
I'm trying to have a lightweight memory profiler for the matlab jobs that are run on my machine. There is either one or zero matlab job instance, but its process id changes frequently (since it is actually called by another script).
So here is the bash script that I put together to log memory usage:
#!/bin/bash
pid=`ps aux | grep '[M]ATLAB' | awk '{print $2}'`
if [[ -n $pid ]]
then
\grep VmSize /proc/$pid/status
else
echo "no pid"
fi
when I run this script in bash like this:
./script.sh
it works fine, giving me the following result:
VmSize: 1289004 kB
which is exactly what I want.
Now, I want to run this periodically. So I run it with watch, like this:
watch ./script.sh
But in this case I only receive:
no pid
Please note that I know the matlab job is still running, because I can see it with the same pid on top, and besides, I know each matlab job take several hours to finish.
I'm pretty sure that something is wrong with the quotes I have when setting pid. I just can't figure out how to fix it. Anyone knows what I'm doing wrong?
PS.
In the man page of watch, it says that commands are executed by sh -c. I did run my script like sh -c ./script and it works just fine, but watch doesn't.
Why don't you use a loop with sleep command instead?
For example:
#!/bin/bash
pid=`ps aux | grep '[M]ATLAB' | awk '{print $2}'`
while [ "1" ]
do
if [[ -n $pid ]]
then
\grep VmSize /proc/$pid/status
else
echo "no pid"
fi
sleep 10
done
Here the script sleeps(waits) for 10 seconds. You can set the interval you need changing the sleep command. For example to make the script sleep for an hour use sleep 1h.
To exit the script press Ctrl - C
This
pid=`ps aux | grep '[M]ATLAB' | awk '{print $2}'`
could be changed to:
pid=$(pidof MATLAB)
I have no idea why it's not working in watch but you could use a cron job and make the script log to a file like so:
#!/bin/bash
pid=$(pidof MATLAB) # Just to follow previously given advice :)
if [[ -n $pid ]]
then
echo "$(date): $(\grep VmSize /proc/$pid/status)" >> logfile
else
echo "$(date): no pid" >> logfile
fi
You'd of course have to create logfile with touch.
You might try just running ps command in watch. I have had issues in the past with watch chopping lines and such when they get too long.
It can be fixed by making the terminal you are running the command from wider or changing the column like this (may need to adjust the 160 to your liking):
export COLUMNS=160;
I need to detect user inactivity on my Linux system, to poweroff it (quite headless wife, and quite expensive electric bills... :-).
I need to schedule the script (in crontab), so no X-depending tool will work, I suppose (no $DISPLAY available).
Any thoughts?
UPDATE
For "user inactivity" I mean user input inactivity (mouse and keyboard).
Xautolock may the right tool for you. It allows you to specify a amount of minutes of inactivity after which a command should get triggered.
You might consider checking how long the screen saver has been running.
#!/bin/bash
screensaver="atlantis"
t=$(
# check for the screensaver
ps h -o start -C $screensaver |\
# hh:mm:ss -> seconds
awk -F: '{print $1"*3600+"$2"*60+"$3}' |\
bc -l 2>/dev/null | sort -n | tail -1
)
if [ "$t" == "" ]
then
exit 0
fi
n=$(
date "+%T" |\
awk -F: '{print $1"*3600+"$2"*60+"$3}' |\
bc -l 2>/dev/null
)
runtime=$(( $n - $t ))
if [ $runtime -gt 3600 ] || [ $runtime -lt 0 ]
then
echo shutdown -h now
fi
Using the time value requires subtracting now from then to get the run time.
Also, in my case, the screensaver program which appears in the process table will vary depending on which screensaver is selected. So, the above program assumes that 'atlantis' is the current screen saver.