I am trying to do following:
Get output of free -mo , take the 2nd line and log it to a file every 30 seconds.
When I run
$free -mo -s 30
It runs and displays output every 30 seconds.
But when I run
$ free -mo -s 30 | head -2 | tail -1
It runs only once. I am not able to figure out what is wrong.
free Manual says free -s 30 run the command every 30 seconds.
head -2 returns only the first 2 lines of output then quits. tail -1 returns the last line, then quits. When any program quits in a pipeline, it kills the entire pipeline, so free is stopped when head and tail finish.
Use free -mo -s 30 &> test.txt &
This will take all of the output from the free command and output it to test.txt and run it in the background.
Try
free -mos 30 | grep 'Mem:' >yourlog.txt
(but you might be better considering something like sar to capture this kind of data - it can also reports lots of other things - just postpone the filtering/extraction until you generate a resport from the data).
Will Hartung is right. Instead do this:
while true; do free -mo | head -2 | tail -1; sleep 30; done
Thanks to your answers. I was trying to monitor memory utilization of a process. I think I got it.
START_TIME=$(date);
cd /data;
INPUT_DATA=$1;
CODE_FILE=$2;
TIMES=$3;
echo "$START_TIME" > "$CODE_FILE.freeMemory_$TIMES.log";
free -mo -s 30 >> "$CODE_FILE.freeMemory_$TIMES.log" &
freepid=$!;
sleep 1m;
#echo "PID generated for free command -- $freepid";
START_TIME=$(date);
i=0;
while [ $i -le $TIMES ]
do
sh runCode.sh $CODE_FILE "output.csv" $INPUT_DATA;
i=`expr $i + 1`
done
END_TIME=$(date);
echo "process started at $START_TIME and ended at $END_TIME " ;
sleep 1m;
kill -9 $freepid;
END_TIME=$(date);
echo "$END_TIME" >> "$CODE_FILE.freeMemory_$TIMES.log";
Related
I am trying to write a small shell script, make it go to sleep for some amount of time like 20 seconds and then run it. Now if i open another terminal and try to run the same script, it shouldn't run as the process is running else where. How do I do it?
I know i should write something, make it go to sleep captures its pid and write a condition that if this pis is running somewhere then don't let it run anywhere. but how do i do it? Please give a code.
echo "this is a process"
sleep 60
testfilepid = `ps ax | grep test1.sh | grep -v grep | tr -s " " | cut -f1 -d " "| tail -1`
echo $testfilepid
if [[ $tesfilepid = " " ]]
sh test1.sh
else
echo "this process is already running"
fi
This is what I tried. when i execute this in 2 windows, both the windows give me the output this is a process.
You could use pgrep to check that your script/process is running, and negate the output, this is a very basic example that could give you an idea:
if ! pgrep -f sleep >/dev/null; then echo "will sleep" && sleep 3; fi
Notice the !, pgrep -f sleep will search for a process matching against full argument lists. (you could customize this to your needs). so if nothing matches your pattern then your script will be called.
I am using a bash script that calls multiple processes which have to start up in a particular order, and certain actions have to be completed (they then print out certain messages to the logs) before the next one can be started. The bash script has the following code which works really well for most cases:
tail -Fn +1 "$log_file" | while read line; do
if echo "$line" | grep -qEi "$search_text"; then
echo "[INFO] $process_name process started up successfully"
pkill -9 -P $$ tail
return 0
elif echo "$line" | grep -qEi '^error\b'; then
echo "[INFO] ERROR or Exception is thrown listed below. $process_name process startup aborted"
echo " ($line) "
echo "[INFO] Please check $process_name process log file=$log_file for problems"
pkill -9 -P $$ tail
return 1
fi
done
However, when we set the processes to print logging in DEBUG mode, they print so much logging that this script cannot keep up, and it takes about 15 minutes after the process is complete for the bash script to catch up. Is there a way of optimizing this, like changing 'while read line' to 'while read 100 lines', or something like that?
How about not forking up to two grep processes per log line?
tail -Fn +1 "$log_file" | grep -Ei "$search_text|^error\b" | while read line; do
So one long running grep process shall do preprocessing if you will.
Edit: As noted in the comments, it is safer to add --line-buffered to the grep invocation.
Some tips relevant for this script:
Checking that the service is doing its job is a much better check for daemon startup than looking at the log output
You can use grep ... <<<"$line" to execute fewer echos.
You can use tail -f | grep -q ... to avoid the while loop by stopping as soon as there's a matching line.
If you can avoid -i on grep it might be significantly faster to process the input.
Thou shalt not kill -9.
I'm trying to have a lightweight memory profiler for the matlab jobs that are run on my machine. There is either one or zero matlab job instance, but its process id changes frequently (since it is actually called by another script).
So here is the bash script that I put together to log memory usage:
#!/bin/bash
pid=`ps aux | grep '[M]ATLAB' | awk '{print $2}'`
if [[ -n $pid ]]
then
\grep VmSize /proc/$pid/status
else
echo "no pid"
fi
when I run this script in bash like this:
./script.sh
it works fine, giving me the following result:
VmSize: 1289004 kB
which is exactly what I want.
Now, I want to run this periodically. So I run it with watch, like this:
watch ./script.sh
But in this case I only receive:
no pid
Please note that I know the matlab job is still running, because I can see it with the same pid on top, and besides, I know each matlab job take several hours to finish.
I'm pretty sure that something is wrong with the quotes I have when setting pid. I just can't figure out how to fix it. Anyone knows what I'm doing wrong?
PS.
In the man page of watch, it says that commands are executed by sh -c. I did run my script like sh -c ./script and it works just fine, but watch doesn't.
Why don't you use a loop with sleep command instead?
For example:
#!/bin/bash
pid=`ps aux | grep '[M]ATLAB' | awk '{print $2}'`
while [ "1" ]
do
if [[ -n $pid ]]
then
\grep VmSize /proc/$pid/status
else
echo "no pid"
fi
sleep 10
done
Here the script sleeps(waits) for 10 seconds. You can set the interval you need changing the sleep command. For example to make the script sleep for an hour use sleep 1h.
To exit the script press Ctrl - C
This
pid=`ps aux | grep '[M]ATLAB' | awk '{print $2}'`
could be changed to:
pid=$(pidof MATLAB)
I have no idea why it's not working in watch but you could use a cron job and make the script log to a file like so:
#!/bin/bash
pid=$(pidof MATLAB) # Just to follow previously given advice :)
if [[ -n $pid ]]
then
echo "$(date): $(\grep VmSize /proc/$pid/status)" >> logfile
else
echo "$(date): no pid" >> logfile
fi
You'd of course have to create logfile with touch.
You might try just running ps command in watch. I have had issues in the past with watch chopping lines and such when they get too long.
It can be fixed by making the terminal you are running the command from wider or changing the column like this (may need to adjust the 160 to your liking):
export COLUMNS=160;
I need a script that will run a series of tail -f commands and output them into a file.
What I need is for tail -f to run for a certain amount of time to grep specific words. The reason it's a certain amount of time is because some of these values don't show up right away as this is a live log.
How can I run something like this for let's say 20 seconds, output the grep command and then continue on to the next command?
tail -f /example/logs/auditlog | grep test
Thanks
timeout 20 tail -f /example/logs/auditlog | grep test
tail -f /example/logs/auditlog | grep test &
pid=$!
sleep 20
kill $pid
What about this:
for (( N=0; $N < 20 ; N++)) ; do tail -f /example/logs/auditlog | grep test ; sleep 1 ; done
EDIT: I misread your question, sorry. You want something like this:
tail -f /example/logs/auditlog | grep test
sleep 20
I have an interesting situation I am trying to script. I have a program that outputs 26,000 lines after 10 seconds when it starts successfully. Otherwise I have to kill it and start it again. I tried doing something like this:
test $(./long_program | wc -l) -eq 26000 && echo "Started successfully"
but that only works if the program finishes running. Is there a clever way to watch the output stream of a command and make decisions accordingly? I'm at a loss, not quite sure even how to start searching for this. Thanks!
What about
./long_program > mylogfile &
pid=$!
sleep 10
# then test on mylogfile length and kill $pid if needed
count=0
until [ $count -eq 26000 ]; do
killall ./longrun
#start in background
./longrun >output.$$ &
sleep 10
count=$(wc -l output.$$ |awk '{print $1}')
done
echo "done"
#disown so it continues after current login quits
disown -h