I am running a process process_a in loop .
I want to get top -H logs stored for all process_a running in loop.
top -b -H -p `pgrep -d, -f process_a`
the above command gave logs for process_a for first loop only
is there a way to get to get top logs for upcoming loops as well ?
This script will repeat forever. I added the -n 1 option. This allows you to rerun the pgrep for each iteration. Note: I used init for the name of the process. Change that to process_a for yours.
#!/bin/sh
while [ true ]; do
top -b -H -n 1 -p `pgrep -d, -f init`
sleep 1
done
Related
So I've been trying to run the following command to check the consumption by the firefox process.
top -b | grep "firefox"
Now I can see the desired output in terminal for it. But now when I try to export it to a file, I'm not able to.
The command I'm running is:
top -b | grep "firefox" >> filename or top -b | grep "firefox" > filename
Please help.
You need the -n parm for top. For example,
top -b -n 1 | grep "firefox" >> firefox.out
Without -n 1 top will keep processing and will never make it to grep..
From the man page for top:
-n :Number-of-iterations limit as: -n number
Specifies the maximum number of iterations, or frames, top
should produce before ending.
Updated code with a while loop. It will loop forever unless you use
something like cntr variable. Remove the cntr code if you want
continuous monitoring:
#!/bin/sh
#
not_done=1
cntr=0
# Look for process firefox every 1 seconds
while [ "$not_done" -eq 1 ]
do
top -b -n 1 | grep "firefox" >> /tmp/firefox.out
sleep 1
((cntr=cntr+1))
# Addition logic to break out of loop after 10 iterations
if [ "$cntr" -eq 10 ]
then
not_done=0
fi
echo "cntr=$cntr"
done
You have to add in the command the flag -n
Change the number of processes to show. You will be prompted to enter the number.
To take a snapshot of a specific process in top utility, execute command with the PID (-p) flag.
Also i recommed if you want to take snapshots for the process one (PID), and get taking three snapshots of the PID:
top -p 678 -b -n3 > monitoring.out
How can I wait for a program to complete and then start another program automatically in another console?
EDIT: The first program can be long running and the other program should start right after the completion of the first program.
tail -f --pid=xyzu /dev/null && second_program
for example, in one console (terminal) invoke cat. In another console, use pstree -p | grep cat to find its process id. Then, in that other console, type tail -f --pid=6169 /dev/null && ls (replace the number with the correct one and use the needed command instead of ls). Then, end cat with CTL-C.
EDIT: Another example is, you want to shutdown the computer automatically when a long running program has completed:
first, find the pid of the long running program:
pstree -p | grep long_running_program
For instance, you find the pid 3373. Now, use the command:
sudo tail -f --pid=3373 /dev/null && shutdown -h now
I've been try to make cURL on a huge loop and I run the cURL into background process with bash, there are about 904 domains that will be cURLed
and the problem is that 904 domains can't all be embedded because of the PID limit on the Linux kernel. I have tried adding pid_max to 4194303 (I read in this discussion Maximum PID in Linux) but after I checked only domain 901 had run in background proccess, before I added pid_max is only around 704 running in the background process.
here is my loop code :
count=0
while IFS= read -r line || [[ -n "$line" ]];
do
(curl -s -L -w "\\n\\nNo:$count\\nHEADER CODE:%{http_code}\\nWebsite : $line\\nExecuted at :$(date)\\n==================================================\\n\\n" -H "X-Gitlab-Event: Push Hook" -H 'X-Gitlab-Token: '$SECRET_KEY --insecure $line >> output.log) &
(( count++ ))
done < $FILE_NAME
Anyone have another solution or fix it to handle huge loop to run cURL into background process ?
a script example.sh can be created
#!/bin/bash
line=$1
curl -s -L -w "\\n\\nNo:$count\\nHEADER CODE:%{http_code}\\nWebsite : $line\\nExecuted at :$(date)\\n==================================================\\n\\n" -H "X-Gitlab-Event: Push Hook" -H 'X-Gitlab-Token: '$SECRET_KEY --insecure $line >> output.log
then the command could be (to limit number of running process at a time to 50)
xargs -n1 -P50 --process-slot-var=count ./example.sh < "$FILE_NAME"
Even if you could run that many processes in parallel, it's pointless - starting that many DNS queries to resolve 900+ domain names in a short span of time will probably overwhelm your DNS server, and having that many concurrent outgoing HTTP requests at the same time will clog your network. A much better approach is to trickle the processes so that you run a limited number (say, 100) at any given time, but start a new one every time one of the previously started ones finishes. This is easy enough with xargs -P.
xargs -I {} -P 100 \
curl -s -L \
-w "\\n\\nHEADER CODE:%{http_code}\\nWebsite : {}\\nExecuted at :$(date)\\n==================================================\\n\\n" \
-H "X-Gitlab-Event: Push Hook" \
-H "X-Gitlab-Token: $SECRET_KEY" \
--insecure {} <"$FILE_NAME" >output.log
The $(date) result will be interpolated at the time the shell evaluates the xargs command line, and there is no simple way to get the count with this mechanism. Refactoring this to put the curl command and some scaffolding into a separate script could solve these issues, and should be trivial enough if it's really important to you. (Rough sketch:
xargs -P 100 bash -c 'count=0; for url; do
curl --options --headers "X-Notice: use double quotes throughout" \
"$url"
((count++))
done' _ <"$FILE_NAME" >output.log
... though this will restart numbering if xargs receives more URLs than will fit on a single command line.)
Is there an option for the Linux top command where I can filter processes by name and write the CPU usage of that process every second to a log file?
top & pgrep
To filter the output of top by process name, you can use pgrep to get a list of PIDs by process name then pass them to the -p option of top.
For example:
top -p $(pgrep -d',' http)
Note: the -d',' option delimits the PIDs with commas, which is what is expected by the top -p.
Note 2: top will return a failure message if there are no running processes that match the name you specify in pgrep.
To write the results of top to a file, use the -n 1 option (only one iteration) and redirect the output to your log file.
top -p $(pgrep -d',' http) -n 1 >> your_log_file
To do that every second, perhaps a while loop with a sleep would do?
while :; do top -p $(pgrep -d',' http) -n 1 >> your_log_file; sleep 1; done
To timestamp each entry, you can append the output of date. E.g.
while :; do top -p $(pgrep -d',' http) -n 1 >> log.txt; date >> log.txt; sleep 1; done
Another option is:
top -b -d 1 -p $(pgrep -d',' java) -n 120 > log.txt
The option -d allows to set the frequency used by top to refresh the
data.
The option -b means that the traditional interface of top is
not used. Instead, it sends everything to the standard output and then you can use a pipe (|) or a redirection (>).
The option -n informs about the number of iterations top should execute.
After that you can type:
cat log.txt | grep USER_OF_PROCESS
You will see the execution time of the process and also %CPU, Memory and all that.
#You can run following script as ./cpurecorder.sh pid filename
#It will generate output file with memory usage and cpu utilisation.
#You can log other variable by searching man for ps.
`enter code here`filepath=/home/rtcsadm # modify as desired
interval=20 # reports per minute
timelimit=6000 # how long to run, in seconds
mydate=`date "+%H:%M:%S"` # the timestamp
freq=$((60/$interval)) # for sleep function
while [ "$SECONDS" -le "$timelimit" ] ; do
ps -p$1 -opid -opcpu -opmem -ocomm -c | grep $1 | sed "s/^/$mydate /" >> $filepath/$2.txt
sleep 3
mydate=`date "+%H:%M:%S"`
done
By using the linux command "top"...
Then press the 'o' or 'O' to activate filter prompt. It will show a line indicating the filter format like this -
add filter #1 (ignoring case) as: [!]FLD?VAL
Then enter a filter like this and hit Enter.
COMMAND=<pattern>
Now top will show only those processes whose COMMAND field contains the "<pattern>" value
(Solution taken from ... https://techantidote.com/filter-top-using-process-name-in-linux/ )
I have a script that I mean to be run from cron that ensures that a daemon that I wrote is working. The contents of the script file are similar to the following:
daemon_pid=`ps -A | grep -c fsdaemon`
echo "daemon_pid: " $daemon_pid
if [ $daemon_pid -eq 0 ]; then
echo "restarting fsdaemon"
/etc/init.d/fsdaemon start
fi
When I execute this script from the command prompt, the line that echoes the value of $daemon_pid is reporting a value of 2. This value is two regardless of whether my daemon is running or not. If, however, I execute the command with back quotes and then examine the $daemon_pid variable, the value of $daemon_pid is now one. I have also tried single stepping through the script using bashdb and, when I examine the variables using that tool, they are what they should be.
My question therefore is: why is there a difference in the behaviour between when the script is executed by the shell versus when the commands in the script are executed manually? I'm sure that there is something very fundamental that I am missing.
You're very likely encountering the grep as part of the 'answer' from ps.
To help fully understand what is happening, turn off the -c option, to see what data is being returned from just ps -A | grep fsdameon.
To solve the issue, some systems have a p(rocess)grep (pgrep). That will work, OR
ps -A | grep -v grep | grep -c fsdaemon
Is a common idiom you will see, but at the expense of another process.
The cleanest solution is,
ps -A | grep -c '[f]sdaemon'
The regular expression syntax should work with all greps, on all systems.
I hope this helps.
The problem is that grep itself shows up... Try running this command with anything after grep -c:
eple:~ erik$ ps -a | grep -c asdfladsf
1
eple:~ erik$ ps -a | grep -c gooblygoolbygookeydookey
1
eple:~ erik$
What does ps -a | grep fsdaemon return? Just look at the processes actually listed... :)
Since this is Linux, why not try the pgrep? This saves you a pipe, and you don't end up with grep reporting back the daemon script itself running.
Aany process with arguments including that name will add to the count - grep, and your script.
psing for a process isn't really reliable, you should use a lock file.
As several people have pointed out already, your process count is inflated because ps | grep detects (1) the script itself and (2) the subprocess created by the backquotes, which inherits the name of the main script. So an easy solution is to change the name of the script to something that doesn't include the name you're looking for. But you can do better.
The "best-practice" solution that I would suggest is to use the facilities provided by your operating system. It's not uncommon for an init script to create a PID file as part of the process of starting your daemon; in other words, instead of just running the daemon itself, you use a wrapper script that starts the daemon and then writes the process ID to a file somewhere. If start-stop-daemon exists on your system (and I think it's fairly common these days), you can use that like so:
start-stop-daemon --start --quiet --background \
--make-pidfile --pidfile /var/run/fsdaemon.pid -- /usr/bin/fsdaemon
(obviously replace the path /usr/bin/fsdaemon as appropriate) to start it, and then
start-stop-daemon --stop --quiet --pidfile /var/run/fsdaemon.pid
to stop it. start-stop-daemon has other options that might be useful to you, which you can investigate by reading the man page.
If you don't have access to start-stop-daemon, you can write a wrapper script to do basically the same thing, something like this to start:
echo "$$" > /var/run/fsdaemon.pid
exec /usr/bin/fsdaemon
and this to stop:
kill $(< /var/run/fsdaemon/pid)
rm /var/run/fsdaemon.pid
(this is pretty crude, of course, but it should normally work).
Anyway, once you have the setup to generate a PID file, whether by using start-stop-daemon or not, you can update your check script to this:
daemon_pid=`ps --no-headers --pid $(< /var/run/fsdaemon.pid) | wc -l`
if [ $daemon_pid -eq 0 ]; then
echo "restarting fsdaemon"
/etc/init.d/fsdaemon restart
fi
(one would think there would be a concise command to check whether a given PID is running, but I don't know it).
If you don't want to (or can't) create a PID file, I would at least suggest pgrep instead of ps | grep, since pgrep will search directly for a process by name and won't find anything that just happens to include the same string.
daemon_pid=`pgrep -x -c fsdaemon`
if [ $daemon_pid -eq 0 ]; then
echo "restarting fsdaemon"
/etc/init.d/fsdaemon restart
fi
The -x means "match exactly", and -c works as with grep.
By the way, it seems a bit misleading to name your variable daemon_pid when it is actually a count.