Linux : RedHat/Fedora
What is the difference between these memory values:
RES from top command
RSS from ps command
If you are talking about the difference between the RES column in top -p $(pidof process) and the RSS column in the ps aux | grep $(pidof process) command, there is no difference, as both the tools get this value from the /proc/$(pidof process)/stat file.
You can always cat /proc/$(pidof process)/status for a human readable format.
Related
I have used the following command for a while to keep headers on ps output.
ps aux | { head -1; grep root; }
The output will look something like the following.
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND
root 142 0.0 0.0 1234567 2520 ?? Ss 3:14AM 0:08.03 /usr/sbin/notifyd
root 55 0.0 0.0 7890123 2460 ?? Ss 3:14AM 0:01.94 /usr/sbin/syslogd
...
However, when used with other command line programs the output is not as expected.
Take the following df example.
df -h
Outputs the following.
Filesystem Size Used Avail Use% Mounted on
/dev/disk1s1 466G 103G 362G 22% /
/dev/disk1s4 466G 1.1G 362G 1% /blah/blah/blah
Using df in a similar syntax as the above example with ps.
df -h | { head -1; grep disk1; }
Outputs the following.
Filesystem Size Used Avail Use% Mounted on
The expectation is that the output would look essentially the same as the straight df -h command.
Why does this differ from ps?
I feel that knowing these differences will help me understand BASH processing more completely.
Thank you!
It's because head is buffering its input. It reads into a large buffer from the pipe, then starts extracting lines from that buffer. After it has read and printed the first N lines, it exits. Then grep starts reading from the pipe. But anything that head already read into its buffer is not available.
The reason it seems to work with ps is because it produces lots of output, which doesn't fit into this buffer. grep is then able to process the rest of the output. But I think if you check carefully you'll see that the result is incomplete.
The output of df is much smaller, it all fits into the buffer that head uses, so there's nothing left for grep to process.
The buffer size is probably something like 4K characters.
You can do what you want with awk:
df -h | awk 'NR == 1 || /disk1/'
ps aux | awk 'NR == 1 || /root/'
NR is the line number, so this prints the line if it's the first line or it matches the regexp.
Sed(1) can also be used to filter output in this case:
ps aux | sed -n '1p; /root/p'
-n: Don't echo line of input to standard output after all commands have been applied to it.
1p; "address" of line1, with 'p' to print pattern space
/root/p; "address" of /regexp/ matching "root", with 'p' to print pattern space
Alternative:
ps aux | sed '1p; /root/p; d;'
Some systems may require ps -aux i.t. dash (-) to prefix options. Linux and *BSD systems do not (cannot be sure how macOS behaves, I don't have such a system to check out).
I'm trying to run a program on a dedicated core in Linux. (I know Jailhouse is a good way to do so, but I have to use off-the-shelf Linux. :-( )
Other processes, such as interrupt handlers, kernel threads, service progresses, may also run on the dedicated core occasionally. I want to disable as many such processes as possible. To do that, I need first pin down the list of processes that may run on the dedicated core.
My question is:
Is there any existing tools that I can use to trace the list of PIDs or processes that run on a specific core over a time interval?
Thank you very much for your time and help in this question!
TL;DR Dirty hacky solution.
DISCLAIMER: At some point stops working "column: line too long" :-/
Copy this to: core-pids.sh
#!/bin/bash
TARGET_CPU=0
touch lastPIDs
touch CPU_PIDs
while true; do
ps ax -o cpuid,pid | tail -n +2 | sort | xargs -n 2 | grep -E "^$TARGET_CPU" | awk '{print $2}' > lastPIDs
for i in {1..100}; do printf "#\n" >> lastPIDs; done
cp CPU_PIDs aux
paste lastPIDs aux > CPU_PIDs
column -t CPU_PIDs > CPU_PIDs.humanfriendly.tsv
sleep 1
done
Then
chmod +x core-pids.sh
./core-pids.sh
Then open CPU_PIDs.humanfriendly.tsv with your favorite editor, and ¡inspect!
The key is in the "ps -o cpuid,pid" bit, for more detailed info, please comment. :D
Explanation
Infinite loop with
ps -o cpuid,pid | tail -n +2 | sort | xargs -n 2 | grep -E "^$TARGET_CPU" | awk '{print $2}' > lastPIDs
ps ax -o cpuid,pid
Show pid's associated to CPU
tail -n +2
remove headers
sort
sort by cpuid
xargs -n 2
remove white spaces at begging
grep -E "^$TARGET_CPU"
filter by CPU id
awk '{print $2}'
get pid column
> lastPIDs
output to file those las pid's for the target CPU id
for i in {1..10}; do printf "#\n" >> lastPIDs; done
hack for pretty .tsv print with the "columns -t" command
cp CPU_PIDs aux
CPU_PIDs holds the whole timeline, we copy it to aux file to allow the next command to use it as input and output
paste lastPIDs aux > CPU_PIDs
Append lastPIDs columns to the whole timeline file CPU_PIDs
column -t CPU_PIDs > CPU_PIDs.humanfriendly.tsv
pretty print whole timeline CPU_PIDs file
Attribution
stackoverflow answer to: ps utility in linux (procps), how to check which CPU is used
by Mikel
stackoverflow answer to: Echo newline in Bash prints literal \n
by sth
stackoverflow answer to: shell variable in a grep regex
by David W.
superuser answer to: Aligning columns in output from a UNIX command
Janne Pikkarainen
nixCraft article: HowTo: Unix For Loop 1 to 100 Numbers
The best way to obtain what you want is to operate as follows:
Use the isolcpus= Linux kernel boot parameter to "free" one core from the Linux scheduler
Disable the irqbalance daemon (in case it is executing)
Set the IRQs affinities to the other cores by manually writing the CPU mask on /proc/irq/<irq_number>/smp_affinity
Finally, run your program setting the affinity to the dedicated core through the taskset command.
In this case, such core will only execute your program. For checking, you can type ps -eLF and look at the PSR column (which specifies the CPU number).
Not a direct answer to the question, but I am usually using perf context-switches software event to identify the perturbation of the system or other processes on my benchmarks
I'm trying to figure out what's the difference between
ps | wc
and
ps r | wc
In linux the | pipe character issues the output of one command to the input of another. In this case ps output is sent to wc. So the difference in the two commands is really the difference between ps and ps r.
You can issue these commands separately and see the difference in output. You can also view the manpage for the ps command with man ps where you will find:
r Restrict the selection to only running processes.
So, the difference is that one counts the number of processes for the user, while the other counts only the running processes.
By default, ps selects all processes with the same effective user ID (euid=EUID) as the current user and associated with the same terminal as the invoker.
By specifying the 'r', you are restricting it to only running processes.
By piping it through to wc, you get the newline, word, and byte counts respectively.
I am working on unix.
I want to knwo the current cpu usage of a process.
I understood that ps give the average of cpu used till the process is up - it is not the current usage.
Is there a way to print only the cpu from the top command without 10 more parameters and
headers? I know how to do it with awk - this is not the way i want to do it.
top -p 20705 -bc -n 1 | tail -n 2 | awk '{ print $9}' | head -n 1
If there is another simple way to do it, not reading /proc/stat...
If there is a simple way doing it from c++, it is also ok.
Most likely, you will need to read /proc/stat, However, here is an interesting article with C code that may help you out. To understand and use the output from the program you should do man 5 proc. And here is the source code.
The bottom line is that you will need to read from /proc/stat to do what you want.
to see cpu usage of a proccess whose pid is 24556
ps -p 24556 -o \%cpu=
to see mem usage of a process named syslogd
ps -C syslogd -o \%mem=
I am trying to use the ps -o command to get just specific info about processes matching a certain name. However, I am having some issues on this, when I try to use this even to just get all processes, like so, it just returns a subset of what a normal ps -ef would return (it doesn't return nearly the same number of results so its not returning all running processes)
ps -ef -o pid,time,comm
I want to try something like this (below) but incorporate the ps -o to just get specific info from it (just the PID)
ps -ef |grep `whoami`| grep firefox-bin
Any advice is appreciated as to how to do this properly, thanks
This will get you the PID of a process by name:
pidof name
Which you can then plug back in to ps for more detail:
ps -p $(pidof name)
This is a bit old, but I guess what you want is: ps -o pid -C PROCESS_NAME, for example:
ps -o pid -C bash
EDIT: Dependening on the sort of output you expect, pgrep would be more elegant. This, in my knowledge, is Linux specific and result in similar output as above. For example:
pgrep bash
ps -fC PROCESSNAME
ps and grep is a dangerous combination -- grep tries to match everything on each line (thus the all too common: grep -v grep hack). ps -C doesn't use grep, it uses the process table for an exact match. Thus, you'll get an accurate list with: ps -fC sh rather finding every process with sh somewhere on the line.
Sometimes you need to grep the process by name - in that case:
ps aux | grep simple-scan
Example output:
simple-scan 1090 0.0 0.1 4248 1432 ? S Jun11 0:00
Sorry, much late to the party, but I'll add here that if you wanted to capture processes with names identical to your search string, you could do
pgrep -x PROCESS_NAME
-x Require an exact match of the process name, or argument list if -f is given.
The default is to match any substring.
This is extremely useful if your original process created child processes (possibly zombie when you query) which prefix the original process' name in their own name and you are trying to exclude them from your results. There are many UNIX daemons which do this. My go-to example is ninja-dev-sync.