When I run a top and get it to show swap usage, I get the following output. However, I have disabled swap with swapoff -a prior to starting firefox. What is shown in the SWAP field here then?
When I do cat /proc/meminfo, I get a nonzero value for a field named SwapCached. What is this? My guess it this is the aggregate of all the SWAP values shown in top. How are these related to total memory used by a process?
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ SWAP
COMMAND
1604 dumrat 20 0 287m 62m 26m R 1 3.1 0:05.03 225m firefox-bin
1415 dumrat 9 -11 94264 4668 3552 S 0 0.2 0:00.10 87m pulseaudio
My best guess id this.
When you say swapoff, it prevents tasks from further 'swapping' (techically, it's paging, not swapping), but does not remove already swapped pages from swap devices. Often various shared libraries go to swap right at the moment of loading: they are here to stay for long time, no point wasting time swapping them when the load is high. These libraries are in RAM as long as they are needed by active processes, but also in swap space.
Maybe Firefox uses some of these librsries that are already mapped to swap space — Xlib, GTK, etc, and this swap space is counter to its 'SWAP' column. Linux tends to count all shared pages to each process that shares them, RAM or not.
Again, this is my guess; take with a grain of salt.
Related
I come up with this question while using the ps aux command.
Here I can see that a few processes are at 0% CPU 0% MEM 0 VSZ 0 RSS.
If a daemon is not using any memory, why and how could it be displayed in the first place ? I kind of understand that 0% CPU usage mean the process is not currently in use but wouldn't 0% MEM mean no process at all ?
I wanted to check if this was somehow systems daemon specific so I made a simple C program with an infinite loop, without any variables.
void main()
{
while (1){}
}
This time VSZ and RSS have actual values, while MEM staying at 0%.
What is happening here ?
%MEM is probably not fully documented on your system. AIX manual about ps command says:
%MEM
Calculated as the sum of the number of working segment and code
segment pages in memory times 4 (that is, the RSS value), divided by
the size of the real memory in use, in the machine in KB, times 100,
rounded to the nearest full percentage point. This value attempts to
convey the percentage of real memory being used by the process.
Unfortunately, like RSS, it tends the exaggerate the cost of a process
that is sharing program text with other processes. Further, the
rounding to the nearest percentage point causes all of the processes
in the system that have RSS values under 0.005 times real memory size
to have a %MEM of 0.0.
As you could have suspected by examining the output, some rounding have been applied. So if the value is too low %0.0 is printed.
And, this measure percentage of the real memory usage, which means that it doesn't reflect the size of the process but only which part of the process is actually mapped to real memory.
In your first case %0.0 for CPU just means that the process exists but actually does nothing and it is probably in a waiting state (or consuming a very small percentage of the processing power), not "that it is is not currently in use". In your second case, your process is active, it is in fact very busy (this is what %97.7 reflects), but what it does is stupid (infinite loop doing nothing).
To understand all of this, you may read about process state, process scheduling and virtual memory.
While Jean-Baptiste's answer is correct as far as it goes, I believe it's more significant in this case that all of the 0 memory in all three fields processes you're noting are kernel threads. Their memory is all kernel memory, and doesn't show up on top or ps. You can tell it's a kernel thread on linux both by the command being encapsulated by brackets and by the process consuming no memory in the VSZ column. (That's the column that represents basically everything that could be considered the process's memory. It's only 0 for kernel threads, and that only because they don't properly report their memory.
Also note that with a start time in 2018 and having consumed no more than 1 minute 41 seconds, none of those jobs are really very active.
I am trying to figure out how much memory a program requires. I need to run multiple instances of this program and I want to figure out how much additional RAM I need. I did some research and to find the amount of memory used I did: top, which gave me the following
Mem: 32880876k total, 7573308k used, 25307568k free, 192956k buffers
Swap: 8241148k total, 0k used, 8241148k free, 4800560k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27464 root 20 0 3107m 1.9g 19m S 99.6 6.0 0:37.78 krypton
RES, if I understood correctly is the resident memory which is the amount of memory that resides in the actual RAM. When I ran 2 instances of the program i noticed that the RES field for both were at around 5Gb each but the total memory used (shown at the top) did not go up by 10Gb as I had expected, but instead went up by less than 1Gb.
I did some research on this and found out that RES also includes shared memory. Using the same scenario as above, I noticed the shared memory SHR was at around 4.7Gb for each. So performing a simple subtraction yields around 0.3Gb, so does that mean that when I run 2 instances of my program that I need 4.7Gb+(0.3Gb)*2=5.3Gb of RAM? Also does the SHR memory include virtual memory as well?
If so, do I need to subtract the VIRT from SHR to find out the amount of memory which comes from the RAM? If this memory usage were to go up linearly, would running 20 instances take 4.7Gb+(0.3Gb)*20=10.7Gb of RAM?
You want to inspect /proc/pid/smaps. Run 2 instances and then look for Private_* fields.
I'm running node.js on a Raspberry Pi 3 B with the following free memory:
free -m
total used free shared buffers cached
Mem: 973 230 742 6 14 135
-/+ buffers/cache: 80 892
Swap: 99 0 99
How can I configure node (v7) to not use all the free memory? To prolong the SD card life, I would like to prevent it from going to swap.
I am aware of --max_old_space_size:
node --v8-options | grep -A 5 max_old
--max_old_space_size (max size of the old space (in Mbytes))
type: int default: 0
I know some of the answer is application specific, however what are some general tips to limit node.js memory consumption to prevent swapping? Also any other tips to squeeze more free ram out of the pi would be appreciated.
I have already set the memory split so that the GPU has the minimum 16 megs of RAM allocated.
The only bulletproof way to prevent swapping is to turn off swapping in the operating system (delete or comment out any swap lines in /etc/fstab for permanent settings, or use swapoff -a to turn off all swap devices for the current session). Note that the kernel is forced to kill random processes when there is no free memory available (this is true both with and without swap).
In node.js, what you can limit is the size of V8's managed heap, and the --max-old-space-size flag you already mentioned is the primary way for doing that. A value around 400-500 (megabytes) probably makes sense for your Raspberry. There's also --max-semi-space-size which should be small and you can probably just stick with the default, and --max-executable-size for generated code (how much you need depends on the app you run; I'd just stick with the default).
That said, there's no way to limit the overall memory usage of the process, because there are other memory consumers outside the managed heap (e.g. node.js itself, V8's parser and compiler). There is no way to set limits on all kinds of memory usage. (Because what would such a limit do? Crash when memory is needed but not available? The kernel will take care of that anyway.)
Trying to understand what's going on with my server.
It's a 2 cpu server, so:
$> grep 'model name' /proc/cpuinfo | wc -l
2
While on load avergae, queue is showing ~8 :
$> uptime
16:31:30 up 123 days, 9:04, 1 user, load average: 8.37, 8.48, 8.55
So You can assume, load is really high and things are pailing up, there is some load on the system and it's not just a spike.
However, Looking at top cpu consumers:
> ps -eo pcpu,pid,user,args | sort -k 1 -r | head -6
%CPU PID USER COMMAND
8.3 27187 **** server_process_c
1.0 22248 **** server_process_b
0.5 22282 **** server_process_a
0.0 31167 root head -6
0.0 31166 root sort -k 1 -r
0.0 31165 root ps -eo pcpu,pid,user,args
Results of free command:
total used free shared buffers cached
Mem: 7986 7934 52 0 9 2446
-/+ buffers/cache: 5478 2508
Swap: 17407 60 17347
This is the result on an ongoing basis, e.g. not even
a single CPU is being used, top consumer, is always ~8.5%.
My Question: What are my ways to track down the root of the high load?
Based on your free output, there are times when system memory is exhausted so swap buffer is used (see column used = 60). Total memory used used - (buffers + cached) which result almost zero. It means there are time when all physical RAM is consumed.
For server, try to avoid page fault which may cause swapping data from system memory to swap buffer (or vice versa) as much as possible because accessing hard drive is very slow than system RAM.
In your top output, try to investigate wa column. Higher percentage value means CPU spend more times waiting for data IO from disk rather than doing meaningful computation.
Cpu(s): 87.3%us, 1.2%sy, 0.0%ni, 27.6%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Try to reduce daemon or service that you do not need to reduce memory footprint and consider to add more RAM to the system.
For 2 CPU(s) server, ideal load is less than 2.0 (each CPU load is less than 1.0). Load of 8.0 means each CPU load is roughly 4.0 which is not very good.
Have you tried the htop command? It shows more information in a helpful way sometimes.
I have a process that appears to be deadlocked:
# strace -p 5075
Process 5075 attached - interrupt to quit
futex(0x419cf9d0, FUTEX_WAIT, 5095, NULL
It is sitting on the "futex" system call, and seems to be indefinitely waiting on a lock. The process is shown to be consuming a large amount of CPU when "top" is run:
# top -b -n 1
top - 23:13:18 up 113 days, 4:19, 1 user, load average: 1.69, 1.74, 1.72
Tasks: 269 total, 1 running, 268 sleeping, 0 stopped, 0 zombie
Cpu(s): 8.1%us, 0.1%sy, 0.0%ni, 91.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 12165696k total, 3810476k used, 8355220k free, 29440k buffers
Swap: 8388600k total, 43312k used, 8345288k free, 879988k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5075 omdb 18 0 2373m 1.7g 26m S 199.7 14.9 102804:11 java
The process is also shown to be in a "S" - Sleep state, which makes sense if it's waiting on some resource. However, I don't understand why CPU utilization would be close to 200% if the process is in the sleep state. Why does top report such high CPU utilization on a sleeping process? Shouldn't its CPU utilization be zero?
There is no correlation between CPU usage as reported by top and process state. The man page says (emphasis mine):
%CPU -- CPU usage
The task's share of the elapsed CPU time since the last screen update, expressed as a percentage of total CPU time.
So, your process indeed used a huge amount of processor time since the last screen update. It is sleeping, yes, but that's because the currently running process is top itself (which makes sense, since it's currently updating the screen).
Does your application fork child processes? The strace output may indicate that the main process is just waiting for child processes to finish their work. If so, you could try running
strace -f -p 5075
to trace the child processes as well.
The top output is perfectly normal.
The load average calculations include processes that are waiting on something (mutexes/futexes, IO etc) as well as processes that are actually using the CPU. Test it by, say, running something like:
dd if=/dev/sda of=/dev/null
and watching top output to see what happens. It will increase the load average by 1.
If you look at this line:
Cpu(s): 8.1%us, 0.1%sy, 0.0%ni, 91.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
the "id" in "91.8%id" means "idle". So the CPU isn't actually doing much at all.
Let me add my two cents.
Top shows state of the process at a particular moment of time.But IT DOES NOT mean that this process was all the previous time in this state.
This sugestion is completely wrong.
The process could switch between R and S state million times between previous top time and current top moment so if process switches rapidly between R and S state
you can easiky catch it in S state.
However, it uses cpu time between switches.
So please feel the difference between cpu_usage thing ( it describes a period of time ) and state thing ( it describes a particular moment of time ).
Let me give a clear example.
Some person have stolen 3 apples from your pocket during last 10 minutes.
However, right now it does not steal apples from your pocket.
stolen apples = cpu_usage,
the fact that the person does not steal apples right now = state of process
Then, it is completely wrong to get one characteristic and try to predict another one.