The TOP command results:
Mem: 3991840k total, 1496328k used, 2495512k free, 156752k buffers
**Swap**: 3905528k total, **3980k** used, 3901548k free, 447860k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ **SWAP** COMMAND
28250 www-data 20 0 430m 210m 21m R 63 5.4 0:07.29 **219m** apache2
28266 www-data 20 0 256m 40m 21m S 30 1.0 0:01.94 **216m** apache2
28206 www-data 20 0 260m 44m 21m S 27 1.1 0:10.27 **215m** apache2
28259 www-data 20 0 256m 40m 21m S 26 1.0 0:02.21 **216m** apache2
The details list shows a group of apache2 processes are using SWAP memory about 210m+ each, but the summary reports only 3980k is used. The total SWAP memory in the detail list is much greater than in the summary. Do the two swap refer the same thing?
Quoted from http://www.linuxforums.org/articles/using-top-more-efficiently_89.html :
VIRT=RES+SWAP
As explained previously, VIRT includes anything inside task's
address space, no matter it is in RAM,
swapped out or still not loaded from
disk. While RES represents total RAM
consumed by this task. So, SWAP here
means it represents the total amount
of data being swapped out OR still not
loaded from disk. Don't be fooled by
the name, it doesn't just represent
the swapped out data.
Related
Memory occupied by unknown (VMware/CentOS)
Hello.
We have a server that has memory full used issue, but can not find what is eating memory.
Usage of memory has increased few days ago 40% -> neary 100% and stayed there since then.
We’d like to kill whatever eating memory.
[Env]
cat /etc/redhat-release
CentOS release 6.5 (Final)
# arch
x86_64
[status]
#free
total used free shared buffers cached
Mem: 16334148 15682368 651780 0 10168 398956
-/+ buffers/cache: 15273244 1060904
Swap: 8388600 129948 8258652
Result of top (some info are masked with ???)
#top -a
top - 10:19:14 up 49 days, 11:13, 1 user, load average: 1.05, 1.05, 1.10
Tasks: 145 total, 1 running, 143 sleeping, 0 stopped, 1 zombie
Cpu(s): 11.1%us, 18.4%sy, 0.0%ni, 69.5%id, 0.8%wa, 0.0%hi, 0.2%si, 0.0%st
Mem: 16334148k total, 15684824k used, 649324k free, 9988k buffers
Swap: 8388600k total, 129948k used, 8258652k free, 387824k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
17940 ??? 20 0 7461m 6.5g 6364 S 16.6 41.5 1113:27 java
4982 ??? 20 0 941m 531m 5756 S 2.7 3.3 611:22.48 java
3213 root 20 0 2057m 354m 2084 S 99.8 2.2 988:43.79 python
28270 ??? 20 0 835m 157m 5464 S 0.0 1.0 106:48.55 java
1648 root 20 0 197m 10m 1452 S 0.0 0.1 42:35.95 python
1200 root 20 0 246m 7452 808 S 0.0 0.0 2:37.42 rsyslogd
Processes that are using memory (some info are masked with ???)
# ps aux --sort rss
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1200 0.0 0.0 251968 7452 ? Sl Sep12 2:37 /sbin/rsyslogd -i /var/run/syslogd.pid -c 5
root 1648 0.0 0.0 202268 10604 ? Ss Sep12 42:36 /usr/lib64/???
??? 28270 0.1 0.9 855932 161092 ? Sl Sep14 106:49 /usr/java/???
root 3213 96.1 2.0 2107704 332932 ? Ssl Oct31 992:25 /usr/lib64/???
??? 4982 0.8 3.3 964096 544328 ? Sl Sep12 611:25 /usr/java/???
??? 17940 6.6 41.5 7649356 6781076 ? Sl Oct20 1113:49 /usr/java/???
Memory is almost 100% used, but with ps and top, we can only find processes that uses half of it.
We have checked slab cache, but it was not the cause.
Slab is only 90444 kB.
Nothing is found in syslog too.
Anyone has any idea how to detect what is eating memory?
Thank you in advance.
Run free -m and see the difference. Column available shows real free memory.
And take a look at the https://www.linuxatemyram.com/
we have restarted server and solved this case.
I was checking my server resource usage and noticed that the "cma" process is using a lot of RAM.
top - 15:04:54 up 127 days, 21:00, 1 user, load average: 0.27, 0.33, 0.24
Tasks: 157 total, 1 running, 156 sleeping, 0 stopped, 0 zombie
Cpu(s): 6.9%us, 0.3%sy, 0.0%ni, 92.6%id, 0.1%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 4043700k total, 4006616k used, 37084k free, 146968k buffers
Swap: 1052248k total, 1052240k used, 8k free, 1351364k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4308 root 16 0 2080m 977m 4708 S 0.0 24.8 0:00.02 cma
4396 root 15 0 2080m 977m 4708 S 0.0 24.8 0:00.10 cma
4397 root 16 0 2080m 977m 4708 S 0.0 24.8 3:47.36 cma
4398 root 15 0 2080m 977m 4708 S 0.0 24.8 2:31.40 cma
4399 root 15 0 2080m 977m 4708 S 0.0 24.8 0:00.34 cma
4400 root 18 0 2080m 977m 4708 S 0.0 24.8 0:00.00 cma
4403 root 15 0 2080m 977m 4708 S 0.0 24.8 0:47.36 cma
4404 root 18 0 2080m 977m 4708 S 0.0 24.8 0:00.07 cma
4405 root 18 0 2080m 977m 4708 S 0.0 24.8 0:00.04 cma
4406 root 15 0 2080m 977m 4708 S 0.0 24.8 0:12.14 cma
4408 root 19 0 2080m 977m 4708 S 0.0 24.8 0:00.00 cma
I found this forum post from last year and apparently these processes have to do with McAfee virus scanning.
I ran pmap on one of the processes and this is the last line of output:
mapped: 2130892K writeable/private: 2113632K shared: 40K
Is this process really using 2.1GB of memory? Is Top reporting the memory usage accurately>
Thanks!
The VIRT column tells you the total size of the virtual memory segments mapped into the process - this includes the executable itself, libraries, data segments, stack, heap, memory mapped files, etc. In a sense, it is the total amount of memory that the process currently has permission to touch in one way or another (read, write, execute). The process is not necessarily using all of that, which is one of several reasons that the RES column reports a smaller number. RES is the total size of the subset of the VIRT size that is actually currently in physical memory at the moment. It is a better (but still not great) measure of how much memory the process is actually using - the fact that it is in memory indicates that it has been or is currently being actively used. However, if your system has lots of memory, a portion of that RES number may have been touched 3 days ago, and not since, so it may not be actively in use. Conversely, if you are short on memory, the process may be trying to actively use more than RES currently indicates, which will result in paging/swapping activity and performance issues.
Then there's the tendency for some types of memory (executables, libraries) to be shared between multiple instances of a program, the existence of IPC-type shared memory, and several other things that all factor into "how much memory is this process using?"...
In other words, it's not as simple a question as you might imagine...
My app is running slow after a few days. Using the unix "top" command it seems there is not a lot of free memory. See below. Even if I stop the application about the same memory shows used. Any ideas why? Does this amount of memory look normal with no app running on a small gear application? How can I reboot the virtual machine?
Below is the output of the "top" command with no app running. Shows
7513700k total, 7327484k used, 186216k free
top - 22:06:26 up 14 days, 5:42, 0 users, load average: 1.83, 2.82, 3.21
Tasks: 3 total, 1 running, 2 sleeping, 0 stopped, 0 zombie
Cpu(s): 10.2%us, 26.6%sy, 1.6%ni, 57.4%id, 4.0%wa, 0.0%hi, 0.0%si, 0.2%st
Mem: 7513700k total, 7327484k used, 186216k free, 170244k buffers
Swap: 6249464k total, 4210036k used, 2039428k free, 925320k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
48736 3558 20 0 14908 1176 944 R 0.7 0.0 0:00.04 top
48374 3558 20 0 102m 2684 848 S 0.0 0.0 0:00.00 sshd
48383 3558 20 0 106m 2072 1436 S 0.0 0.0 0:00.19 bash
What type of app are you running? Also, since openshift uses cgroups you'll want to see what your usage is within your cgroup (top output shows the whole system). Try including the output from for i in $(oo-cgroup-read all);do echo “oo-cgroup-read $i” && oo-cgroup-read $i; done and pay close attention to your memory limits.
I am running a script that loads big files. I ran the same script in a single core OpenSuSe server and quad core PC. As expected in my PC it is much more faster than in the server. But, the script slows down the server and makes it impossible to do anything else.
My script is
for 100 iterations
Load saved data (about 10 mb)
time myscript (in PC)
real 0m52.564s
user 0m51.768s
sys 0m0.524s
time myscript (in server)
real 32m32.810s
user 4m37.677s
sys 12m51.524s
I wonder why "sys" is so high when i run the code in server. I used top command to check the memory and cpu usage.
It seems there is still free memory, so swapping is not the reason. % sy is so high, its probably the reason for the speed of server but I dont know what is causing % sy so high. The process that is using highest percent of CPU (99%) is "myscript". %wa is zero in the screenshot but sometimes it gets very high (50 %).
When the script is running, load average is greater than 1 but have never seen to be as high as 2.
I also checked my disc:
strt:~ # hdparm -tT /dev/sda
/dev/sda:
Timing cached reads: 16480 MB in 2.00 seconds = 8247.94 MB/sec
Timing buffered disk reads: 20 MB in 3.44 seconds = 5.81 MB/sec
john#strt:~> df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 245G 102G 131G 44% /
udev 4.0G 152K 4.0G 1% /dev
tmpfs 4.0G 76K 4.0G 1% /dev/shm
I have checked these things but I am still not sure what is the real problem in my server and how to fix it. Can anyone identify a probable reason for the slowness? What could be the solution?
Or is there anything else I should check?
Thanks!
You're getting a high sys activity because the load of the data you're doing takes system calls that happen in kernel. To resolve your slowness problems without upgrading the server might be possible. You can modify scheduling priority. See the man pages for nice and renice. See here and especially:
Niceness values range from -20 (the highest priority, lowest niceness) and 19 (the lowest priority, highest niceness).
$ ps -lp 941
F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD
4 S 0 941 1 0 70 -10 - 1713 poll_s ? 00:00:00 sshd
$ nice -n 19 ./test.sh
My niceness value is 19
$ renice -n 10 -p 941
941 (process ID) old priority -10, new priority 10
In the man pages of top command, it is given that VRT column shows memory consumption in kb(kilo bits). When i am running my application in linux, memory consumed is shown as 157m. Does this 157m mean 157 mega bytes or 157 mega bits? Any clarifiction is appreciated.
It's in MegaBytes. Put the output from your top next to the output from ps aux.
> ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 780 72 ? S Jun26 0:09 init [3]
mysql 28670 2.1 42.1 2733944 1708028 ? Sl Sep24 1910:21 /usr/sbin/mysqld
>top
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
28670 mysql 15 0 2667m 1.6g 4164 S 104 42.2 1910:37 mysqld