I know there are page cache and buffer cache in Linux. Page cache is for file system, and buffer cache is for device.
To free pagecache:
echo 1 > /proc/sys/vm/drop_caches
To free dentries and inodes:
echo 2 > /proc/sys/vm/drop_caches
To free pagecache, dentries and inodes:
echo 3 > /proc/sys/vm/drop_caches
My question:
Is buffer cache covered in drop_caches?
If not, then is there a method to only free buffer caches, leaving the page cache still cached?
If there isn't, why it's unnecessary or impossible?
Ans:-
sync; echo 3 > /proc/sys/vm/drop_caches
Here’s the output of free before and after running the above command.
Before
# free -m
total used free shared buffers cached
Mem: 502 487 15 0 15 170
-/+ buffers/cache: 301 201
Swap: 3153 375 2777
After
# free -m
total used free shared buffers cached
Mem: 502 345 157 0 1 44
-/+ buffers/cache: 300 202
As you can see the highlighted numbers, the buffers and cached values have dropped significantly after I ran the command. Again, this is not necessary. In fact, it’s not recommended at all but there it is if you’re curious to know.
Related
my RAM got filled completely, and the buff/cache occupied all the memory pl z do let me know how to make it clear when I run the command free -m I get this output,
this is a Linux VM made on azure for WordPress bitnami
$ free -m
total used free shared buff/cache available
Mem: 64320 19319 3164 115 41837 44168
Swap: 0
image
I tried to reproduce the same in my environment I got some buff/cache like below.
To clear the buff/cache try to use the below command.
Every Linux has three options to clear cache
First - clear page cache only
echo 1 > /proc/sys/vm/drop_caches
Second - clear dentries and inodes
echo 2 > /proc/sys/vm/drop_caches
Third - clear page cache and dentries and inodes
echo 3 > /proc/sys/vm/drop_caches
When I try to run this above script, there is 912M of total RAM, and 144M as used. And before I have 352M as free after exciting the script my buff/cache were reduced from 415 to 58 and free Ram increased upto 708M
Reference:
Clear RAM Memory Cache, Buffer, and Swap Space on Linux Credits by Arpit-saini
We develop a Linux application that uses DPDK and which must also be heavily optimised for speed.
We must specify huge pages for use by DPDK and also for general dynamic memory allocation. For the latter we use the libhugetlbfs library.
sudo mount -t hugetlbfs none /mnt/hugetlbfs
We specify the huge pages in the bootcmd line as follows:
hugepagesz=1G hugepages=20 hugepages=0 default_hugepagesz=1G
We are using Centos 7 and full boot cmdline is:
$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-957.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet hugepagesz=1G hugepages=20 default_hugepagesz=1G irqaffinity=0,1 isolcpus=4-23 nosoftlockup mce=ignore_ce idle=poll
These values are fairly arbitrary. With these values, I see:
$ free -m
total used free shared buff/cache available
Mem: 47797 45041 2468 9 287 2393
Swap: 23999 0 23999
So 2.468GB RAM is free out of 48GB. So a very large amount of memory is allocated to huge pages and I want to reduce this.
My question is what would be sensible values for them?
I am confused by the interpretation of the parameters. I see:
$ cat /proc/meminfo
<snip>
HugePages_Total: 43
HugePages_Free: 43
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 1048576 kB
and also:
$ cat /sys/kernel/mm/hugepages/hugepages-1048576kB/free_hugepages
43
Why are 43 pages reported when my parameters only specify 20 pages of 1G?
I would like some guidelines on:
huge page size/quantity that I might need for DPDK?
huge page size/quantity that I might need for malloc?
I know these are highly application dependant but some guidelines would be helpful. Also how could I detect if the huge pages were insufficent for the application?
Additional info:
$ cat /proc/mounts | grep huge
cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime 0 0
Update 4 March:
My boot cmdline is now:
$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-957.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet hugepagesz=1G hugepages=20 default_hugepagesz=1G irqaffinity=0,1 isolcpus=4-23 nosoftlockup mce=ignore_ce idle=poll transparent_hugepage=never
and transparent hugepages are disabled (I activated a custom tune profile):
$ cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
but I still see 43 hugepages:
$ cat /sys/kernel/mm/hugepages/hugepages-1048576kB/free_hugepages
43
whereas I have only specified 20 in cmdline. Why is this?
With the command free -g, I am able to get the total occupied size and free size of RAM in Linux. But want to understand which tasks or process taking more size, so that I can free up the RAM size.
total used free shared buffers cached
Mem: 125 121 4 0 6 94
-/+ buffers/cache: 20 105
Swap: 31 0 31
Go for top command
then press shift+f
press a for pid information
ALso check
ps -eo pmem,vsz,pid
man ps
checkout pmem,vsz,pid.......
hope it helps..
thanks for the question !
You can use below command to find running processes sorted by memory use:
ps -eo pmem,pcpu,rss,vsize,args | sort -k 1 -r | less
I'm trying to run a simulation on my local computer in university, but after some iterations it's being killed by a SIGKILL. Even when I check the available swap space it shows that still I have enough space !!!
:~$ free -m
total used free shared buffers cached
Mem: 3937 2091 1845 0 64 677
-/+ buffers/cache: 1349 2587
Swap: 3860 738 3122
The same story repeats when I use another server by ssh
:~$ free -m
total used free shared buffers cached
Mem: 129043 98281 30761 52 4 32901
-/+ buffers/cache: 65375 63668
Swap: 4095 120 3975
When I run it on my own laptop it works properly.
I'd really appreciate if help me out.
Are you checking the swap space after the fact or during the run? If there is a memory crunch the operating system's out of memory killer (OOM Killer) may kill the process ( depending on the configuration this could be the worst offender, random or anything else). Execute "sar" command and see the system state around the time your process got killed.
I'm running Ubuntu Trusty 14.04 on a new machine with 8GB of RAM, and it seems to be locking up periodically and nothing is in syslog file. I've installed Nagios and have been watching the graphs, and it looks like memory is going high from 7% to 72% in just a span of 10 mins. Only node process are running on server. In top I found all process are running very normal memory consumption. Even after stopping node process. Memory remains with same utilization.
free agrees, claiming I'm using more than 5.7G of memory:
free -h
total used free shared buffers cached
Mem: 7.8G 6.5G 1.3G 2.2M 233M 612M
-/+ buffers/cache: 5.7G 2.1G
Swap: 2.0G 0B 2.0G
This other formula for totaling the memory roughly agrees:
# ps -e -orss=,args= | sort -b -k1,1n | awk '{total = total + $1}END{print total}'
503612
If the processes only total 500 MiB, where's the rest of the memory going?
I've got solution on this... so just wanna to update the same...
echo 2 > /proc/sys/vm/drop_caches
This resolved my issue. So I have added the same in my cron for every 5 mins on each of ubuntu server