On the command line this can be found out using the 'free' utility and 'cat /proc/meminfo'.
What would be the different ways to find out the physical RAM size in Linux programatically from a :
Userspace Application
Kernel Module
What API calls are available ?
#include <unistd.h>
long long physical_mem_bytes = (long long) sysconf (_SC_PHYS_PAGES) * sysconf (_SC_PAGESIZE);
Other than the command line ulimit, I don't know of a way of finding maximum memory for an individual process.
Programmatically, Linux won't tell you the actual physical size. Instead you should read this info from SMBIOS with, e.g.,
sudo dmidecode -t memory | fgrep -ie 'size:'
This will give you results like the following (from a box with 4 RAM banks, only 2 installed):
Maximum Memory Module Size: 16384 MB
Maximum Total Memory Size: 65536 MB
Installed Size: 2048 MB (Single-bank Connection)
Enabled Size: 2048 MB (Single-bank Connection)
Installed Size: Not Installed
Enabled Size: Not Installed
Installed Size: 2048 MB (Single-bank Connection)
Enabled Size: 2048 MB (Single-bank Connection)
Installed Size: Not Installed
Enabled Size: Not Installed
Size: 2048 MB
Size: No Module Installed
Size: 2048 MB
Size: No Module Installed
Add the reported sizes (or Enabled Sizes, but some BIOSes empirically don't report that) to get (in this case) 4096 MB. (Extra points for code that automates the parsing and arithmetic, but you can probably do that in your head nearly as reliably.)
To check your computation, run
fgrep -e 'MemTotal:' /proc/meminfo
The value reported by /proc/meminfo should not be more than the value you compute from dmidecode. In this case, empirically I get
MemTotal: 3988616 kB
cat /proc/meminfo
specifically from memory, I got this result from what Jared said
sudo dmidecode -t memory
there you can read the specs for each individual memory slot, so you will read something like 2048MB, in my case I have 2 of these being 4gb, despite my non PAE kernel only shows about 3.3gb and all other applications wont say the real physical memory, only dmidecode, thx!
Related
We develop a Linux application that uses DPDK and which must also be heavily optimised for speed.
We must specify huge pages for use by DPDK and also for general dynamic memory allocation. For the latter we use the libhugetlbfs library.
sudo mount -t hugetlbfs none /mnt/hugetlbfs
We specify the huge pages in the bootcmd line as follows:
hugepagesz=1G hugepages=20 hugepages=0 default_hugepagesz=1G
We are using Centos 7 and full boot cmdline is:
$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-957.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet hugepagesz=1G hugepages=20 default_hugepagesz=1G irqaffinity=0,1 isolcpus=4-23 nosoftlockup mce=ignore_ce idle=poll
These values are fairly arbitrary. With these values, I see:
$ free -m
total used free shared buff/cache available
Mem: 47797 45041 2468 9 287 2393
Swap: 23999 0 23999
So 2.468GB RAM is free out of 48GB. So a very large amount of memory is allocated to huge pages and I want to reduce this.
My question is what would be sensible values for them?
I am confused by the interpretation of the parameters. I see:
$ cat /proc/meminfo
<snip>
HugePages_Total: 43
HugePages_Free: 43
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 1048576 kB
and also:
$ cat /sys/kernel/mm/hugepages/hugepages-1048576kB/free_hugepages
43
Why are 43 pages reported when my parameters only specify 20 pages of 1G?
I would like some guidelines on:
huge page size/quantity that I might need for DPDK?
huge page size/quantity that I might need for malloc?
I know these are highly application dependant but some guidelines would be helpful. Also how could I detect if the huge pages were insufficent for the application?
Additional info:
$ cat /proc/mounts | grep huge
cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime 0 0
Update 4 March:
My boot cmdline is now:
$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-957.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet hugepagesz=1G hugepages=20 default_hugepagesz=1G irqaffinity=0,1 isolcpus=4-23 nosoftlockup mce=ignore_ce idle=poll transparent_hugepage=never
and transparent hugepages are disabled (I activated a custom tune profile):
$ cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
but I still see 43 hugepages:
$ cat /sys/kernel/mm/hugepages/hugepages-1048576kB/free_hugepages
43
whereas I have only specified 20 in cmdline. Why is this?
I start mpirun with command:
mpirun -np 2 prog
and get next output:
--------------------------------------------------------------------------
WARNING: It appears that your OpenFabrics subsystem is configured to only
allow registering part of your physical memory. This can cause MPI jobs to
run with erratic performance, hang, and/or crash.
This may be caused by your OpenFabrics vendor limiting the amount of
physical memory that can be registered. You should investigate the
relevant Linux kernel module parameters that control how much physical
memory can be registered, and increase them to allow registering all
physical memory on your machine.
See this Open MPI FAQ item for more information on these Linux kernel module
parameters:
http://www.open-mpi.org/faq/?category=openfabrics#ib-..
Local host: node107
Registerable memory: 32768 MiB
Total memory: 65459 MiB
Your MPI job will continue, but may be behave poorly and/or hang.
--------------------------------------------------------------------------
hello from 0
hello from 1
[node107:48993] 1 more process has sent help message help-mpi- btl-openib.txt / reg mem limit low
[node107:48993] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
Other installed soft (Intel MPI library) work fine, without any errors and using all 64GB memory.
For OpenMPI I don't use any PBS manager (Torque, slurm, etc.), I work on single node. I get to the node by command
ssh node107
For command
cat /etc/security/limits.conf
I get next output:
...
* soft rss 2000000
* soft stack 2000000
* hard stack unlimited
* soft data unlimited
* hard data unlimited
* soft memlock unlimited
* hard memlock unlimited
* soft nproc 10000
* hard nproc 10000
* soft nofile 10000
* hard nofile 10000
* hard cpu unlimited
* soft cpu unlimited
...
For command
cat /sys/module/mlx4_core/parameters/log_num_mtt
I get output:
0
Command:
cat /sys/module/mlx4_core/parameters/log_mtts_per_seg
output:
3
Command:
getconf PAGESIZE
output:
4096
With this params and by formula
max_reg_mem = (2^log_num_mtt) * (2^log_mtts_per_seg) * PAGE_SIZE
max_reg_mem = 32768 bytes, nor 32GB, how specified in openmpi warning.
What is the reason for this? Can openmpi don't use Mellanox and params log_num_mtt, log_mtts_per_seg? How I can configure OpenFabrics to use all 64GB memory?
I solve this problem by installing newest version of OpenMPI (2.0.2).
In /etc/modprobe.d/mlx4_core.conf, put the following module parameter:
options mlx4_core log_mtts_per_seg=5
Reload the mlx4_core module:
rmmod mlx4_ib;
rmmod mlx4_core;
modprobe mlx4_ib
Check if log_mtts_per_seg shows up as configured above:
cat /sys/module/mlx4_core/parameters/log_mtts_per_seg
Within a bash script i need to get the total disk size and the currently used size of the complete disk.
I know i can get the total disk size without needed to be root with this command:
cat /sys/block/sda/size
This command will output the count of blocks on device SDA.
Multiply it with 512 and you'll get the amount of bytes on this device.
This is sufficient with the total disk size.
Now for the currently used space. I want to get this value without being root.
I can assume the device name is SDA.
Now there is this command: df
I thought i could use this command but it seems this command only outputs data of the currently mounted partitions.
Is there a way to get a total of space used on disk SDA without needing to be root and not all partitions needs to be mounted?
Let's assume the following example:
/dev/sda1 80GB Linux partition 20GB Used
/dev/sda2 80GB Linux Home partition 20GB Used
/dev/sda3 100GB Windows Parition. 30GB Used
Let's assume my SDA disk is partitioned like above. But while i'm on Linux my Windows partition (sda3) is not mounted.
The output of df will give me a grand total of 40 GB Used so it doesn't take sda3 in account.
Again the question:
Is there a way without root to get me a grand total of 70 GB used space?
I think stat command should help. You can get the get the partitions from /proc/partitions.
Sample stat command output:
$ stat -f /dev/sda1
File: "/dev/sda1"
ID: 0 Namelen: 255 Type: tmpfs
Block size: 4096 Fundamental block size: 4096
Blocks: Total: 237009 Free: 236970 Available: 236970
Inodes: Total: 237009 Free: 236386
You can use df.
df -h --output='used' /home
Used
3.2G
If you combine this with some sed or awk you can have the value you seek
I have an application that uses hugepage and the application suddenly crashed due to some bug.
After crashing, since the application does not release the hugepage properly, the free hugepage number is not increased in sys filesystem.
$ sudo cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
0
$ sudo cat /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
1024
Is there a way to release the hugepages by force?
Sometimes need to check all directory that hugetlbfs has been mounted.
So,
find mounted directory by command mount | grep huge.
check every directory except especially /dev/hugepages.
delete all 2M-sized files. (2M is the size of hugepage)
Use ipcs -m to list the shared memory segments.
Use ipcrm to remove the left over shared memory segments.
Edit on 06/24/2019:
Ok, so, the above answer, while correct as far as it goes, was a bit brief. In particular, if you have a host with multiple DB instances, and only one is crashed how can you determine which (if any) memory segments should be cleaned up?
Well, this too, can be done. For each running instance, connect w/ / as sysdba, then do oradebug setmypid (any pid will do, as all Oracle PIDs connect to the SGA). Then do oradebug ipc. That will (hopefully) return IPC information written to the trace file. So, go to the udump (or diag_dest) directory, and look for your trace file. It will contain all the IPC information for the instance. This will include ShmId. Look through the file for the ShmId(s) that this instance is using. Now look at the output of ipcs -m.
When you have done that for all the running instances, any memory segment output by ipcs -m that shows non-zero memory allocation, and that you cannot account for in the oradebug ipc information from any running instance, must be the left over memory segments from the crashed instance. Use ipcrm to remove it/them.
When doing this on a host with multiple running instances, this can be a bit fraught. Please proceed with caution. You don't want to remove the SGA of a running instance!
Hope that helps....
HugeTLB can either be used for shared memory (and Mark J. Bobak's answer would deal with that) or the app mmaps files created in a hugetlb filesystem. If the app crashes without removing those files they survive and keep corresponding memory 'allocated'.
Check hugeTLB filesystem and see if there are any leftover files from the app. Removing them would release the memory.
If you follow the instruction below, you can get rid of the allocated hugepages:
1) Let's check the hugepages which were free at restart
dpdk#dpdkvm:~$ ls /mnt/huge/
empty
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ cat /proc/meminfo
...
HugePages_Total: 256
HugePages_Free: 256
...
2) Starting a dpdk application with wrong parameters, producing an error
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ sudo ./build/kni -c 0x03 -n 2 -- -P -p 0x03 --config="(0,0,1),(1,0,1)"
...
EAL: Error - exiting with code: 1
Cause: No supported Ethernet device found
3) When I check hugepages, there is not any free
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ cat /proc/meminfo
...
HugePages_Total: 256
HugePages_Free: 0
...
4) Now, when I check the mounted hugepage directory, I can see the files which are not given back to OS by dpdk application.
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ ls /mnt/huge/
...
rtemap_0 rtemap_137 rtemap_176 rtemap_214 rtemap_253 rtemap_62
...
5) Finally, if you remove the files starting with rtemap, you can give the hugepages back
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ sudo rm /mnt/huge/*
[sudo] password for dpdk:
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ cat /proc/meminfo
...
HugePages_Total: 256
HugePages_Free: 256
...
your hugetlb may be used by shared memory or mmap files.
try to remove the shared memories or umount the hugetlb fs
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 9 years ago.
Improve this question
Please do not waste anymore of your time on this question...I ended up deleting the whole VM and creating another. The time it took me to do this is less than the time it would take to fix the issue. I have couple of SSDs in RAID mode.
Thank you for all those who tried to troubleshoot the issue!
I am having this problem with ubnuntu not showing active swap spaces when I run the command cat /proc/swaps. Here is a list of commands I ran. I even added a new swap space (file: /swapfile1) just to make sure that at least one swap space, but still I get nothing.
hebbo#ubuntu-12-lts:~$ sudo fdisk -l
[sudo] password for hebbo:
Disk /dev/sda: 26.8 GB, 26843545600 bytes
255 heads, 63 sectors/track, 3263 cylinders, total 52428800 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000e3a7a
Device Boot Start End Blocks Id System
/dev/sda1 * 46569472 52426751 2928640 82 Linux swap / Solaris
/dev/sda2 2046 46567423 23282689 5 Extended
/dev/sda5 2048 46567423 23282688 83 Linux
Partition table entries are not in disk order
hebbo#ubuntu-12-lts:~$ sudo su
root#ubuntu-12-lts:/home/hebbo# cat /proc/swaps
Filename Type Size Used Priority
root#ubuntu-12-lts:/home/hebbo# dd if=/dev/zero of=/swapfile1 bs=1024 count=524288
524288+0 records in
524288+0 records out
536870912 bytes (537 MB) copied, 1.18755 s, 452 MB/s
root#ubuntu-12-lts:/home/hebbo# mkswap /swapfile1
Setting up swapspace version 1, size = 524284 KiB
no label, UUID=cb846612-5f27-428f-9f83-bbe24b410a78
root#ubuntu-12-lts:/home/hebbo# chown root:root /swapfile1
root#ubuntu-12-lts:/home/hebbo# chmod 0600 /swapfile1
root#ubuntu-12-lts:/home/hebbo# swapon /swapfile1
root#ubuntu-12-lts:/home/hebbo# cat /proc/swaps
Filename Type Size Used Priority
root#ubuntu-12-lts:/home/hebbo#
Any idea how to fix this?
This is ubuntu 12.04 LTS running kernel 3.9.0 in a vmware VM.
Thanks in advance!
To activate /swapfile1 after Linux system reboot, add entry to /etc/fstab file. Open this file using a text editor such as vi:
# vi /etc/fstab
Add the following line:
/swapfile1 swap swap defaults 0 0
Save and close the file. Next time Linux comes up after reboot, it enables the new swap file for you automatically.
Have a look here for more info.
I just tried it and it works on my box.
Linux fileserver 3.8.0-32-generic #47~precise1-Ubuntu SMP Wed Oct 2 16:19:35 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
ortang#fileserver:~$ cat /proc/swaps
Filename Type Size Used Priority
/dev/dm-2 partition 4194300 0 -1
ortang#fileserver:~$ sudo su
root#fileserver:/home/ortang# dd if=/dev/zero of=/swapfile bs=1M count=512
512+0 records in
512+0 records out
536870912 bytes (537 MB) copied, 0.695721 s, 772 MB/s
root#fileserver:/home/ortang# chmod 600 /swapfile
root#fileserver:/home/ortang# mkswap /swapfile
Setting up swapspace version 1, size = 524284 KiB
no label, UUID=63cdcf3d-ba03-42ce-b598-15b6aa3ca67d
root#fileserver:/home/ortang# swapon /swapfile
root#fileserver:/home/ortang# cat /proc/swaps
Filename Type Size Used Priority
/dev/dm-2 partition 4194300 0 -1
/swapfile file 524284 0 -2
One thing i can imagine why it is working on my box, is that i already have a working swap partition, and it seems you don't.
It could also be caused by the kernel you use, 3.9.0 is not the regular 12.04.3 LTS kernel? Have you built the kernel yourself?
Whats the output of
grep CONFIG_SWAP /boot/config-`uname -r`
or
zcat /proc/config.gz | grep CONFIG_SWAP
is swap enabled in your kernel?
I ended up deleting the whole VM and creating another. The time it took me to do this is less than the time it would take to fix the issue. I have couple of SSDs in RAID mode. And I already had all the downloads on the same host machine. All in all ~7 minutes.
Thanks for all those who helped troubleshoot the issue.