I have a question about what COULD be the possible impact to a system, if we have the following kernel shmmax and shmall kernel settings:
# Controls the maximum shared segment size, in bytes
kernel.shmmax = 135456844800
# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 107374182400
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
# free -mt
total used free shared buffers cached
Mem: 258065 230766 27299 11935 1481 58723
-/+ buffers/cache: 170561 87504
Swap: 16383 52 16331
Total: 274449 230818 43631
On that specific server we run only Oracle DB, are the settings correct and if not what could be the impact? Thanks a lot in advance!
Related
When I use the dumpe2fs command to look at the Block Group of the ext4 filesystem, I see "free inodes" and "unused inodes".
I want to know the difference between them ?
Why do they have different values in Group 0 ?
Group 0: (Blocks 0-32767) [ITABLE_ZEROED]
Checksum 0xd1a1, unused inodes 0
Primary superblock at 0, Group descriptors at 1-3
Reserved GDT blocks at 4-350
Block bitmap at 351 (+351), Inode bitmap at 367 (+367)
Inode table at 383-892 (+383)
12 free blocks, 1 free inodes, 1088 directories
Free blocks: 9564, 12379-12380, 12401-12408, 12411
Free inodes: 168
Group 1: (Blocks 32768-65535) [ITABLE_ZEROED]
Checksum 0x0432, unused inodes 0
Backup superblock at 32768, Group descriptors at 32769-32771
Reserved GDT blocks at 32772-33118
Block bitmap at 352 (+4294934880), Inode bitmap at 368 (+4294934896)
Inode table at 893-1402 (+4294935421)
30 free blocks, 0 free inodes, 420 directories
Free blocks: 37379-37384, 37386-37397, 42822-42823, 42856-42859, 42954-42955, 44946-44947, 45014-45015
Free inodes:
The "unused inodes" reported are inodes at the end of the inode table for each group that have never been used in the lifetime of the filesystem, so e2fsck does not need to scan them during repair. This can speed up e2fsck pass-1 scanning significantly.
The "free inodes" are the current unallocated inodes in the group. This number includes the "unused inodes" number, so that they will still be used if there are many (typically very small) inodes allocated in a single group.
From:
https://unix.stackexchange.com/a/715165/536354
According to this documentation, CPU shares is calculated as
process_cpu.shares = min( 1024*(application_memory / 8 GB), 1024)
According to this formula, if an application is assigned 1GB memory, then it should get 128 CPU shares. 1024*(1/8). However, if we SSH into the application and check the cpu.shares we get 122
cat /sys/fs/cgroup/cpu/cpu.shares
122
Here are the observations:
Memory
Calculated cpu.share
Observed cpu.share
Difference
1GB
128
122
~5% difference
1.5GB/1536MB
192
184
~5% difference
2GB
256
256
3GB
384
384
4GB
512
512
5GB
640
634
~1% difference
5.5GB/5632MB
704
696
~2% difference
8GB
1024
1024
Why is it that for some values there is this discrepancy (Such as 1G,1.5G,5G etc..) while others (like 2,3,4 - 6,7,8) are consistent with the calculation. I believe that I am missing something from Cgroups perspective for this calculation. Is this specific to CF or is it something to do with the way Linux calculates resource allocation in Cgroups in general? Is there a headroom always reserved?
I have a server which has 16 GB memory.
Now I need to set my shmmax and shmall, because the server default is (checked with ipcs -l)
------ Messages Limits --------
max queues system wide = 32000
max size of message (bytes) = 8192
default max size of queue (bytes) = 16384
------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 18014398509465599
max total shared memory (kbytes) = 18014398509465599
min seg size (bytes) = 1
------ Semaphore Limits --------
max number of arrays = 32000
max semaphores per array = 32000
max semaphores system wide = 1024000000
max ops per semop call = 500
semaphore max value = 32767
It seems terrible, shmall and shmmax is bigger than my 16 GB.
So I want to change the setting to
shmmax -> 16GB/4
shmall -> 16GB/2
But I can't be sure what unit I have set
shmmax --> 4420960256
shmall --> 8620960256
But is the unit for my number? byte or KB?
Because ipcs -l is showing KB....
echo "kernel.shmmax=4420960256" >> /etc/sysctl.conf
echo 4420960256> /proc/sys/kernel/shmmax
echo "kernel.shmall=8620960256" >> /etc/sysctl.conf
echo 8620960256> /proc/sys/kernel/shmall
thanks for help, but the postgresql just crash and get killed by yesterday, it shows :
This error usually means that PostgreSQL's request for a shared memory segment exceeded available memory, swap space, or huge pages. To reduce the request size (currently 4420960256 bytes), reduce PostgreSQL's shared memory usage, perhaps by reducing shared_buffers or max_connections.
my setting =>
shared_buffers = 4GB
effective_cache_size = 12GB
Just use:
lsipc
On my Ubuntu 16.04 LTS I get:
RESOURCE DESCRIPTION LIMIT USED USE%
MSGMNI Number of message queues 32000 0 0.00%
MSGMAX Max size of message (bytes) 8192 - -
MSGMNB Default max size of queue (bytes) 16384 - -
SHMMNI Shared memory segments 4096 20 0.49%
SHMALL Shared memory pages 2097152 4915 0.23%
SHMMAX Max size of shared memory segment (bytes) 4294967296 - -
SHMMIN Min size of shared memory segment (bytes) 1 - -
SEMMNI Number of semaphore identifiers 128 0 0.00%
SEMMNS Total number of semaphores 32000 0 0.00%
SEMMSL Max semaphores per semaphore set. 250 - -
SEMOPM Max number of operations per semop(2) 100 - -
SEMVMX Semaphore max value 32767 - -
which clearly states the measure unit for the values I have specified in /etc/sysctl.conf. So for me SHMMAX is in bytes while SHMALL in pages (see getconf PAGE_SIZE).
Just leave the setting the way it is – essentially, that means “unlimited” in your case. One less limit you could bang your head against!
The amount of shared memory allocated by PostgreSQL is fixed and mostly determined by shared_buffers. Just make sure you don't set that to exceed your RAM (4GB would be perfect), and there is no danger whatsoever.
For the record: experimentation on my system shows that the unit of kernel.shmmax is bytes, while the unit of kernel.shmall is memory pages (check getconf PAGESIZE).
Why are file sizes all different?
In Windows 10 I can see all of these sizes:
11,116 KB
10.8 MB
11,382,240 Bytes
11,382,784 Bytes
If I use the Console Window:
D:\My Programs\2017\MeetSchedAssist\Inno\Output>dir *.exe
Volume in drive D is DATA
Volume Serial Number is A8B0-A5C6
Directory of D:\My Programs\2017\MeetSchedAssist\Inno\Output
03/04/2018 08:50 11,382,240 MeetSchedAssistSetup.exe
1 File(s) 11,382,240 bytes
0 Dir(s) 719,837,487,104 bytes free
D:\My Programs\2017\MeetSchedAssist\Inno\Output>
I understand that perhaps on the physical media it has to round it to physically take a certain amount of space, but that line above:
Size: 10.8 MB (11,382,240 bytes)
Huh? Why does it not say 11.38 MB?
Once upon a time it has been defined that
1 kB = 1024 B
1 MB = 1024 kB
If you divide your bytes figure all the way down to MB, you'll get all those figures.
Now that they noticed that many people tend to walk into that trap, they have redefined the unit multiples and defined new ones
1 kiB = 1024 B
1 MiB = 1024 kiB
1 kB = 1000 B
1 MB = 1000 kB
but this scheme is not so widespread (seems to be more common with total size specs of storage media).
Funny sidenote: I guess I am not the only one who has learned it the old way and now mixes it up with the current definition all the time. I'd say problems like this are the root cause for humanity being mostly conservatively oriented.
My embedded system runs Linux 3.10.14.
While running, my application prints out this message.
ERR: Memory overflow! free bytes=56000, bytes used=4040000, bytes to allocate=84000
But when I do "free", it seems I have enough free memory.
/ # free
total used free shared buffers
Mem: 27652 20788 6864 0 0
-/+ buffers: 20788 6864
Swap: 0 0 0
Any possible root cause of the error message?
Or how can I use free memory to the last 1 byte?
Please comment if I am missing any information.
Thank you!
According to the output of "free", we can see totally there are 27652 bytes, 20788 bytes are used and 6864 bytes are free.
The print from your application, it seems to try to allocate 84000 bytes, but there are only 56000 bytes available。
So there is a question that how much memory do your system have? 27652 bytes or
4096000 bytes？
The printout is got from the system ？