Linux memory allocation - order changed by 1 - linux

I will try to describe the issue as much as I can. Though, I won't be able to post all the relevant code.
The case is as follow,
I made few changes in the code, all of them were in user-space. I didn't change anything in the kernel code.
After compiling and working for a while with this release I suddenly noticed that the Ephemeral ports range has been changed.
After investigating I encountered that this was caused because of a change in the order of magnitude (in memory allocation). But as I said before, no-one touched this code.
Here is some of the log messages of the linux bootup from before and after this change. You can notice that change in the order I mentioned before.
After the change:
[000000.000] Determined physical RAM map:
[000000.000] memory: 0000000007000000 # 0000000000c10000 (usable)
[000000.015] reserve bootmem for memoops 0000000000020000 # 0000000007bf0000
[000000.019] Primary instruction cache 32kB, virtually tagged, 4 way, 64 sets, linesize 128 bytes.
[000000.019] Primary data cache 16kB, 64-way, 2 sets, linesize 128 bytes.
[000000.020] PID hash table entries: 512 (order: 9, 16384 bytes)
[000000.020] Using 500.000 MHz high precision timer.
[000000.227] Dentry cache hash table entries: 16384 (order: 5, 131072 bytes)
[000000.240] Inode-cache hash table entries: 8192 (order: 4, 65536 bytes)
[000000.266] Memory: 112408k/114688k available (2062k kernel code, 2136k reserved, 533k data, 200k init, 0k highmem)
Before the change:
[000000.000] Determined physical RAM map:
[000000.000] memory: 0000000007400000 # 0000000000c00000 (usable)
[000000.016] reserve bootmem for memoops 0000000000020000 # 0000000007fe0000
[000000.020] Primary instruction cache 32kB, virtually tagged, 4 way, 64 sets, linesize 128 bytes.
[000000.020] Primary data cache 16kB, 64-way, 2 sets, linesize 128 bytes.
[000000.020] PID hash table entries: 1024 (order: 10, 32768 bytes)
[000000.228] Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
[000000.242] Inode-cache hash table entries: 16384 (order: 5, 131072 bytes)
[000000.269] Memory: 116280k/118784k available (2062k kernel code, 2344k reserved, 533k data, 200k init, 0k highmem)
Note: I already tried to take off my changes and recompile, but the issue is still there for some reason.
Maybe someone know what might affect this? How can this happen?

The application codes will not bring any influence to kernel's booting, because it begin to run after kernel booting.
This issue should be caused by your physical memory change, you can verified via kernel source codes:
mm/page_alloc.c
/* round applicable memory size up to nearest megabyte */
numentries = nr_kernel_pages;
numentries += (1UL << (20 - PAGE_SHIFT)) - 1;
numentries >>= 20 - PAGE_SHIFT;
numentries <<= 20 - PAGE_SHIFT;
....
log2qty = ilog2(numentries);
....
printk(KERN_INFO "%s hash table entries: %ld (order: %d, %lu bytes)\n",
tablename,
(1UL << log2qty),
ilog2(size) - PAGE_SHIFT,
size);
If the boot parameter keep the same, you can check:
if hardware keep the same
did you boot via different method, such as change from legacy to UEFI mode
if you have multi target with the same config(mem size, cpu, chipset etc.), you can verified on another board to avoid the hardware problem on this board.

Related

Linux shmmax and shmall - how to set correct unit?

I have a server which has 16 GB memory.
Now I need to set my shmmax and shmall, because the server default is (checked with ipcs -l)
------ Messages Limits --------
max queues system wide = 32000
max size of message (bytes) = 8192
default max size of queue (bytes) = 16384
------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 18014398509465599
max total shared memory (kbytes) = 18014398509465599
min seg size (bytes) = 1
------ Semaphore Limits --------
max number of arrays = 32000
max semaphores per array = 32000
max semaphores system wide = 1024000000
max ops per semop call = 500
semaphore max value = 32767
It seems terrible, shmall and shmmax is bigger than my 16 GB.
So I want to change the setting to
shmmax -> 16GB/4
shmall -> 16GB/2
But I can't be sure what unit I have set
shmmax --> 4420960256
shmall --> 8620960256
But is the unit for my number? byte or KB?
Because ipcs -l is showing KB....
echo "kernel.shmmax=4420960256" >> /etc/sysctl.conf
echo 4420960256> /proc/sys/kernel/shmmax
echo "kernel.shmall=8620960256" >> /etc/sysctl.conf
echo 8620960256> /proc/sys/kernel/shmall
thanks for help, but the postgresql just crash and get killed by yesterday, it shows :
This error usually means that PostgreSQL's request for a shared memory segment exceeded available memory, swap space, or huge pages. To reduce the request size (currently 4420960256 bytes), reduce PostgreSQL's shared memory usage, perhaps by reducing shared_buffers or max_connections.
my setting =>
shared_buffers = 4GB
effective_cache_size = 12GB
Just use:
lsipc
On my Ubuntu 16.04 LTS I get:
RESOURCE DESCRIPTION LIMIT USED USE%
MSGMNI Number of message queues 32000 0 0.00%
MSGMAX Max size of message (bytes) 8192 - -
MSGMNB Default max size of queue (bytes) 16384 - -
SHMMNI Shared memory segments 4096 20 0.49%
SHMALL Shared memory pages 2097152 4915 0.23%
SHMMAX Max size of shared memory segment (bytes) 4294967296 - -
SHMMIN Min size of shared memory segment (bytes) 1 - -
SEMMNI Number of semaphore identifiers 128 0 0.00%
SEMMNS Total number of semaphores 32000 0 0.00%
SEMMSL Max semaphores per semaphore set. 250 - -
SEMOPM Max number of operations per semop(2) 100 - -
SEMVMX Semaphore max value 32767 - -
which clearly states the measure unit for the values I have specified in /etc/sysctl.conf. So for me SHMMAX is in bytes while SHMALL in pages (see getconf PAGE_SIZE).
Just leave the setting the way it is – essentially, that means “unlimited” in your case. One less limit you could bang your head against!
The amount of shared memory allocated by PostgreSQL is fixed and mostly determined by shared_buffers. Just make sure you don't set that to exceed your RAM (4GB would be perfect), and there is no danger whatsoever.
For the record: experimentation on my system shows that the unit of kernel.shmmax is bytes, while the unit of kernel.shmall is memory pages (check getconf PAGESIZE).

Which bytes of the file remain mapped?

A file is mapped with the the system call:
mmap(65536, 32768, READ, FLAGS, fd, 0)
Pages are 8 KB, so 4 pages worth of the file were mapped (32768/8k = 4 pages). Then the following call is carried out:
munmap(65536, 8192)
Which removes the specified part of the memory map. Which bytes of the file remain mapped? The answer key says that pages 2 and 3 remain, so only bytes 16384 through 32767 remain; however, I'm not sure this is right. Since the len argument (second arg) of the munmap command is 8192 byes, shouldn't only page 0 be removed, leaving bytes 8192 through 32767?
Both you and the answer key are wrong, but in different ways.
Memory pages on most systems are 4 KB (4096 bytes), not 8 KB. I have never heard of a system with 8 KB memory pages.
This makes the entire mapping of 32768 bytes come out to 8 pages. Unmapping the first 8192 bytes (2 pages) would leave the remaining 6 pages (2 through 8) in place.

Who could specifically explain the remaining 2 bytes in first sector of disk?

We know the size of sector in disk is 512 bytes. We also know the first sector mainly record MBR and partition table. The size of MBR is 446 bytes and the size of partition table is 64 bytes. But the sum of the size of MBR and partition is 510 bytes isn't equal 512 bytes. What do we use the remaining 2 bytes to do?
The last two bytes are the boot signature. The BIOS is supposed to check that these bytes have values 0x55 and then 0xAA before loading the bootstrap code from sector 0, so this isn't really a Linux question.
https://support.microsoft.com/en-us/kb/149877
Wikipedia has tables for the common layouts, and the last two bytes are used for the "boot signature" (55 AA).
The BIOS is supposed to check this before trying to launch into the booting process.

RAM memory mapping - Need clarification

Total RAM size is 512 MB.
On my WEC7 device control panel, I'm seeing total memory as:
Storage memory: 53792 KB
Program memory: 376140 KB
So, total size is : 419MB.
My config.bib has following:
SECTION_BASE 80000000 00001000 RESERVED
ARGS 80001000 00001000 RESERVED
RSVD 80002000 001BA000 RESERVED
EMAC 801BC000 00009000 RESERVED
RSVD1 801C5000 0003B000 RESERVED
FBUFFER 95B00000 00200000 RESERVED
#define NK_START 80200000
#define NK_SIZE 05E00000
#define RAM_START 86000000
#define RAM_SIZE 0FB00000
According to this, RAM_SIZE is 251MB.
AFAIK, this is Program memory + Storage memory. Is my understanding is correct? If yes, why this difference? If no, what is the correct explanation for this?
My image_cfg.h has following line:
#define STATIC_MAPPING_RAM_SIZE (384)
And oemaddrtab_cfg.inc file has:
g_oalAddressTable
DCD 0x80000000, 0x00100000, STATIC_MAPPING_RAM_SIZE ; RAM image mapping; 0x80000000+384MB=0x98000000
DCD 0x9B000000, 0xFC000000, 64 ; 64 MB Peripheral device space (As per datasheet)
DCD 0x9F100000, 0x00000000, 1 ;Mapping Boot region
DCD 0x00000000, 0x00000000, 0 ; Terminate table
NK size:
nk.bin: 51MB
nk.nb0: 94MB
Anybody please explain why I am getting 419MB of memory, and also please explain more about these memory mapping...
In addition to the amount you specify in the config file you can have an additional RAM region passed to the system through the OEMGetExtendedDRAM function in your OAL. For BSPs supporting devices with different amounts of RAM is common to have the minimum amount configured in the bib file, detect if additional RAM is available, and return it using the above function.

How to debug a Linux kernel that freezes during boot?

I have a legacy device with a binary Linux 2.6.18 kernel that boots normally to its rootfs. However, if I try to compile this kernel from the source, the resulting kernel binary will freeze during the boot. I don't have the .config file used to build the previous kernel binary that is currently booting normally.
The boot is freezing and no error output is provided. Here is the boot log:
Linux version 2.6.18-6.2 (myuser#host) (gcc version 4.2.0 20070124 (prerelease) - BRCM 10ts-20080721) #10 SMP Sun Apr 28 18:25:24 BRT 2013
Fetching vars from bootloader... OK (E,d,B,C)
Detected 512 MB on MEMC0 (strap 0x23430310)
Board strapped at 512 MB, default is 256 MB
Options: sata=1 enet=1 emac_1=1 no_mdio=0 docsis=0 ebi_war=0 pci=1 smp=1
CPU revision is: 0002a044
FPU revision is: 00130001
Primary instruction cache 32kB, physically tagged, 2-way, linesize 64 bytes.
Primary data cache 64kB, 4-way, linesize 64 bytes.
<6>Synthesized TLB refill handler (23 instructions).
<6>Synthesized TLB load handler fastpath (37 instructions).
<6>Synthesized TLB store handler fastpath (37 instructions).
<6>Synthesized TLB modify handler fastpath (36 instructions).
Determined physical RAM map:
memory: 10000000 # 00000000 (usable)
memory: 10000000 # 20000000 (usable)
Using 32MB for memory, overwrite by passing mem=xx
User-defined physical RAM map:
node [00000000, 02000000: RAM]
node [02000000, 0e000000: RSVD]
node [20000000, 10000000: RAM]
<5>Reserving 224 MB upper memory starting at 02000000
<7>On node 0 totalpages: 65536
<7> DMA zone: 65536 pages, LIFO batch:15
<7>On node 1 totalpages: 65536
<7> Normal zone: 65536 pages, LIFO batch:15
Built 2 zonelists. Total pages: 131072
<5>Kernel command line: root=/dev/mtdblock3 rw rootfstype=jffs2 console=ttyS0,115200
PID hash table entries: 4096 (order: 12, 16384 bytes)
mips_counter_frequency = 202000000 from Calibration, = 202500000 from header(CPU_MHz/2)
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 286336k/524288k available (2924k kernel code, 237760k reserved, 544k data, 164k init, 0k highmem)
Mount-cache hash table entries: 512
Checking for 'wait' instruction... available.
plat_prepare_cpus: ENABLING 2nd Thread...
TP0: prom_boot_secondary: Kick off 2nd CPU...
CPU revision is: 0002a044
FPU revision is: 00130001
Primary instruction cache 32kB, physically tagged, 2-way, linesize 64 bytes.
Primary data cache 64kB, 4-way, linesize 64 bytes.
Synthesized TLB refill handler (23 instructions).
Brought up 2 CPUs
migration_cost=1000
NET: Registered protocol family 16
registering PCI controller with io_map_base unset
registering PCI controller with io_map_base unset
SCSI subsystem initialized
usbcore: registered new driver usbfs
usbcore: registered new driver hub
NET: Registered protocol family 2
IP route cache hash table entries: 16384 (order: 4, 65536 bytes)
TCP established hash table entries: 65536 (order: 7, 524288 bytes)
TCP bind hash table entries: 32768 (order: 6, 262144 bytes)
TCP: Hash tables configured (established 65536 bind 32768)
TCP reno registered
brcm-pm: disabling power to USB block
brcm-pm: disabling power to ENET block
brcm-pm: disabling power to SATA block
squashfs: version 3.2-r2 (2007/01/15) Phillip Lougher
JFFS2 version 2.2. (NAND) (SUMMARY) (C) 2001-2006 Red Hat, Inc.
io scheduler noop registered
io scheduler anticipatory registered (default)
io scheduler deadline registered
io scheduler cfq registered
Serial: 8250/16550 driver $Revision: 1.1.1.1 $ 3 ports, IRQ sharing disabled
serial8250: ttyS0 at MMIO 0x0 (irq = 22) is a 16550A
serial8250: ttyS1 at MMIO 0x0 (irq = 66) is a 16550A
serial8250: ttyS2 at MMIO 0x0 (irq = 67) is a 16550A
loop: loaded (max 8 devices)
brcm-pm: enabling power to ENET block
How do I go about debugging this? Any insights on possible solutions to the freeze are welcome as well.
One way to deal with this is to enable CONFIG_EARLY_PRINTK and add some printk() statements in kernel code that you suspect is freezing (most likely some drivers configuration parameters are wrong).
Also, you might be able to get old kernel config by looking at /boot/config-*, or at /proc/config.gz (it will exist only if old kernel had option CONFIG_IKCONFIG_PROC enabled).
Add initcall_debug to CONFIG_CMDLINE (kernel command line).
CONFIG_CMDLINE="root=/dev/ram0 rw mem=512M#0x0 initrd=0x800000,16M console=ttyS0,38400n8 rootfstype=ext2 init=/bin/busybox init -s initcall_debug"
There are some debugger options like kdb and kgdb, but I've always found them flaky and temperamental. Probably more-so if you can't even get your machine to boot. I concur with the CONFIG_EARLY_PRINTK advise, and would advise you to make sure you get kernel output on boot (not "quiet"), but it seems you have this already.
The "GPIO" suggestion above could work - but is very system-dependent and cumbersome. That said, I think you want an answer better than "Start adding a lot of printk's". You can start with the offending ethernet driver (BRC-PM?) or try removing that to see if that's related.
It'll take some investigation - sorry, but no "magic bullet"! :-O
From the last line of log,
brcm-pm: enabling power to ENET block
looks this is the issue of connected power supply to the system. it is not able to souce the power properly and that's why system is freezing.

Resources