Why do I get console output from ACPID when devtmpfs is mounted and logged when not? - linux

I have a weird issue which I diagnosed all the way back to simply being that if I mount devtmpfs then launch ACPID daemon, i get console output, if I don't mount devtmpfs, the ACPI daemon output goes to the log. I also get some rpcbind messages when reboot is issued, but I think it's the same thing (normally logs instead of output to screen).
apcid: starting up with netlink and the input layer
apcid: 1 rule loaded
apcid: waiting for events: event logging is off
Does anyone have any ideas why? I can still use udev to create and load all my devices either way (mounting devtmpfs or not). I tried setting up /etc/sysctl.conf to use kernel.printk = 3 4 1 3 (the default without it was 3 4 1 7), but that made no difference.
TIA

The answer is that I had to restart syslogd and klogd after mounting devtmpfs. Output to syslog that failed when to console due to apcid using LOG_CONS option. This was busybox, kill and trying to start again didn't work, start-stop-daemon worked.

Related

How to troubleshoot an expected CDROM device on custom Linux kernel?

I'm looking for some hints while troubleshooting missing CDROM device.
The problem is, missing configuration option for my custom kernel (linux-5.4.78).
My current .config has:
CONFIG_CDROM=y
CONFIG_BLK_DEV_SR=y
CONFIG_VHOST_SCSI=y
CONFIG_BLK_SCSI_REQUEST=y
CONFIG_SCSI_MOD=y
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
CONFIG_SCSI_NETLINK=y
CONFIG_SCSI_PROC_FS=y
CONFIG_SCSI_SPI_ATTRS=y
CONFIG_SCSI_FC_ATTRS=y
CONFIG_SCSI_ISCSI_ATTRS=y
CONFIG_SCSI_SAS_ATTRS=y
CONFIG_SCSI_SAS_LIBSAS=y
CONFIG_SCSI_SAS_HOST_SMP=y
CONFIG_SCSI_LOWLEVEL=y
CONFIG_ISCSI_TCP=y
CONFIG_ISCSI_BOOT_SYSFS=y
CONFIG_SCSI_CXGB3_ISCSI=y
CONFIG_SCSI_CXGB4_ISCSI=y
CONFIG_SCSI_BNX2_ISCSI=y
CONFIG_BE2ISCSI=y
CONFIG_SCSI_HPSA=y
CONFIG_SCSI_3W_9XXX=y
CONFIG_SCSI_3W_SAS=y
CONFIG_SCSI_ACARD=y
CONFIG_SCSI_AACRAID=y
CONFIG_SCSI_AIC7XXX=y
CONFIG_SCSI_AIC79XX=y
CONFIG_SCSI_AIC94XX=y
CONFIG_SCSI_HISI_SAS=y
CONFIG_SCSI_HISI_SAS_PCI=y
CONFIG_SCSI_MVSAS=y
CONFIG_SCSI_MVSAS_TASKLET=y
CONFIG_SCSI_MVUMI=y
CONFIG_SCSI_DPT_I2O=y
CONFIG_SCSI_ADVANSYS=y
CONFIG_SCSI_ARCMSR=y
CONFIG_SCSI_ESAS2R=y
CONFIG_SCSI_MPT3SAS=y
CONFIG_SCSI_MPT2SAS=y
CONFIG_SCSI_SMARTPQI=y
CONFIG_SCSI_UFSHCD=y
CONFIG_SCSI_UFSHCD_PCI=y
CONFIG_SCSI_UFSHCD_PLATFORM=y
CONFIG_SCSI_UFS_CDNS_PLATFORM=y
CONFIG_SCSI_UFS_HISI=y
CONFIG_SCSI_UFS_BSG=y
CONFIG_SCSI_HPTIOP=y
CONFIG_SCSI_BUSLOGIC=y
CONFIG_SCSI_FLASHPOINT=y
CONFIG_SCSI_MYRB=y
CONFIG_SCSI_MYRS=y
CONFIG_VMWARE_PVSCSI=y
CONFIG_SCSI_SNIC=y
CONFIG_SCSI_DMX3191D=y
CONFIG_SCSI_FDOMAIN=y
CONFIG_SCSI_FDOMAIN_PCI=y
CONFIG_SCSI_GDTH=y
CONFIG_SCSI_ISCI=y
CONFIG_SCSI_IPS=y
CONFIG_SCSI_INITIO=y
CONFIG_SCSI_INIA100=y
CONFIG_SCSI_PPA=y
CONFIG_SCSI_IMM=y
CONFIG_SCSI_STEX=y
CONFIG_SCSI_SYM53C8XX_2=y
CONFIG_SCSI_SYM53C8XX_MMIO=y
CONFIG_SCSI_IPR=y
CONFIG_SCSI_IPR_TRACE=y
CONFIG_SCSI_IPR_DUMP=y
CONFIG_SCSI_QLOGIC_1280=y
CONFIG_SCSI_QLA_FC=y
CONFIG_SCSI_QLA_ISCSI=y
CONFIG_SCSI_LPFC=y
CONFIG_SCSI_DC395x=y
CONFIG_SCSI_AM53C974=y
CONFIG_SCSI_WD719X=y
CONFIG_SCSI_PMCRAID=y
CONFIG_SCSI_PM8001=y
CONFIG_SCSI_BFA_FC=y
CONFIG_SCSI_VIRTIO=y
CONFIG_SCSI_CHELSIO_FCOE=y
CONFIG_SCSI_LOWLEVEL_PCMCIA=y
CONFIG_SCSI_DH=y
CONFIG_SCSI_DH_RDAC=y
CONFIG_SCSI_DH_HP_SW=y
CONFIG_SCSI_DH_EMC=y
CONFIG_SCSI_DH_ALUA=y
CONFIG_ISCSI_TARGET=y
CONFIG_ISCSI_TARGET_CXGB4=y
CONFIG_QED_ISCSI=y
I'm expecting to see /dev/sr0. It's not there. dmesg is mute about sr0.
However, I'm able to see it using stock kernel and I've identified it was bring by BLK_DEV_SR on my target:
# ls -l /dev/sr0
brw-rw---- 1 root optical 11,0 Apr 21 15:02 /dev/sr0
# readlink /sys/dev/block/11\:0/device/driver
../../../../../../../../../../../../bus/scsi/driver/sr
I'd appreciate any help.
If your custom linux has udev, try udevadm monitor.
When you eject or insert a cd, you should see a change event on the terminal with the device path.
Also it's normally standard for a cdrom drive, no matter the actual device path, to be forwarded to /media/cdrom

How to check if /init starts /etc/inittab

I have an embedded ARM system with processor AT91SAM9G45.
System consists of two components:
Linux kernel (4.14.79)
Busybox 1.29.3 as initramfs image.
I connect to the device using putty and connecting to serial port.
When kernel starts, everything goes fine. Kernel unpacks initramfs image, all files are found and listed (I see it by debug messages). But when it starts /init, log messages are:
Freeing unused kernel memory: 384K
This architecture does not have kernel memory protection.
run_init_process BEFORE /init
run_init_process AFTER /init, result = 0
Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004
/init is symlink to /bin/busybox. I tried to replace /init with /sbin/init, /bin/busybox, /linuxrc, but results are the same.
/etc/inittab file:
# Begin /etc/inittab
id::initdefault:
si::sysinit:/etc/init.d/rc S
#l0::wait:/etc/rc.d/init.d/rc 0
#l1::wait:/etc/rc.d/init.d/rc 1
#l2::wait:/etc/rc.d/init.d/rc 2
#l3::wait:/etc/rc.d/init.d/rc 3
#l4::wait:/etc/rc.d/init.d/rc 4
#l5::wait:/etc/rc.d/init.d/rc 5
#l6::wait:/etc/rc.d/init.d/rc 6
ca::ctrlaltdel:/sbin/shutdown -t1 -a -r now
su::once:/sbin/sulogin
1::respawn:/sbin/getty ttyS1 115200
2::respawn:/sbin/getty ttyS2 115200
3::respawn:/sbin/getty ttyS3 115200
4::respawn:/sbin/getty ttyS4 115200
5::respawn:/sbin/getty ttyS5 115200
6::respawn:/sbin/getty ttyS6 115200
# End /etc/inittab
/etc/init.d/rcS file (this file is allowed to execute):
#!/bin/busybox sh
echo "Hello world!"
I don't know if even /init process starts parsing /etc/inittab or it falls before getting /etc/inittab by some reasons I cannot find out. Maybe there are
some mistakes in my /etc/inittab and /etc/init.d/rcS files. Maybe there are some errors with terminal (/etc/init.d/rcS cannot write to stdout cause it's blocked, suspended, being used by another process and so on).
How to definitely get sured, that /etc/inittab is started?
Welcome to StackOverflow.
I see there is space between rc and S
si::sysinit:/etc/init.d/rc S
change it to
si::sysinit:/etc/init.d/rcS
let me know if it works.
/init is symlink to /bin/busybox.
The typical /init file in an initramfs built by Buildroot that incorporates Busybox is a script of seven lines:
#!/bin/sh
# devtmpfs does not get automounted for initramfs
/bin/mount -t devtmpfs devtmpfs /dev
exec 0</dev/console
exec 1>/dev/console
exec 2>/dev/console
exec /sbin/init $*
Note the comment ("devtmpfs does not get automounted for initramfs") and the mount command for /dev.
It's /sbin/init (rather than /init) that is linked to /bin/busybox.
IOW without the proper setup of the /dev directory, userland has no I/O capabilty.
Only after devtmpfs has been mounted should the init program in Busybox be executed, which will then access /etc/inittab.
See Is there a way to get Linux to treat an initramfs as the final root filesystem?
and
Make CONFIG_DEVTMPFS_MOUNT apply to initramfs/initmpfs

Cannot change readahead value in linux

I am new to Linux and I am trying to set the readahead value as told by this article: https://www.ca.com/us/services-support/ca-support/ca-support-online/knowledge-base-articles.tec616116.html
When I try the command:
blockdev --setra 2048 /dev/sdb
Nothing happens. I've also tried with sudo. When I run blockreport I always get this report:
So I also tried blockdev --setra 2048 /dev/sr0 since that seems to be the device name, but then I get the error "device sr0 not found".
Do you know what I am doing wrong when trying to set the readahead value?
A restart of the system was required to take effect.
/dev/sr0 is a device on the scsi controller. /dev/cdrom is a symlink to either /dev/sr0 or /dev/hdc or whichever block device is appropriate.
I think you are trying to change RA for which the actual medium is not present.

How to release hugepages from the crashed application

I have an application that uses hugepage and the application suddenly crashed due to some bug.
After crashing, since the application does not release the hugepage properly, the free hugepage number is not increased in sys filesystem.
$ sudo cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
0
$ sudo cat /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
1024
Is there a way to release the hugepages by force?
Sometimes need to check all directory that hugetlbfs has been mounted.
So,
find mounted directory by command mount | grep huge.
check every directory except especially /dev/hugepages.
delete all 2M-sized files. (2M is the size of hugepage)
Use ipcs -m to list the shared memory segments.
Use ipcrm to remove the left over shared memory segments.
Edit on 06/24/2019:
Ok, so, the above answer, while correct as far as it goes, was a bit brief. In particular, if you have a host with multiple DB instances, and only one is crashed how can you determine which (if any) memory segments should be cleaned up?
Well, this too, can be done. For each running instance, connect w/ / as sysdba, then do oradebug setmypid (any pid will do, as all Oracle PIDs connect to the SGA). Then do oradebug ipc. That will (hopefully) return IPC information written to the trace file. So, go to the udump (or diag_dest) directory, and look for your trace file. It will contain all the IPC information for the instance. This will include ShmId. Look through the file for the ShmId(s) that this instance is using. Now look at the output of ipcs -m.
When you have done that for all the running instances, any memory segment output by ipcs -m that shows non-zero memory allocation, and that you cannot account for in the oradebug ipc information from any running instance, must be the left over memory segments from the crashed instance. Use ipcrm to remove it/them.
When doing this on a host with multiple running instances, this can be a bit fraught. Please proceed with caution. You don't want to remove the SGA of a running instance!
Hope that helps....
HugeTLB can either be used for shared memory (and Mark J. Bobak's answer would deal with that) or the app mmaps files created in a hugetlb filesystem. If the app crashes without removing those files they survive and keep corresponding memory 'allocated'.
Check hugeTLB filesystem and see if there are any leftover files from the app. Removing them would release the memory.
If you follow the instruction below, you can get rid of the allocated hugepages:
1) Let's check the hugepages which were free at restart
dpdk#dpdkvm:~$ ls /mnt/huge/
empty
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ cat /proc/meminfo
...
HugePages_Total: 256
HugePages_Free: 256
...
2) Starting a dpdk application with wrong parameters, producing an error
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ sudo ./build/kni -c 0x03 -n 2 -- -P -p 0x03 --config="(0,0,1),(1,0,1)"
...
EAL: Error - exiting with code: 1
Cause: No supported Ethernet device found
3) When I check hugepages, there is not any free
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ cat /proc/meminfo
...
HugePages_Total: 256
HugePages_Free: 0
...
4) Now, when I check the mounted hugepage directory, I can see the files which are not given back to OS by dpdk application.
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ ls /mnt/huge/
...
rtemap_0 rtemap_137 rtemap_176 rtemap_214 rtemap_253 rtemap_62
...
5) Finally, if you remove the files starting with rtemap, you can give the hugepages back
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ sudo rm /mnt/huge/*
[sudo] password for dpdk:
dpdk#dpdkvm:~/dpdk-1.8.0/examples/kni$ cat /proc/meminfo
...
HugePages_Total: 256
HugePages_Free: 256
...
your hugetlb may be used by shared memory or mmap files.
try to remove the shared memories or umount the hugetlb fs

What is the significance of the numbers in the name of the flush processes for newer linux kernels?

I am running kernel 2.6.33.7.
Previously, I was running v2.6.18.x. On 2.6.18, the flush processes were named pdflush.
After upgrading to 2.6.33.7, the flush processes have a format of "flush-:".
For example, currently I see flush process "flush-8:32" popping up in top.
In doing a google search to try to determine an answer to this question, I saw examples of "flush-8:38", "flush-8:64" and "flush-253:0" just to name a few.
I understand what the flush process itself does, my question is what is the significance of the numbers on the end of the process name? What do they represent?
Thanks
Device numbers used to identify block devices. A kernel thread may be spawned to handle a particular device.
(On one of my systems, block devices are currently numbered as shown below. They may change from boot to boot or hotplug to hotplug.)
$ grep ^ /sys/class/block/*/dev
/sys/class/block/dm-0/dev:254:0
/sys/class/block/dm-1/dev:254:1
/sys/class/block/dm-2/dev:254:2
/sys/class/block/dm-3/dev:254:3
/sys/class/block/dm-4/dev:254:4
/sys/class/block/dm-5/dev:254:5
/sys/class/block/dm-6/dev:254:6
/sys/class/block/dm-7/dev:254:7
/sys/class/block/dm-8/dev:254:8
/sys/class/block/dm-9/dev:254:9
/sys/class/block/loop0/dev:7:0
/sys/class/block/loop1/dev:7:1
/sys/class/block/loop2/dev:7:2
/sys/class/block/loop3/dev:7:3
/sys/class/block/loop4/dev:7:4
/sys/class/block/loop5/dev:7:5
/sys/class/block/loop6/dev:7:6
/sys/class/block/loop7/dev:7:7
/sys/class/block/md0/dev:9:0
/sys/class/block/md1/dev:9:1
/sys/class/block/sda/dev:8:0
/sys/class/block/sda1/dev:8:1
/sys/class/block/sda2/dev:8:2
/sys/class/block/sdb/dev:8:16
/sys/class/block/sdb1/dev:8:17
/sys/class/block/sdb2/dev:8:18
/sys/class/block/sdc/dev:8:32
/sys/class/block/sdc1/dev:8:33
/sys/class/block/sdc2/dev:8:34
/sys/class/block/sdd/dev:8:48
/sys/class/block/sdd1/dev:8:49
/sys/class/block/sdd2/dev:8:50
/sys/class/block/sde/dev:8:64
/sys/class/block/sdf/dev:8:80
/sys/class/block/sdg/dev:8:96
/sys/class/block/sdh/dev:8:112
/sys/class/block/sdi/dev:8:128
/sys/class/block/sr0/dev:11:0
/sys/class/block/sr1/dev:11:1
/sys/class/block/sr2/dev:11:2
You should also be able to figure this out by searching for those numbers in /proc/self/mountinfo, eg:
$ grep 8:32 /proc/self/mountinfo
25 22 8:32 / /var rw,relatime - ext4 /dev/mapper/sysvg-var rw,barrier=1,data=ordered
This has the side benefit of working with nfs as well:
$ grep 0:73 /proc/self/mountinfo
108 42 0:73 /foo /mnt/foo rw,relatime - nfs host.domain.com:/volume/path rw, ...
Note, the data I included here is fabricated, but the mechanism works just fine.

Resources