Kernel modules not loaded during boot - linux

Question

Observing that some kernel modules are not being loaded in the latest kernel 5.15.34-v7.
So I have built a core-image-base from meta-raspberrypi (0135a02) and while trying access the camera using Picamera got some errors. The errors mainly complain about mmal drivers not present.
root#raspberrypi3:~# python3
Python 3.10.4 (main, Mar 23 2022, 20:25:24) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from picamera import PiCamera
>>> camera = PiCamera()
mmal: mmal_vc_shm_init: could not initialize vc shared memory service
mmal: mmal_vc_component_create: failed to initialise shm for 'vc.camera_info' (7:EIO)
mmal: mmal_component_create_core: could not create component 'vc.camera_info' (7)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.10/site-packages/picamera/camera.py", line 408, in __init__
self._init_revision(options)
File "/usr/lib/python3.10/site-packages/picamera/camera.py", line 480, in _init_revision
with mo.MMALCameraInfo() as camera_info:
File "/usr/lib/python3.10/site-packages/picamera/mmalobj.py", line 2425, in __init__
super(MMALCameraInfo, self).__init__()
File "/usr/lib/python3.10/site-packages/picamera/mmalobj.py", line 696, in __init__
mmal_check(
File "/usr/lib/python3.10/site-packages/picamera/exc.py", line 184, in mmal_check
raise PiCameraMMALError(status, prefix)
picamera.exc.PiCameraMMALError: Failed to create MMAL component b'vc.camera_info': I/O error
>>>
>>>
root#raspberrypi3:~#
After digging through my system found an older build (don't know why I didn't delete it but thankfully it gave some insight into the issue), I tried booting that image and everything seems to be working fine.
So I checked out to the commit which the older build was using (63a3d8cb17c5d1affe8f2848f45fcc6a706f9412), and the camera worked fine(though I had to make few changes, which are not significant for this issue). While analyzing the bootlogs found that the latest build (0135a02) doesn't load all the drivers.
Also I have observed that the kernel module are compressed in the 5.15.34 kernel, eg: root#raspberrypi3:~# ls /lib/modules/5.15.34-v7/kernel/drivers/usb/gadget/libcomposite.ko.xz and while trying load the modules using modprobe getting the following error:
root#raspberrypi3:~# ls /lib/modules/5.15.34-v7/kernel/drivers/usb/gadget/legacy/
g_acm_ms.ko.xz g_cdc.ko.xz g_hid.ko.xz g_midi.ko.xz g_printer.ko.xz g_webcam.ko.xz gadgetfs.ko.xz
g_audio.ko.xz g_ether.ko.xz g_mass_storage.ko.xz g_multi.ko.xz g_serial.ko.xz g_zero.ko.xz
root#raspberrypi3:~# modprobe gadgetfs
modprobe: FATAL: Module gadgetfs not found in directory /lib/modules/5.15.34-v7
My question is what and where the changes have happened to the kernel between 63a3d8cb17c5d1affe8f2848f45fcc6a706f9412 (5.10) and 0135a02 (5.15) , so that I can look into and adapt the changes required ?
Note: All the commit hashes which are mentioned above are of meta-raspberrypi repo.
Logs
lsmod logs
5.15.34
root#raspberrypi3:~# lsmod
Module Size Used by
root#raspberrypi3:~#
5.10.81
root#raspberrypi3:~# lsmod
Module Size Used by
rfcomm 49152 2
cmac 16384 3
algif_hash 16384 1
nfc 86016 0
aes_arm_bs 24576 2
crypto_simd 16384 1 aes_arm_bs
cryptd 24576 2 crypto_simd
algif_skcipher 16384 1
af_alg 28672 6 algif_hash,algif_skcipher
bnep 20480 2
hci_uart 40960 1
btbcm 16384 1 hci_uart
bluetooth 421888 31 hci_uart,bnep,btbcm,rfcomm
ecdh_generic 16384 2 bluetooth
ecc 36864 1 ecdh_generic
ipv6 503808 26
brcmfmac 331776 0
brcmutil 24576 1 brcmfmac
sha256_generic 16384 0
bcm2835_v4l2 49152 0
cfg80211 782336 1 brcmfmac
bcm2835_codec 40960 0
bcm2835_isp 32768 0
v4l2_mem2mem 36864 1 bcm2835_codec
rfkill 32768 4 bluetooth,nfc,cfg80211
bcm2835_mmal_vchiq 36864 3 bcm2835_isp,bcm2835_codec,bcm2835_v4l2
videobuf2_dma_contig 20480 2 bcm2835_isp,bcm2835_codec
videobuf2_vmalloc 16384 1 bcm2835_v4l2
videobuf2_memops 16384 2 videobuf2_dma_contig,videobuf2_vmalloc
videobuf2_v4l2 32768 4 bcm2835_isp,bcm2835_codec,bcm2835_v4l2,v4l2_mem2mem
videobuf2_common 61440 5 bcm2835_isp,bcm2835_codec,bcm2835_v4l2,v4l2_mem2mem,videobuf2_v4l2
raspberrypi_hwmon 16384 0
videodev 253952 6 bcm2835_isp,bcm2835_codec,videobuf2_common,bcm2835_v4l2,v4l2_mem2mem,videobuf2_v4l2
mc 45056 6 bcm2835_isp,bcm2835_codec,videobuf2_common,videodev,v4l2_mem2mem,videobuf2_v4l2
vc_sm_cma 32768 2 bcm2835_isp,bcm2835_mmal_vchiq
uio_pdrv_genirq 16384 0
uio 20480 1 uio_pdrv_genirq
fixed 16384 0
root#raspberrypi3:~#

Answer 1

Make sure you have kernel-modules installed:
IMAGE_INSTALL_append = " kernel-modules"
EDIT
The package that provides all kernel modules is kernel-modules, or each modules is within a separate package kernel-module-<module_name>. For meta-raspberrypi, they set kernel-modules as a package not essential for boot, means that if the package is not found, the board should boot normal:
meta-raspberrypi/conf/machine/include/rpi-base.inc
MACHINE_EXTRA_RRECOMMENDS += "kernel-modules udev-rules-rpi"
In previous meta-raspberrypi branches, it was a rpi image recipe rpi-basic-image.bb:
# Base this image on core-image-minimal
include recipes-core/images/core-image-minimal.bb
# Include modules in rootfs
IMAGE_INSTALL += " \
kernel-modules \
"
SPLASH = "psplash-raspberrypi"
IMAGE_FEATURES += "ssh-server-dropbear splash"
do_image:prepend() {
bb.warn("The image 'rpi-basic-image' is deprecated, please use 'core-image-base' instead")
}
So, the only thing needed for integrating the kernel modules is kernel-modules package, either by the image example, or try:
local.conf
MACHINE_EXTRA_RRECOMMENDS_remove = "kernel-modules"