Learn kernel module, module.ko cause module is already loaded? - linux

I am learning kernel module now, so I setup a Ubuntu 16.04 with kernel 4.4.0-28-generic in a virtual machine.
I installed this packages
# dpkg -l | grep linux
ii console-setup-linux 1.108ubuntu15 all Linux specific part of console-setup
ii libselinux1:amd64 2.4-3build2 amd64 SELinux runtime shared libraries
ii linux-base 4.0ubuntu1 all Linux image base package
ii linux-firmware 1.157.2 all Firmware for Linux kernel drivers
ii linux-generic 4.4.0.28.30 amd64 Complete Generic Linux kernel and headers
ii linux-headers-4.4.0-28 4.4.0-28.47 all Header files related to Linux kernel version 4.4.0
ii linux-headers-4.4.0-28-generic 4.4.0-28.47 amd64 Linux kernel headers for version 4.4.0 on 64 bit x86 SMP
ii linux-headers-generic 4.4.0.28.30 amd64 Generic Linux kernel headers
ii linux-image-4.4.0-28-generic 4.4.0-28.47 amd64 Linux kernel image for version 4.4.0 on 64 bit x86 SMP
ii linux-image-extra-4.4.0-28-generic 4.4.0-28.47 amd64 Linux kernel extra modules for version 4.4.0 on 64 bit x86 SMP
ii linux-image-generic 4.4.0.28.30 amd64 Generic Linux kernel image
ii linux-libc-dev:amd64 4.4.0-28.47 amd64 Linux Kernel Headers for development
ii linux-sound-base 1.0.25+dfsg-0ubuntu5 all base package for ALSA and OSS sound systems
ii linux-source 4.4.0.28.30 all Linux kernel source with Ubuntu patches
ii linux-source-4.4.0 4.4.0-28.47 all Linux kernel source for version 4.4.0 with Ubuntu patches
ii util-linux 2.27.1-6ubuntu3.1 amd64 miscellaneous system utilities
I already decompresses the kernel source package /usr/src/linux-source-4.4.0.tar.bz2 to /home/test/WorkSpace/Kernel/linux-source-4.4.0.
My system uname is
# uname -a
Linux ubuntu-ldm 4.4.0-28-generic #47-Ubuntu SMP Fri Jun 24 10:09:13 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
and I write a test module hello.c
#include <linux/init.h>
#include <linux/module.h>
MODULE_LICENSE("Dual BSD/GPL");
static int hello_init (void) {
printk (KERN_ALERT "Hello, World\n");
return 0;
}
static void hello_exit (void) {
printk (KERN_ALERT "Goodbye, cruel world\n");
}
module_init (hello_init);
module_exit (hello_exit);
Makefile
obj-m += module.o
module-objs := hello.o
all:
make modules M=`pwd` -C /home/test/WorkSpace/Kernel/linux-source-4.4.0
clean:
make modules clean M=`pwd` -C /home/test/WorkSpace/Kernel/linux-source-4.4.0
But something confused me. I make my module and try to insmod, and I got an error
# sudo insmod module.ko
insmod: ERROR: could not insert module module.ko: Invalid module format
What cause this error?
Did I use wrong version of kernel source?
for osgx
There is no additional line in output of dmesg after I try to insmod. The modinfo shows this
# modinfo module.ko
filename: /home/test/WorkSpace/LDM/hello/module.ko
license: Dual BSD/GPL
depends:
vermagic: 4.4.13 SMP mod_unload
I try to change the source path in the Makefile, remake it, and I get the new .ko
The modinfo of the new .ko is
# modinfo module.ko
filename: /home/joshua/WorkSpace/LDM/hello/module.ko
license: Dual BSD/GPL
srcversion: 82C361DBCB1C9BB5CA1DB07
depends:
vermagic: 4.4.0-28-generic SMP mod_unload modversions
But the problem still there, and the dmesg log looks like this
[ 28.540701] module: module verification failed: signature and/or required key missing - tainting kernel
[ 28.540879] module: module is already loaded
It looks like there is a module named module already in the system, so I try to change the target name in the Makefile to hello.o, make the new target name as hello.ko, after that the module works.
But when I ran lsmod | grep module, there is no module named module?

One possibility is that the module is loaded statically.
Then it won't show under lsmod, but you can still see it with:
ls /sys/module
See also: https://unix.stackexchange.com/questions/364956/how-can-insmod-fail-with-kernel-module-is-already-loaded-even-is-lsmod-does-not

Related

How to compile a linux kernel module for different linux kernel

I am sort of new to kernel programming, but i have been struggling a ton with this issue for days now. I have a machine with linux kernel '5.10.0-kali7-amd64' and im using it for development of a linux kernel module for Ubutnu 16.04.4 '4.4.0-119-generic', but i can't figure out any way that i can compile it on my machine for that version and for it to actually work on the 4.4.0 kernel machine.
The closest i've got is this:
I downloaded source from https://launchpad.net/ubuntu/xenial/+package/linux-headers-4.4.0-119
and installed with dpkg
I then downloaded and installed the 4.4.0-119-generic from https://www.ubuntuupdates.org/package/core/xenial/main/updates/linux-image-4.4.0-119-generic
Both of them installed with no issue.
I compiled my module by using in my Makefile make -C /lib/modules/4.4.0-119-generic/build M=$(PWD) modules which also worked and compiled my hello world module.
However when uploaded to the 4.4.0 machine the insmod errored saying insmod: ERROR: could not insert module rootkitMy.ko: Invalid module format. The dmesg says: module: rootkit: Unknown rela relocation: 4 I then compiled my source code on the 4.4.0 machine and created a module with literally the exact same modinfo, but that one did work.
here are the modinfos for both:
filename: /rootkit.ko
version: 0.01
description: Rootkit hook
author: Bl4ckC4t
license: GPL
srcversion: 46604268C8D1B7FA5115CB4
depends:
vermagic: 4.4.0-119-generic SMP mod_unload modversions retpoline
filename: /rootkitMy.ko
version: 0.01
description: Rootkit hook
author: Bl4ckC4t
license: GPL
srcversion: 46604268C8D1B7FA5115CB4
depends:
vermagic: 4.4.0-119-generic SMP mod_unload modversions retpoline
rootkitMy.ko was compiled on the 5.10 machine and didn't work while the rootkit.ko was compiled on the 4.4.0 machine and did work properly when injected with insmod What can I do to compile a working module from my 5.10 machine?
I managed to resolve the issue. Unknown rela relocation: 4
is an insmod error you get due to a change in the way the kernel handles PLT, more specifically the R_X86_64_PC32 and R_X86_64_PLT32. With binutils >= 2.31, the linker has decided to use R_X86_64_PLT32 relocations, which aren't supported in the older kernel.
To fix this:
I downloaded an older version of binutils (2.26.1) from https://ftp.gnu.org/gnu/binutils/
extracted the folder from the archive
compiled the binutils to /usr/local/binutils-2.6 by running
./configure --prefix=/usr/local/binutils-2.6
make
sudo make install
exported the new binutils to my path and recompiled my module export PATH=/usr/local/binutils-2.6/bin:$PATH
And now it works!

Running 32-bit ARM binary on aarch64 not working despite CONFIG_COMPAT

I've got a 64-bit ARM machine that I'd like to run 32-bit ARM binaries on. As a test case I've built a small hello world for 32-bit ARM using the arm-linux-gnueabihf-gcc toolchain. file shows it as:
root#ubuntu:/home/ubuntu# file hello
hello: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=61ffe5e22117a6d4c2ae37a1f4c76617d3e5facc, not stripped
But trying to run it produces:
root#ubuntu:/home/ubuntu# ./hello
bash: ./hello: cannot execute binary file: Exec format error
Based on a prior question, I checked whether the kernel was built with the CONFIG_COMPAT option, and it was:
root#ubuntu:/home/ubuntu# grep CONFIG_COMPAT= /boot/config-$(uname -r)
CONFIG_COMPAT=y
I also verified that the armhf architecture has been added and that the armhf version of the loader is present. Note that the loader itself doesn't run either:
root#ubuntu:/home/ubuntu# dpkg --print-foreign-architectures
armhf
root#ubuntu:/home/ubuntu# dpkg -l libc6:armhf
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-=============-============-=================================
ii libc6:armhf 2.30-0ubuntu2 armhf GNU C Library: Shared libraries
root#ubuntu:/home/ubuntu# file /lib/arm-linux-gnueabihf/ld-2.30.so
/lib/arm-linux-gnueabihf/ld-2.30.so: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, BuildID[sha1]=dff2b536287d61ddca68f3e001e14b7c235bbf68, stripped
root#ubuntu:/home/ubuntu# /lib/arm-linux-gnueabihf/ld-2.30.so
bash: /lib/arm-linux-gnueabihf/ld-2.30.so: cannot execute binary file: Exec format error
Other relevant system info:
root#ubuntu:/home/ubuntu# cat /proc/cpuinfo
processor : 0
BogoMIPS : 400.00
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics cpuid asimdrdm
CPU implementer : 0x43
CPU architecture: 8
CPU variant : 0x1
CPU part : 0x0af
CPU revision : 2
root#ubuntu:/home/ubuntu# uname -a
Linux ubuntu 5.3.0-24-generic #26-Ubuntu SMP Thu Nov 14 01:14:25 UTC 2019 aarch64 aarch64 aarch64 GNU/Linux
I'm running out of things to try at this point. Any idea how to get the kernel to recognize these binaries and run them?
Ok, it looks like the underlying problem here is not in software, unfortunately. The CPU we're using, the Cavium ThunderX2, is one of the few 64-bit ARM chips that does not have aarch32 support. Quoting from WikiChip:
Only the 64-bit AArch64 execution state is support. No 32-bit AArch32 support.
So, this explains why it's not able to run 32-bit ARM binaries. I'm still fairly sure that other 64-bit ARM chips, like the Cortex-A57, are able to do this.
Update: older 32-bit ARM binaries do indeed work on aarch64 with a CPU that supports it, as shown below on an AWS ARM a1.metal instance:
ubuntu#ip-172-31-12-156:~$ cat /proc/cpuinfo | tail
processor : 15
BogoMIPS : 166.66
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x0
CPU part : 0xd08
CPU revision : 3
ubuntu#ip-172-31-12-156:~$ uname -a
Linux ip-172-31-12-156 4.15.0-1054-aws #56-Ubuntu SMP Thu Nov 7 16:18:50 UTC 2019 aarch64 aarch64 aarch64 GNU/Linux
ubuntu#ip-172-31-12-156:~$ dpkg --print-foreign-architectures
armhf
ubuntu#ip-172-31-12-156:~$ file hello_hf
hello_hf: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-, for GNU/Linux 3.2.0, BuildID[sha1]=c95f0c46dfab925db53506751d7677156e334e5c, not stripped
ubuntu#ip-172-31-12-156:~$ ./hello_hf
hello, world!
This question has become more relevant as 32bit ARM chips are being phased out.
Me, I like to see old 32 bit binaries supported on new 64 bit hardware. Not so important, but convenient.
I took out my old Raspberry Pi Zero (32 bit) and Raspberry Pi 3B+ (64 bit) and for the first time installed a 64 bit OS on the 3B+ despite warnings of bugs and such and did some compiling.
No special make parameters. Just plain ordinary 32 bit compiling.
Runs fine on 32 bit and 64 bit.
Controllers and VMs will be running 32 bit architecture for many more years.
4GB memory limit is a pain yes. But for small apps its not a problem.

Is there a need to recomplie my linux kernel?

I am a beginner learning linux kernel module development. I am following a tutorial that says to recompile my kernel so as to enable various debugging features like forced module unloading e.t.c. Is is okay if I do that? Does it effects my pre-built kernel. In what cases that I am forced to insert a module into a running kernel and the kernel won't allow me to do so?
It is perfectly okay to compile and install a kernel to do kernel module development. If you are in ubuntu, you can follow the following steps to make sure that you are using the same kernel sources as your booted machine.
Step 1. Find out the linux being used in your booting from /boot/grub/grub.cfg file. Look for the entry agains 'linux ' in the boot option entries that you select while booting up.
Example excerpt : linux /boot/vmlinuz-3.13.0-24-generic root=UUID=e377a464-92db-4c07-86a9-b151800630c0 ro quiet splash $vt_handoff
Step 2. Look for the name of the package with the same version using the following command.
dpkg -l | grep linux | grep 3.13.0-24-generic
Example output:
$ dpkg -l | grep linux | grep 3.13.0-24-generic
ii linux-headers-3.13.0-24-generic 3.13.0-24.46 amd64 Linux kernel headers for version 3.13.0 on 64 bit x86 SMP
ii linux-image-3.13.0-24-generic 3.13.0-24.46 amd64 Linux kernel image for version 3.13.0 on 64 bit x86 SMP
ii linux-image-extra-3.13.0-24-generic 3.13.0-24.46 amd64 Linux kernel extra modules for version 3.13.0 on 64 bit x86 SMP
Step 3. Download sources of the package "linux-headers-3.13.0-24-generic" to get the same kernel that was used in your PC.
$ apt-get source linux-headers-3.13.0-24-generic
Step 4. Use the config file that is available at /boot/ folder as the config file to compile this kernel source
Example :
$ ls /boot/config-3.13.0-24-generic (Notice the same version used in this file)
Step 5. Turn on your debugging symbols on this config to do your testing.
Recompiling kernel help us to learn how kernel work.
latest kernel patches can be applied through kernel compile and install.
We can enable debug flag through compilation.
We can remove the not needed code.
Helps to add your own kernel code and test your code.
It is easy to recompile and install the linux kernel but it takes more time if we compile using low speed computer or VM.

Linaro ARM toolchain prebuild binaries in 64 bit OS

I have an Ubuntu 14.04 Trusty 64 bit machine. My OS and kernel are all 64 bit
uname -a
Linux --- 3.13.0-43-generic #72-Ubuntu SMP Mon Dec 8 19:35:06 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
I downloaded the pre-built tool-chain binaries from Linaro as I’m working on something with Raspberry Pi, but I’m unable to compile my code because the binaries that I had downloaded were 32 bit, it is not getting recognized on my 64 bit machine.
Could anyone give me some pointers? (if i can find a 64bit pre-built tool-chain from Linaro) or any other solution that I can deploy on my Ubuntu to workaround the problem
My error as below
some#somemachine:~/tools/compilers/linaro/gcc-linaro-arm-none-eabi-4.9-2014.09_linux/bin$ file ./arm-none-eabi-gcc-4.9.2
./arm-none-eabi-gcc-4.9.2: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.15, stripped
The file (yeap! its there in the directory)
some#somemachine:~/tools/compilers/linaro/gcc-linaro-arm-none-eabi-4.9-2014.09_linux/bin$ arm-none-eabi-gcc -v
bash: /home/some/tools/compilers/linaro/gcc-linaro-arm-none-eabi-4.9-2014.09_linux/bin/arm-none-eabi-gcc: No such file or directory
i found the problem. I had to do little bit of googling.
sudo apt-get install lib32z1 lib32ncurses5 lib32bz2-1.0
will fix the problem. This will enable 32 bit applications run normally on 64 bit machines + might have to install other *.i386 libraries
AFAICT, Linaro now provides native 64-bit versions of the tool-chain.
For example, the 14.11 release for the bare-metal ABI:
http://releases.linaro.org/14.11/components/toolchain/binaries/arm-none-eabi
http://releases.linaro.org/14.11/components/toolchain/binaries/arm-none-eabi/gcc-linaro-4.9-2014.11-x86_64_arm-eabi.tar.xz

Cross compiling a kernel module: how to set configs right

I'm trying to cross-compile an external module for the Beaglebone (Linux, ARM).To avoid kernel version issues I grabbed a kernel tarball and cross-compiled for ARM with the CodeSourcery toolchain on the host machine (x86). Then I cross-compiled an external hello-world module with the exact same toolchain against the exact same kernel sources. I used this simple makefile:
obj-m += hello-1.o
all:
make -C /home/***/****/linux-3.2.0-beaglebone-20120411.00
M=$(PWD) modules
clean:
make -C /home/***/****/linux-3.2.0-beaglebone-20120411.00 M=$(PWD) clean
When I copy the ko-file over to target and try to insmod it I get: "insmod: error inserting 'hello-1.ko': -1 Invalid module format" which (from what I learned in this group and elsewhere) usually stems from conflicting kernel versions, and indeed uname -a on the target gives:
Linux beaglebone 3.2.18 #1 Wed May 30 14:21:54 CEST 2012 armv7l GNU/Linux
while modinfo hello-1.ko gives:
srcversion: 140276773A3090F6F33891F
depends:
vermagic: 3.2.0+ mod_unload modversions ARMv5 p2v8
So version 3.2.18 vs. 3.2.0+ (why the +?) and armv7l vs. ARMv5!
Does anyone know why I get different versions albeit I compiled against the same kernel-sources (maybe some configs)?
Any suggestions are appreciated!
best,
Chris
answer is, you should run 'the kernel you compiled on beagle board.
'+' means that you modified your source tree.
And finally make sure that you're using correct defconfig. 'make beagle_defconfig' should work.

Resources