Missing slab.h from linux includes - Ubuntu 16 VM - linux

I'm working on a linux kernel driver and need access to the kmalloc and kfree functions. From my research these should be available in the slab.h header, but that file doesn't exist in my filesystem.
I tried updating my includes using this solution: https://askubuntu.com/questions/75709/how-do-i-install-kernel-header-files but it shows that I already have all relevant files.
My system is a VMWare Ubuntu 16.04 installation running kernel 4.15.0.
Any ideas?

Here is a very simple demo module that calls kmalloc and kfree:
demo.c:
#define pr_fmt(fmt) "demo: " fmt
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/slab.h>
MODULE_LICENSE("GPL");
static int __init demo_init(void) {
void *buf;
buf = kmalloc(1000, GFP_KERNEL);
pr_info("kmalloc returned %p\n", buf);
kfree(buf);
return 0;
}
static void __exit demo_exit(void) {
}
module_init(demo_init);
module_exit(demo_exit);
Makefile:
ifneq ($(KERNELRELEASE),)
# KBuild part of Makefile
obj-m += demo.o
else
# Normal part of Makefile
#
# Kernel build directory specified by KDIR variable
# Default to running kernel's build directory if KDIR not set externally
KDIR ?= "/lib/modules/`uname -r`/build"
all:
$(MAKE) -C "$(KDIR)" M=`pwd` modules
clean:
$(MAKE) -C "$(KDIR)" M=`pwd` clean
endif
You can just run make to build the module for the currently running kernel version:
$ make
Or you can set KDIR to build the module for an arbitrary kernel version (defined by ${KERNELVER} in the following example):
$ make KDIR="/lib/modules/${KERNELVER}/build"
(If KDIR is unspecified, the Makefile sets it to the build path for the currently running kernel: "/lib/modules/`uname -r`/build".)
If it builds successfully, then you definitely have the kernel headers installed!
To test the module, run:
$ sudo /sbin/insmod demo.ko
$ sudo /sbin/rmmod demo
$ sudo dmesg
There should be a message on the kernel log similar to this showing the return value from the kmalloc() call:
[TIMESTAMP] demo: kmalloc returned xxxxxxxxxxxxxxx
The module also calls kfree() to free the allocated block.

Related

How to compile a "hello world" /init process?

I'm trying to learn how to build a minimal initramfs for Linux. I set CONFIG_INITRAMFS_SOURCE="asd", created asd directory, inside there's an init file compiled using gcc -fpic -static main.c -o init. In the kernel log I'm getting the following error:
Run /init as init process
Failed to execute /init (error -8)
Here's the input source code:
int main() {
printf("Hello, world!\n");
for(;;);
}
What am I doing wrong?
It works with a defconfig kernel and CONFIG_INITRAMFS_SOURCE set. In the directory I had to make the node dev/console.
mkdir dev
sudo mknod --mode=0600 dev/console c 5 1
After running -kernel linux*/arch/x86_64/boot/bzImage in qemu Hello, world! is shown after calling init.

Compiling mptfc kernel module for CentOS 7 -- insmod returns "invalid parameters"

I'm trying to compile the mptfc driver for a CentOS 7 box. We have some legacy hardware that we need to support on CentOS 7 for dependency reasons.
mptfc was included in CentOS 6, but in CentOS 7 they decided to not include it in the kernel any more. (source: Redhat)
I noticed that elrepo created a RPM for the driver (from this article), but I had to dig for it. Eventually found it in some mirrors that provided archive capabilities, but the computer would not finish booting the kernel. Unfortunately, I don't have that output to display, but it wasn't pretty. Lots of error messages.
Because I can not use CentOS 6, I decided the next choice was to compile the kernel module myself. I have never done that, so I am running into a few issues.
First, I downloaded the headers for the kernel I am running (3.10.0-957.el7.x86_64) using:
yum install linux-headers
Then I downloaded the source code on a different machine for Linux 3.10 from Github. I copied the drivers/message/fusion directory from the source code to the machine I am compiling and put it in my home folder. I modified the Makefile to look like:
...
obj-$(CONFIG_FUSION_SPI) += mptbase.o mptscsih.o mptspi.o
obj-m += mptbase.o mptscsih.o mptfc.o
obj-m += mptbase.o mptscsih.o mptsas.o
obj-$(CONFIG_FUSION_CTL) += mptctl.o
obj-$(CONFIG_FUSION_LAN) += mptlan.o
KDIR := /lib/modules/$(shell uname -r)/build
PWD := $(shell pwd)
default:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
Then I ran make. It compiles fine, but when I try to do the following:
insmod mptfc.ko
I get the following:
[root#host fusion]# insmod mptfc.ko
insmod: ERROR: could not insert module mptfc.ko: Invalid parameters
And dmesg returns:
[root#host fusion]# dmesg | tail
[184711.751960] mptfc: disagrees about version of symbol mptscsih_qcmd
[184711.751970] mptfc: Unknown symbol mptscsih_qcmd (err -22)
I've never compiled kernel modules before, so I'm not sure where to start debugging this.
My end goal is to get the legacy pci card that uses mptfc working. If there is a better way to get the driver into the kernel, and working, that works as well.

How to build and run a Linux kernel module on QEMU?

My teacher gives me a linux kernel vmlinuz-3.17.2 and a rootfs.ext2 which can be loaded to qemu. And he asks me to build a simplest kernel module which prints a hello world as homework.
Firstly, I download the kernel source and run make oldconfig
Secondly, I make the config to be PREEMPT and without modversions
(according to uname -a of vmlinuz running in qemu) , then make prepare
Thirdly, I compile the kernel mod and copy hello.ko in rootfs.ext2
Finally, In qemu, I run insmod hello.ko which exit without any prompt
and echo $? returns 0.
However, I can't see anything in dmesg or /var/log/messages
Is there anything wrong? How can I do with this?
There is also nothing to be printed when I run rmmod hello.ko successfully.
My log level is 7 4 1 7
I have make my hello.c as follows:
#include <linux/init.h>
#include <linux/module.h>
static int __init hello_init(void)
{
pr_info("Hello World");
return -1;
// I changed this to -1 deliberately, Because It seems that the code is not executed.
}
static void __exit hello_exit(void)
{
printk(KERN_ERR "Goodbye, cruel world\n");
}
MODULE_LICENSE("GPL");
module_init(hello_init);
module_exit(hello_exit);
Buildroot
Buildroot is the easiest way to do it:
minimal out-of-tree example: How to add a Linux kernel driver module as a Buildroot package?
minimal in-tree example: https://github.com/cirosantilli/buildroot/tree/kernel-module-2016.05
how to debug the module with GDB: How to debug Linux kernel modules with QEMU?
Tested on Ubuntu 16.04.

linux/module.h No such file or Directory

For my thesis I am creating a Manet using the protocol ARAN. To install the protocol I'm using this manual, but the first step, the creation of trace_route, I received errors such as:
-linux/module.h: No such file or directory
-linux/procs_Fs: No such file or directory
-linux/skbuff: No such file or directory
I searched the web and found out that the problem is in the headers, but I do not find the solution ...
P.S. I am using Ubuntu 10.04 LTS Kernel 2.6.33 recompiled
You're missing the Linux kernel headers which allow you to compile code against the Linux kernel.
To install just the headers in Ubuntu:
$ sudo apt-get install linux-headers-$(uname -r)
To install the entire Linux kernel source in Ubuntu:
$ sudo apt-get install linux-source
Note that you should use the kernel headers that match the kernel you are running.
**/*source file name is basic.c */**
#include <linux/init.h>
#include <linux/module.h>
/*MODULE_LICENSE("Dual BSD/GPL");*/
static int hello_init(void)
{
printk(KERN_ALERT "Hello, world\n");
return 0;
}
static void hello_exit(void)
{
printk(KERN_ALERT "Goodbye, cruel world\n");
}
module_init(hello_init);
module_exit(hello_exit);
=====================================
now make file for ubuntu
/*at first type on ur terminal that $(uname -r) then u will get the version..
that is using on ur system */
obj-m +=basic.o
KDIR =//usr/src/linux-headers-3.13.0-44-generic
all:
$(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules
clean:
rm -rf *.o *.ko *.mod.* *.symvers *.order
================================================
To run the code
$sudo insmode basic.ko
$dmesg
u will get the output
$sudo rmmod basic.ko
$dmesg

How can I link to a specific glibc version?

When I compile something on my Ubuntu Lucid 10.04 PC it gets linked against glibc. Lucid uses 2.11 of glibc. When I run this binary on another PC with an older glibc, the command fails saying there's no glibc 2.11...
As far as I know glibc uses symbol versioning. Can I force gcc to link against a specific symbol version?
In my concrete use I try to compile a gcc cross toolchain for ARM.
You are correct in that glibc uses symbol versioning. If you are curious, the symbol versioning implementation introduced in glibc 2.1 is described here and is an extension of Sun's symbol versioning scheme described here.
One option is to statically link your binary. This is probably the easiest option.
You could also build your binary in a chroot build environment, or using a glibc-new => glibc-old cross-compiler.
According to the http://www.trevorpounds.com blog post Linking to Older Versioned Symbols (glibc), it is possible to to force any symbol to be linked against an older one so long as it is valid by using the same .symver pseudo-op that is used for defining versioned symbols in the first place. The following example is excerpted from the blog post.
The following example makes use of glibc’s realpath, but makes sure it is linked against an older 2.2.5 version.
#include <limits.h>
#include <stdlib.h>
#include <stdio.h>
__asm__(".symver realpath,realpath#GLIBC_2.2.5");
int main()
{
const char* unresolved = "/lib64";
char resolved[PATH_MAX+1];
if(!realpath(unresolved, resolved))
{ return 1; }
printf("%s\n", resolved);
return 0;
}
Setup 1: compile your own glibc without dedicated GCC and use it
Since it seems impossible to do just with symbol versioning hacks, let's go one step further and compile glibc ourselves.
This setup might work and is quick as it does not recompile the whole GCC toolchain, just glibc.
But it is not reliable as it uses host C runtime objects such as crt1.o, crti.o, and crtn.o provided by glibc. This is mentioned at: https://sourceware.org/glibc/wiki/Testing/Builds?action=recall&rev=21#Compile_against_glibc_in_an_installed_location Those objects do early setup that glibc relies on, so I wouldn't be surprised if things crashed in wonderful and awesomely subtle ways.
For a more reliable setup, see Setup 2 below.
Build glibc and install locally:
export glibc_install="$(pwd)/glibc/build/install"
git clone git://sourceware.org/git/glibc.git
cd glibc
git checkout glibc-2.28
mkdir build
cd build
../configure --prefix "$glibc_install"
make -j `nproc`
make install -j `nproc`
Setup 1: verify the build
test_glibc.c
#define _GNU_SOURCE
#include <assert.h>
#include <gnu/libc-version.h>
#include <stdatomic.h>
#include <stdio.h>
#include <threads.h>
atomic_int acnt;
int cnt;
int f(void* thr_data) {
for(int n = 0; n < 1000; ++n) {
++cnt;
++acnt;
}
return 0;
}
int main(int argc, char **argv) {
/* Basic library version check. */
printf("gnu_get_libc_version() = %s\n", gnu_get_libc_version());
/* Exercise thrd_create from -pthread,
* which is not present in glibc 2.27 in Ubuntu 18.04.
* https://stackoverflow.com/questions/56810/how-do-i-start-threads-in-plain-c/52453291#52453291 */
thrd_t thr[10];
for(int n = 0; n < 10; ++n)
thrd_create(&thr[n], f, NULL);
for(int n = 0; n < 10; ++n)
thrd_join(thr[n], NULL);
printf("The atomic counter is %u\n", acnt);
printf("The non-atomic counter is %u\n", cnt);
}
Compile and run with test_glibc.sh:
#!/usr/bin/env bash
set -eux
gcc \
-L "${glibc_install}/lib" \
-I "${glibc_install}/include" \
-Wl,--rpath="${glibc_install}/lib" \
-Wl,--dynamic-linker="${glibc_install}/lib/ld-linux-x86-64.so.2" \
-std=c11 \
-o test_glibc.out \
-v \
test_glibc.c \
-pthread \
;
ldd ./test_glibc.out
./test_glibc.out
The program outputs the expected:
gnu_get_libc_version() = 2.28
The atomic counter is 10000
The non-atomic counter is 8674
Command adapted from https://sourceware.org/glibc/wiki/Testing/Builds?action=recall&rev=21#Compile_against_glibc_in_an_installed_location but --sysroot made it fail with:
cannot find /home/ciro/glibc/build/install/lib/libc.so.6 inside /home/ciro/glibc/build/install
so I removed it.
ldd output confirms that the ldd and libraries that we've just built are actually being used as expected:
+ ldd test_glibc.out
linux-vdso.so.1 (0x00007ffe4bfd3000)
libpthread.so.0 => /home/ciro/glibc/build/install/lib/libpthread.so.0 (0x00007fc12ed92000)
libc.so.6 => /home/ciro/glibc/build/install/lib/libc.so.6 (0x00007fc12e9dc000)
/home/ciro/glibc/build/install/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007fc12f1b3000)
The gcc compilation debug output shows that my host runtime objects were used, which is bad as mentioned previously, but I don't know how to work around it, e.g. it contains:
COLLECT_GCC_OPTIONS=/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crt1.o
Setup 1: modify glibc
Now let's modify glibc with:
diff --git a/nptl/thrd_create.c b/nptl/thrd_create.c
index 113ba0d93e..b00f088abb 100644
--- a/nptl/thrd_create.c
+++ b/nptl/thrd_create.c
## -16,11 +16,14 ##
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
+#include <stdio.h>
+
#include "thrd_priv.h"
int
thrd_create (thrd_t *thr, thrd_start_t func, void *arg)
{
+ puts("hacked");
_Static_assert (sizeof (thr) == sizeof (pthread_t),
"sizeof (thr) != sizeof (pthread_t)");
Then recompile and re-install glibc, and recompile and re-run our program:
cd glibc/build
make -j `nproc`
make -j `nproc` install
./test_glibc.sh
and we see hacked printed a few times as expected.
This further confirms that we actually used the glibc that we compiled and not the host one.
Tested on Ubuntu 18.04.
Setup 2: crosstool-NG pristine setup
This is an alternative to setup 1, and it is the most correct setup I've achieved far: everything is correct as far as I can observe, including the C runtime objects such as crt1.o, crti.o, and crtn.o.
In this setup, we will compile a full dedicated GCC toolchain that uses the glibc that we want.
The only downside to this method is that the build will take longer. But I wouldn't risk a production setup with anything less.
crosstool-NG is a set of scripts that downloads and compiles everything from source for us, including GCC, glibc and binutils.
Yes the GCC build system is so bad that we need a separate project for that.
This setup is only not perfect because crosstool-NG does not support building the executables without extra -Wl flags, which feels weird since we've built GCC itself. But everything seems to work, so this is only an inconvenience.
Get crosstool-NG and configure it:
git clone https://github.com/crosstool-ng/crosstool-ng
cd crosstool-ng
git checkout a6580b8e8b55345a5a342b5bd96e42c83e640ac5
export CT_PREFIX="$(pwd)/.build/install"
export PATH="/usr/lib/ccache:${PATH}"
./bootstrap
./configure --enable-local
make -j `nproc`
./ct-ng x86_64-unknown-linux-gnu
./ct-ng menuconfig
The only mandatory option that I can see, is making it match your host kernel version to use the correct kernel headers. Find your host kernel version with:
uname -a
which shows me:
4.15.0-34-generic
so in menuconfig I do:
Operating System
Version of linux
so I select:
4.14.71
which is the first equal or older version. It has to be older since the kernel is backwards compatible.
Now you can build with:
env -u LD_LIBRARY_PATH time ./ct-ng build CT_JOBS=`nproc`
and now wait for about thirty minutes to two hours for compilation.
Setup 2: optional configurations
The .config that we generated with ./ct-ng x86_64-unknown-linux-gnu has:
CT_GLIBC_V_2_27=y
To change that, in menuconfig do:
C-library
Version of glibc
save the .config, and continue with the build.
Or, if you want to use your own glibc source, e.g. to use glibc from the latest git, proceed like this:
Paths and misc options
Try features marked as EXPERIMENTAL: set to true
C-library
Source of glibc
Custom location: say yes
Custom location
Custom source location: point to a directory containing your glibc source
where glibc was cloned as:
git clone git://sourceware.org/git/glibc.git
cd glibc
git checkout glibc-2.28
Setup 2: test it out
Once you have built he toolchain that you want, test it out with:
#!/usr/bin/env bash
set -eux
install_dir="${CT_PREFIX}/x86_64-unknown-linux-gnu"
PATH="${PATH}:${install_dir}/bin" \
x86_64-unknown-linux-gnu-gcc \
-Wl,--dynamic-linker="${install_dir}/x86_64-unknown-linux-gnu/sysroot/lib/ld-linux-x86-64.so.2" \
-Wl,--rpath="${install_dir}/x86_64-unknown-linux-gnu/sysroot/lib" \
-v \
-o test_glibc.out \
test_glibc.c \
-pthread \
;
ldd test_glibc.out
./test_glibc.out
Everything seems to work as in Setup 1, except that now the correct runtime objects were used:
COLLECT_GCC_OPTIONS=/home/ciro/crosstool-ng/.build/install/x86_64-unknown-linux-gnu/bin/../x86_64-unknown-linux-gnu/sysroot/usr/lib/../lib64/crt1.o
Setup 2: failed efficient glibc recompilation attempt
It does not seem possible with crosstool-NG, as explained below.
If you just re-build;
env -u LD_LIBRARY_PATH time ./ct-ng build CT_JOBS=`nproc`
then your changes to the custom glibc source location are taken into account, but it builds everything from scratch, making it unusable for iterative development.
If we do:
./ct-ng list-steps
it gives a nice overview of the build steps:
Available build steps, in order:
- companion_tools_for_build
- companion_libs_for_build
- binutils_for_build
- companion_tools_for_host
- companion_libs_for_host
- binutils_for_host
- cc_core_pass_1
- kernel_headers
- libc_start_files
- cc_core_pass_2
- libc
- cc_for_build
- cc_for_host
- libc_post_cc
- companion_libs_for_target
- binutils_for_target
- debug
- test_suite
- finish
Use "<step>" as action to execute only that step.
Use "+<step>" as action to execute up to that step.
Use "<step>+" as action to execute from that step onward.
therefore, we see that there are glibc steps intertwined with several GCC steps, most notably libc_start_files comes before cc_core_pass_2, which is likely the most expensive step together with cc_core_pass_1.
In order to build just one step, you must first set the "Save intermediate steps" in .config option for the intial build:
Paths and misc options
Debug crosstool-NG
Save intermediate steps
and then you can try:
env -u LD_LIBRARY_PATH time ./ct-ng libc+ -j`nproc`
but unfortunately, the + required as mentioned at: https://github.com/crosstool-ng/crosstool-ng/issues/1033#issuecomment-424877536
Note however that restarting at an intermediate step resets the installation directory to the state it had during that step. I.e., you will have a rebuilt libc - but no final compiler built with this libc (and hence, no compiler libraries like libstdc++ either).
and basically still makes the rebuild too slow to be feasible for development, and I don't see how to overcome this without patching crosstool-NG.
Furthermore, starting from the libc step didn't seem to copy over the source again from Custom source location, further making this method unusable.
Bonus: stdlibc++
A bonus if you're also interested in the C++ standard library: How to edit and re-build the GCC libstdc++ C++ standard library source?
Link with -static. When you link with -static the linker embeds the library inside the executable, so the executable will be bigger, but it can be executed on a system with an older version of glibc because the program will use it's own library instead of that of the system.
In my opinion, the laziest solution (especially if you don't rely on latest bleeding edge C/C++ features, or latest compiler features) wasn't mentioned yet, so here it is:
Just build on the system with the oldest GLIBC you still want to support.
This is actually pretty easy to do nowadays with technologies like chroot, or KVM/Virtualbox, or docker, even if you don't really want to use such an old distro directly on any pc. In detail, to make a maximum portable binary of your software I recommend following these steps:
Just pick your poison of sandbox/virtualization/... whatever, and use it to get yourself a virtual older Ubuntu LTS and compile with the gcc/g++ it has in there by default. That automatically limits your GLIBC to the one available in that environment.
Avoid depending on external libs outside of foundational ones: like, you should dynamically link ground-level system stuff like glibc, libGL, libxcb/X11/wayland things, libasound/libpulseaudio, possibly GTK+ if you use that, but otherwise preferrably statically link external libs/ship them along if you can. Especially mostly self-contained libs like image loaders, multimedia decoders, etc can cause less breakage on other distros (breakage can be caused e.g. if only present somewhere in a different major version) if you statically ship them.
With that approach you get an old-GLIBC-compatible binary without any manual symbol tweaks, without doing a fully static binary (that may break for more complex programs because glibc hates that, and which may cause licensing issues for you), and without setting up any custom toolchain, any custom glibc copy, or whatever.
An alternative is to just discard the version info and let the linker default to whatever version it has.
To do this, you may want to check out PatchELF1:
$ nm --dynamic --undefined-only --with-symbol-versions MyLib.so \
| grep GLIBC | sed -e 's#.\+###' | sort --unique
GLIBC_2.17
GLIBC_2.29
$ nm --dynamic --undefined-only --with-symbol-versions MyLib.so | grep GLIBC_2.29
U exp#GLIBC_2.29
U log#GLIBC_2.29
U log2#GLIBC_2.29
U pow#GLIBC_2.29
$ patchelf --clear-symbol-version exp \
--clear-symbol-version log \
--clear-symbol-version log2 \
--clear-symbol-version pow MyLib.so
This is extremely useful if you don't have the source at hand (or have a hard time understanding the code2).
1 Although patchelf is available on many distributions, they are very likely to be outdated.
The --clear-symbol-version flag was added in version 0.12, which, unfortunately, does not fully remove version requirements.
You will need to compile manually from this merge request before it gets merged.
2 FYI, I was cross-compiling LuaJIT.
This repo:
https://github.com/wheybags/glibc_version_header
provides a header file that takes care of the details described in the accepted answer.
Basically:
Download the header of the corresponding GCC you want to link against
Add -include /path/to/header.h to your compiler flags
You may also need to add -D_REENTRANT if you're linking pthread
I also add the linker flags:
-static-libgcc -static-libstdc++ -pthread
But those are dependent on your app's requirements.

Resources