I have started reading the book Essential Linux Device Drivers. I am following the process for compiling and booting into a new kernel. However I am unable to boot into the newly compiled kernel - I select the new kernel in grub and then ..nothing...the screen just stays the same.
Here are the steps I am taking, as described in the book (I am using kernel 2.6.24 as that is what the book is based on) -
cd /usr/src/linux-2.6.24 (the base dir of the kernel I downloaded)
make clean
cp arch/x86/configs/i386_defconfig .config
make bzImage cp arch/x86/boot/bzImage /boot/vmlinuz
The book says that 'you might need to alert your bootloader about the arrival of the new boot image. If you are using the GRUB bootloader, it figures this out automatically'.
Well it didn't figure it out automatically in my case as there was no option to select this kernel in GRUB. So I did it manually by typing update-grub in the terminal. And this didn't work either. However once I changed the filename from vmlinuz to vmlinuz-2.6.24 and type update-grub it picked it up. So on page 11 of the book there seems to already be multiple things left out...
The book says to then reboot the machine.
Anyway, I am now able to select this kernel in GRUB but as I said above it doeesn't boot properly, the screen just goes blank and never changes. So am I missing something? I have followed the instructions in the book exactly.
Most automatic grub setups include kernel options to hide the messages generated as the kernel attempts to boot. For example:
/boot/vmlinuz-x.x.x.x-generic root=UUID=something ro quiet splash
While on the grub menu line of the kernel you wish to boot, press 'e' to temporarily edit it, and delete options such as quiet and splash (but leave the ro).
Then boot the temporarily modified line, and see the progress messages in order to get an idea where the boot is failing.
The default configuration probably requires an initrd (e.g. because the file system and root device needs their driver, from modules, which is what initrd provides).
You could either configure your kernel appropriately (carefully enabling as in-kernel, not in-modules, the essential drivers) for your particular hardware, or use a procedure to build a kernel package with its initrd.
You should find a procedure appropriate for your linux distribution and habits, e.g. something like this or that. I am using on Debian:
time fakeroot make-kpkg -j2 --initrd --revision=3.4.2 --append-to-version=-amd64 binary
Related
Debugging the Linux Kernel with kgdb over rs-232 needs several preparation steps. I found awesome documentation, but no single-source that is fully self-contained and summarizes all steps needed, does not explain for ages, and has been tested. And also covers Yocto.
Is there any source that covers all that is needed in one single and short description ?
I.e.:
What files are needed in the directory GDB is started from (e.g. kernel awareness, source, vmlinux) and how to get theese, where to put it ?
When and where to get a cross-gdb from ?
ALL kernel config options needed, also the not-obvious ones (like CONFIG_RANDOMIZE_BASE)
How to configure the serial ports
Explaining a working back and forth of breaking into debugee and debugger to get started.
Explaining one rock-solid option of stopping the kernel that runs everywhere.
Explaining how to get this done not only for PC-PC debugging, but also for Yocto targets.
Debugging the Linux Kernel via a Nullmodem-Cable:
It took me a while to get a kgdb connection with Linux kernel awareness fully running. I share my way of doing this with Ubuntu Eoan (optional: Yocto Warrior) in 2020 here:
Tested with:
Debugging a linux based Intel PC from an Intel MacBook running MacOS Catalina. Using the gdb from the Homebrew package "i386-elf-gdb“. (wituout „-tui“ option in GDB)
Debugging a linux based ARM target (i.mx6, Yocto) from a linux based Intel PC.
Prerequisites:
You need two computers and a serial nullmodem cable. Check the cable by firiing up a serial termianl (e.g. screen or putty) on both hosts, connecting to your serial port (e.g. /dev/ttyS0 or /dev/ttyUSB0) and print characters from each station to the other. Remember the /dev/tty ports you confirmed.
Preparation:
You need on the first debuggee computer, we call it „target":
Special kernel installed that contains symbols, kgdb support etc.
Learn how to compile and install a kernel and use in make menuconfig belows configuration. You can search for Sybmbols with F8 or the / key in menuconfig.
(E.g. wiki.ubuntu.com. There take care in the first paragraph to execute deb-src before apt-get :)
# CONFIG_SERIAL_KGDB_NMI is not set
CONFIG_CONSOLE_POLL=y
# CONFIG_DEBUG_INFO is not set
CONFIG_KGDB=y
CONFIG_KGDB_SERIAL_CONSOLE=y
# CONFIG_KGDB_TESTS is not set
# CONFIG_KGDB_KDB is not set
CONFIG_FRAME_POINTER=y
CONFIG_DEBUG_INFO=y
# CONFIG_DEBUG_INFO_REDUCED is not set
# CONFIG_DEBUG_INFO_SPLIT is not set
CONFIG_DEBUG_INFO_DWARF4=y
CONFIG_GDB_SCRIPTS=y
CONFIG_STRIP_ASM_SYMS=y
# CONFIG_RANDOMIZE_BASE is not set
(Note for advanced Yocto use, skip if you're debugging a PC:
In yocto I created in my layer a file: recipes-kernel/linux/linux-mainline_%.bbappend with the content:
FILESEXTRAPATHS_prepend := "${THISDIR}/files:"
SRC_URI += "file://kgdb.cfg“
And in files/kgdb.cfg I added the config fragment shown above (without the on ARM unavailable options CONFIG_RANDOMIZE_BASE and CONFIG_FRAME_POINTER)
)
You need on the second debugger computer, we call it „debugger pc":
Full kernel source code, same code you used to compile the kernel above. (If you compiled the .o and .ko objects in place and not in a build-folder you better not copy the directory from the other pc, where you called make etc. in, but then better grab fresh sources again.)
vmlinux file containing the symbols (lies in the kernel source root, or build folder on the highest level after kernel make).
vmlinux-gdb.py file that was made during the kernel build (also lies at the same position on the highest level.).
All scripts in the folder scripts/gdb (Folder scripts in the same toplevel-position. If you use a dedicated build folder use the script folder from there, not from the source folder.)
(Advanced: If both computers don’t match in CPU, like Intel and Arm, a cross-gdb build. Ignore if you're on Intel/AMD.)
Note for advanced Yocto use, I did something like (ignore if you debug a PC):
bitbake -c patch virtual/kernel #(apply the changed kernel config from above)
bitbake -f -c compile virtual/kernel #(unpack is not sufficient because of vmlinux-gdb.py)
mkdir ~/gdbenv
cp -a tmp/work-shared/phyboard-mira-imx6-14/kernel-source/. ~/gdbenv
cp tmp/work/phyboard_mira_imx6_14-phytec-linux-gnueabi/linux-mainline/4.19.100-phy1-r0.0/build/vmlinux ~/gdbenv
cp tmp/work/phyboard_mira_imx6_14-phytec-linux-gnueabi/linux-mainline/4.19.100-phy1-r0.0/build/vmlinux-gdb.py ~/gdbenv
mkdir ~/gdbenv/scripts
cp -r tmp/work/phyboard_mira_imx6_14-phytec-linux-gnueabi/linux-mainline/4.19.100-phy1-r0.0/build/scripts/gdb ~/gdbenv/scripts
Then (ignore if you're on a PC)
yocto bitbake -c populate_sdk [my-image]
Then (still ignore on PC) install the sdk .sh-installation file from your deploy directory on the debugger pc and start the environment as guided by the output of the install script (remember that command), then use "$GDB" for starting the cross-gdb instead of „gdb".
Debug execution
Launch on the debugger two console screens:
Console 1, ssh: +++++++++++++++++++++++++++++++++++++++
ssh user#192.168.x.y
sudo -s
echo ttyS0,9600n8 > /sys/module/kgdboc/parameters/kgdboc
echo 1 > /proc/sys/kernel/sysrq
echo g > /proc/sysrq-trigger
Console 2, local: ++++++++++++++++++++++++++++++++++++++++
cd ~/gdbenv
gdb -tui ./vmlinux
add-auto-load-safe-path ~/gdbenv
source ~/gdbenv/vmlinux-gdb.py
set serial baud 9600
target remote /dev/ttyS0 (use the tty port you confirmed in the beginning)
b [name of the c funtion you want to debug]
cont
Back to console 1, ssh: +++++++++++++++++++++++++++++++++++++++
[Now trigger the function, e.g. sudo modprobe yourFancyKernelModule]
Back to console 2, local: ++++++++++++++++++++++++++++++++++++++++
Now use gdb functions, like bt, step, next, finish ...
You can also use linux-aware commands. Call "apropos lx“ in gdb for a list of commands.
I have cloned the linux kernel repo on my arch hosted machine (host is ubuntu 16.04). Two weeks ago I was able to boot into the new kernel (it was 4.11.rc06 back then), then I did git pull and recompiled everything and it just hangs after "loading initial ramdisk image...".
So I tried git clean -xfd then make localmoduleconfig answering defaults for everything, then make then make modules_install then mkinitcpio -p linux.4.11.custom and of course sudo cp -v arch/x86_64/boot/bzImage /boot/vmlinuz-linux.4.11.custom.
After I verified it does indeed hang I tried more git pulls, more cleans, but nothing changed.
Running the same kernel from the same source on a real machine boots.
I could not find recorded bug in virtualbox or find an update for ubuntu.
Next I tried debugging it myself by adding to the grub's linux command: debug earlyprintk=vga,keep and even removing the initrd line adding noinitrd to the kernel, but I get no error. Just a screen with the grub's "echo" messages that stays like that forever.
How can I debug it?
Has anyone got any idea what can be done?
To check whether the kernel even starts I would use KDB (kernel's built-in debugger), and see if you get a prompt at startup.
For better debugging I would try to get KGDB (GDB for kernel) working.
You can actually activate both to have all options available. See following link for more information about this:
https://www.kernel.org/doc/htmldocs/kgdb/index.html
I compiled my Linux kernel according to the Linux Device Driver Chapter 4: Debugging Techniques. After I loaded my first hello world module and then checked the output by dmesg, however, all I can see is evbug: ........
I know I can turn off evbug's output by execute sudo rmmod evbug. But, obviously, it is inconvenient to execute this command after each reboot.
How could I disable this module's autoloading? I just want to load it manually when I need it.
You need to blacklist the module. For debian systems see https://wiki.debian.org/KernelModuleBlacklisting. For redhat systems see https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Installation_Guide/rescuemode_drivers-blacklisting.html
I want to make some modifications in the kvm module in my Linux kernel. For this, I did the following:
Obtained the Kernel sources:
apt-get source linux-image-source-$(uname-r)
Modified a function in the file emulate.c - changed some variables and added a printk statement.
Built the kernel:
fakeroot debian/rules clean
fakeroot debian/rules binary-headers binary generic
Installed the packages produced as a result of building:
sudo dpkg -i linux*(version)*.deb
Rebooted the system.
Executed QEMU with kvm enabled.
However the changes I made, didn't seem to reflect when I try to test them in QEMU. Neither did the printk statement execute when I checked with dmesg.
Can anyone please point out which part I am getting wrong?
Installing a new kernel wont necessarily make it boot into it, you may need to change the default in your boot loader. (e.g. grub)
You can check whether the kernel you just compiled and installed is the same as what you booted with using:
cat /proc/version
If this is not as expected, then you need to tweak your Grub config and/or reboot and manually choose the correct kernel.
Having established the correct kernel, you may need to change the level of which messages are reported by the kernel (via dmesg)
This is controlled by a kernel proc file. You can see what the current values are by printing this file:
cat /proc/sys/kernel/printk
Example output:
4 4 1 7
The first argument - messages with severity < 4 (i.e. 0, 1 2 or 3) will be recorded.
The second argument - messages with no specified severity default to 4 and thus not seen by the system in the above example.
So the following will change the log so that all kernel messages are seen:
echo 8 > /proc/sys/kernel/printk
See (for example) http://www.makelinux.net/books/lkd2/ch18lev1sec3 for further information.
I have placed a faulty kernel object in the rc.local. Because of this faulty kernel object,the system crashes on bootup. Now, my aim is to remove that faulty ko insmod from rc.user. However, I cannot access rc.user as my system crashes on bootup. How can I fix this issue?
If you are getting the grub screen then you can
go to runlevel 1 and change the file.
On grub screen you will have to
press a after selecting the Fedora kernel with which you want to boot.
APPEND single after space in that line and press enter
press b to boot in single user mode
You can edit any file in this mode.
The safest & easiest approach when I get into such troubles is to mount the HD (contaning the faulty kernel) as an external hard drive to another linux machine and manually edit the files causing trouble.
Do a fdisk -l with your HD plugged in (thorugh USB). Take notice of the </dev/sd*#>.
mount /media /dev/<sd*#>
Now, you can access your boot or root partition to access the files.