KGDB remote debugging error - linux

I am trying to debug the Linux kernel using kgdb. I am using qemu as target machine. Ubuntu 12.04 is my host OS. I tried this command
cyborg#skynet:~$ qemu-system-arm -M versatilepb -m 128M -kernel zImage -initrd rootfs.img.gz -append "root=/dev/ram rdinit=/bin/sh kgdboc=ttyAMA0,115200 kgdbwait"
It will boot and wait displaying
kgdb: Waiting for connection from remote gdb
And when I try to debug using gdb on linux host using the commands
cyborg#skynet: arm-none-linux-gnueabi-gdb vmlinux
(gdb): set remotebaud 115200
(gdb) target remote /dev/ttyS0
Remote debugging using /dev/ttyS0
Ignoring packet error, continuing...
warning: unrecognized item "timeout" in "qSupported" response
Ignoring packet error, continuing...
Ignoring packet error, continuing...
Ignoring packet error, continuing...
Ignoring packet error, continuing...
Malformed response to offset query, timeout
I am not really getting what I am missing !
P.S. I compiled the linux kernel 2.6.39.4 with KGDB options enabled. And I followed these instructions to run kernel on qemu.

If you reference /dev/ttyS0 on the host gdb will try to use the physical serial port present. You have to connect qemu's emulated serial port to gdb instead.
Reading the qemu manual, apparently you can redirect the emulated port to tcp networking, by adding something like: -serial tcp::1234,server to the qemu command line. In gdb you can then connect using target remote :1234.

There is no real reason to use kgdb when your running within Qemu. You can simply specify something like "-gdb tcp::8888" on the Qemu command line and activate the GDB stub. You can then connect to the stub with an ARM aware GDB:
target remote:8888
And debug away. You may find the following gdb helper scripts useful

Related

Cannot start J-Link GDB Server on Ubuntu

I am currently working on the ARM Cortex-M4 inside the NXP i.MX8M Mini.
I am able to compile a project for M4 on Eclipse IDE on an Ubuntu VM.
I would now like to debug on the M4 via a SEGGER Flasher ARM probe, still from Ubuntu.
My probe is well recognized by Ubuntu, and I can launch the J-Link GDB server by simply typing the command :
$ sudo ./JLinkGDBServerCLExe
However, if I type the same command without sudo, I get :
$ ./JLinkGDBServerCLExe
SEGGER J-Link GDB Server V7.58b Command Line Version
JLinkARM.dll V7.58b (DLL compiled Nov 16 2021 15:04:27)
-----GDB Server start settings-----
GDBInit file: none
GDB Server Listening port: 2331
SWO raw output listening port: 2332
Terminal I/O port: 2333
Accept remote connection: yes
Generate logfile: off
Verify download: off
Init regs on start: off
Silent mode: off
Single run mode: off
Target connection timeout: 0 ms
------J-Link related settings------
J-Link Host interface: USB
J-Link script: none
J-Link settings file: none
------Target related settings------
Target device: Unspecified
Target interface: JTAG
Target interface speed: 4000kHz
Target endian: little
Connecting to J-Link...
Connecting to J-Link failed. Connected correctly?
GDBServer will be closed...
Shutting down...
Could not connect to J-Link.
Please check power, connection and settings.
My problem is that when I start eclipse, I get the same result as starting the GDB server without sudo.
It seems that this is a rights issue, how can I solve it?
As #KamilCuk said, the problem came from the udev rules.
So you just have to copy the rules provided by Segger with J-Link Software on the system:
$ sudo cp 99-jlink.rules /etc/udev/rules.d
Then you have to reboot the system:
$ reboot

Debug QEMU/KVM with gdb via a serial port

I tried to set-up QEMU/KVM to for GDB debuggin with a serial port and couldn't managed to make it work.
According to QEMU GDB documentation
If you want to specify which TCP port to use or to use something other
than TCP for the gdbstub connection, use the -gdb dev option instead
of -s.
Here are my attempts:
I. I looked at available tty devices and picked up the first one /dev/ttyS0
sudo qemu-system-x86_64 -gdb /dev/ttyS0 -S
In GDB I tried
(gdb) target remote /dev/ttyS0
get_tty_state failed: Input/output error
set_tty_state failed: Input/output error
Remote debugging using /dev/ttyS0
Remote communication error. Target disconnected.: Input/output error.
II. Create a chardev and pass it to -gdb option
sudo qemu-system-x86_64 -chardev serial,id=ser0,path=/dev/ttyS0 -gdb /dev/ttyS0 -S
GDB responds with the same error msg:
(gdb) target remote /dev/ttyS0
get_tty_state failed: Input/output error
set_tty_state failed: Input/output error
Remote debugging using /dev/ttyS0
Remote communication error. Target disconnected.: Input/output error.
III. Pass a chardev id to the -gdb option
$ sudo qemu-system-x86_64 -chardev serial,id=ser0,path=/dev/ttyS0 -gdb ser0 -S
qemu-system-x86_64: -gdb ser0: 'ser0' is not a valid char driver
This way QEMU does not start at all.
QUESTION: What is the correct way to set-up QEMU GDB debugging via a serial port?

Unable to load bnxt_en driver intermittently on linux os backed by hypervisor

I have a VM backed by vCenter.
vCenter ESXi have physical adapter "Broadcom BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller" and SR-IOV enabled on this.
VM is connected to 1mgmt network (vmxnet3) and 2 SR-IOV adapters (SRIOVPassthrough).
Upon booting of the VM, only 2 networks shown up. (1mgmt and 1SR-IOV).
Journalctl -k logs showed following error.
[ 4832.408471] bnxt_en 0000:13:00.0 (unnamed net_device) (uninitialized): Error (timeout: 500015) msg {0x0 0x0} len:0
[ 4832.408930] bnxt_en: probe of 0000:13:00.0 failed with error -1
Reboot of machine did not help at all.
For the successful one adapter
bnxt_en 0000:03:00.0 eth1: NIC Link is Up, 25000 Mbps full duplex, Flow control: ON - receive & transmit
bnxt_en 0000:03:00.0 eth1: FEC autoneg off encodings: None
I did rescan of the pci devices and did multiple times reboot without any success.
Any pointers would be really helpful
We've got a similar issue and were able to fix it.
In our case we had the same error message on Debian 10, 11 and Oracle Linux 8 but we installed it directly on hardware without an hypervisor.
But it could be the same issue cause you're using passthrough.
There are two ways to fix it:
Usage of UEFI Boot
Disable PXE Boot and keep Bios / Legacy Boot
Both options fixed it.
Disabling PXE didn't work for us, but we can get the ports back online, by running
echo 0000:af:00.0 > /sys/bus/pci/drivers/bnxt_en/bind
Where 0000:af:00.0 is the PCI number for the port, which can be gotten from dmesg | grep bnxt_en and looking for the port or ports that failed.

DPDK Error: "Cause: No Ethernet ports - bye"

The title is the last line of an error while trying to run a sample application of dpdk, named l2fwd. When I run the app by ./build/l2fwd -l 0-3 -n 4 -- -q 8 -p ffff but at last lines this error occurs:
EAL: Error - exiting with code: 1
Cause: No Ethernet ports - bye
According to dpdk documentation before running app I must execute a command: modprobe ixgbe max_vfs=2,2
I also have a problem with this command, but I'm not sure this is the main reason of failing app execution.
Error of modprobe command relates to ixgbe driver which I can't install it. The error after make of ixgbe is
cc1: error: code model kernel does not support PIC mode
Makefile:197: * * Aborting the build. *** This driver is not supported on kernel versions older than 2.4.0. Stop.
To you a NIC with DPDK we need to bind it to UIO or VFIO driver.
Please have a look at Binding and Unbinding Network Ports to/from the Kernel Modules in DPDK's getting started guide.

linux: running self compiled kernel in qemu: VFS: Unable to mount root fs on unknown wn-block(0,0)

I try to get this running and don't know what I'm doing wrong. I have created an Debian.img (disk in raw format with virtual device manager - gui to libvirt I guess) and installed debian with no troubles. Now I want to get this running with a self compiled kernel. I copied the .config-file from my working (virtual) debian and made no more changes at all. This is what I do:
qemu-system-x86_64 -m 1024M -kernel /path/to/bzImage -hda /var/lib/libvirt/images/Debian.img -append "root=/dev/sda1 console=ttyS0" -enable-kvm -nographic
But during boot I always get this error message.
[ 0.195285] Initializing network drop monitor service
[ 0.196177] List of all partitions:
[ 0.196641] No filesystem could mount root, tried:
[ 0.197292] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[ 0.198355] Pid: 1, comm: swapper/0 Not tainted 3.2.46 #7
[ 0.199055] Call Trace:
[ 0.199386] [<ffffffff81318c30>] ? panic+0x95/0x19e
[ 0.200049] [<ffffffff81680f7d>] ? mount_block_root+0x245/0x271
[ 0.200834] [<ffffffff8168112f>] ? prepare_namespace+0x133/0x169
[ 0.201590] [<ffffffff81680c94>] ? kernel_init+0x14c/0x151
[ 0.202273] [<ffffffff81325a34>] ? kernel_thread_helper+0x4/0x10
[ 0.203022] [<ffffffff81680b48>] ? start_kernel+0x3c1/0x3c1
[ 0.203716] [<ffffffff81325a30>] ? gs_change+0x13/0x13
What I'm doing wrong? Please someone help. Do I need to pass the -initrd option? I tried this already but had no luck yet.
I figured it out by myself. Some time has passed, but as I recall the solution was to provide an initial ramdisk. This is how I got it working with hardware acceleration.
Compiling
make defconfig
CONFIG_EXT4_FS=y
CONFIG_IA32_EMULATION=y
CONFIG_VIRTIO_PCI=y (Virtualization -> PCI driver for virtio devices)
CONFIG_VIRTIO_BALLOON=y (Virtualization -> Virtio balloon driver)
CONFIG_VIRTIO_BLK=y (Device Drivers -> Block -> Virtio block driver)
CONFIG_VIRTIO_NET=y (Device Drivers -> Network device support -> Virtio network driver)
CONFIG_VIRTIO=y (automatically selected)
CONFIG_VIRTIO_RING=y (automatically selected)
---> see http://www.linux-kvm.org/page/Virtio
Enable paravirt in config
Disable NMI watchdog on HOST for using performance counters on GUEST. You may ignore this.
cat /proc/sys/kernel/nmi_watchdog
---> see http://kvm.et.redhat.com/page/Guest_PMU
Start in Qemu
sudo qemu-system-x86_64 -m 1024M -hda /var/lib/libvirt/images/DEbian.img -enable-kvm -initrd /home/username/compiled_kernel/initrd.img-3.2.46 -kernel /home/username/compiled_kernel/bzImage -append "root=/dev/sda1 console=ttyS0" -nographic -redir tcp:2222::22 -cpu host -smp cores=2
Start in KVM
Kernal path: /home/username/compiled_kernel/bzImage
Initrd path: /home/username/compiled_kernel/initrd.img-3.2.46
Kernel arguments: root=/dev/sda1
Hope this helps if someone has the same issues.
This is for AArch64 (arm64) on QEMU case.
I was following this good tutorial: https://ibug.io/blog/2019/04/os-lab-1/
In my case I was met with this error message:
---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(1,0) ]---
I did mknod dev/ram b 1 0 in the initrd.
Later I noticed there was an error message above that line implying the kernel didn't support the ram disk. So I edited .config and set these items:
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=1
CONFIG_BLK_DEV_RAM_SIZE=131072 (= 128MB, the number is in unit of 1014B)
And then the problem was gone! The initrd was mounted on /dev/ram and the first init process ran well.
It turns out running make defconfig didn't set thses values by default for me.
maybe your system image file is bad and can not be mounted.
You may try these command to mount the image file and check if it is a valid root file system for linux.
losetup /dev/loop0 /var/lib/libvirt/images/Debian.img
kpartx -av /dev/loop0
mount /dev/mapper/loop0p1 /mnt/tmp
The most likely thing is that the kernel doesn't know the correct device to boot from.
You can supply this explicitly from the qemu command line. So if the root is on partition 2, you can say:
qemu -kernel /path/to/bzImage \
-append root=/dev/sda2 \
-hda /path/to/hda.img \
.
.
.
Notice I use /dev/sda2 even though the disk is IDE. Even virtual machines seem to use SATA nowadays.
The other possibilities are that as #Houcheng says, your root FS is corrupted, or else that the kernel does not have that particular FS type built in. But I think you would get a different error if that were the case.
QEMU version
QEMU emulator version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.11), Copyright (c) 2003-2008 Fabrice Bellard
running build-root 4.9.6 with the following arguments to be passed
qemu-system-x86_64 -kernel output/images/bzImage -hda output/images/rootfs.qcow2 -boot c -m 128 -append root=/dev/sda -localtime -no-reboot -name rtlinux -net nic -net user -redir tcp:2222::22 -redir tcp:3333::3333
was accepting only /dev/sda as an option for the root fs to mount (it will show you a little hint for the root fs option once it will boot and hang with the following error):
VFS: Cannot open root device "hda" or unknown-block(0,0): error -6
Please append a correct "root=" boot option; here are the available partitions:
0800 61440 sda driver: sd

Resources