is there a simplest possible way to enable linux kernel driver dev_dbg debug messages (actually it's a trace style messages) hopefully without messing up with the kernel patching/recompiling or the driver implementing something extra like debugfs? perhaps there is a way to enable something SIMPLE in the kernel (like one flag?) triggering particular driver or all drivers dev_dbg (it can be filtered with the `dmesg|grep "driverName") output?
the kernel version is 4.14.
there is NO syslog/daemonlog/system log running at all. there is NO network interface and only single serial port is available. the target system is very slow and is very compact so there is NO WAY to add syslog/etc, there is nothing but dmesg where exactly would be good to see the output of the lines like:
dev_dbg(&client->dev, "bla bla bla\n");
some posts already suggested to add debug keyword for the bootargs kernel parameters unfortunately wasn't enough.
the outputs like dev_info are getting into the dmesg with no issue so it's definitely something close. thanks
the simplest way to receive dev_dbg messages without installing/configuring syslog/etc, appeared necessary to do following steps:
provide debug key into bootargs kernel parameters
append #define DEBUG at the first line of the driver file - if the driver is a single file and is using a common Makefile, or append -DDEBUG inside the CC build options if the driver contains of multiple source files and as usually has it's own Makefile
upon the kernel booted and the prompt appear to enable debug level messages by executing either dmesg -n 8 or echo 8 > /proc/sys/kernel/printk
load the driver if the module with the command either insmod <driver name> or modprobe <driver name> or if the driver is integrated into the kernel the insertion commands may vary.
example on how to assign the kernel integrated driver for the i2c bus subsystem:
echo <driver name> <i2c bus address> > /sys/bus/i2c/devices/i2c-0/new_device
side notes:
if the DTS will have a driver record assignment, manually repeated driver assignment will cause the error - in case of i2c subsystem - error EBUSY (-16), the driver will be assigned way before the command prompt and the dmesg messages will be limited to the default level (usually dev_info only)
in case if the driver has been already assigned by DTS and there is no way to exclude it temporary from the tree source - it's useful to detach and reattach it once again after the debug (trace) level messages activated
for the i2c subsystem it would require to execute a command:
echo <driver name> > /sys/bus/i2c/drivers/<drivername>/unbind
then
echo <driver name> > /sys/bus/i2c/drivers/<drivername>/bind
warning:
the kernel drivers trace mechanism will not help on debugging internal driver improper configured or missing service structures. i.e. if the driver is loaded but remain silent with no trace messages means the probe has never been executed because of some kernel expected service structures information were missing or faulty
You need to follow below three steps.
1. Make sure that your kernel is complied with CONFIG_DYNAMIC_DEBUG=y
cat /proc/config.gz | gunzip | grep CONFIG_DYNAMIC_DEBUG
If not then recompile your kernel with CONFIG_DYNAMIC_DEBUG=y
2) After boot up check that debugfs is mounted somewhere or not.
mount | grep debugfs
mostly it get mounted in /sys/kernel/debug if not then you can manually mount it anywhere like below
mount -t debugfs none /sys/kernel/debug
3) Now enable the file name for which you need dev_dbg() logs.
echo 'file <driverfilename.c> +p'>/sys/kernel/debug/dynamic_debug/control
some more commands to play with dynamic_debug/control are at https://www.kernel.org/doc/html/v4.11/admin-guide/dynamic-debug-howto.html
now you should get your debug.
If you still do not see your message then enable prink level
echo "8 4 1 7" > /proc/sys/kernel/printk
If the CONFIG_DYNAMIC_DEBUG option is not set, then dev_dbg/pr_debug can be turned into normal printk() statements with KERN_DEBUG level.
But for that you need to add #define DEBUG at beginning of file.
or add -DDEBUG while compiling
or Enable CONFIG_[SUBSYSTEM]_DEBUG=y
I used below command to turn on the Kernel log for Android. I think for Linux to it should work:
echo 'file <driverfilename.c> +p'>/sys/kernel/debug/dynamic_debug/control
You can see the driver logs in "dmesg"
Thanks
MJ
Adding "debug" to the kernel parameters will lift the kernel events log level to the KERN_DEBUG (level 7) that will pollute the kmsg buffer with debug messages for the entire kernel code and it's not usually what we want while debugging our kernel module. The same is right when we change the current kernel log level via /proc/sys/kernel/printk.
To gain fine-grained control of all log levels above the KERN_ERR for your kernel module only without messing with global kernel log level configuration just add the following lines at the beginning of your module code:
#undef dev_dbg
#undef dev_info
#undef dev_warn
#undef dev_notice
#define dev_dbg dev_err
#define dev_info dev_err
#define dev_warn dev_err
#define dev_notice dev_err
You can mount the debug filesystem manually by executing the following command:
mount -t debugfs none /sys/kernel/debug
Once done, do:
ls /sys/kernel/debug
All debug information and message would be stored there without having to re-compile.
Also, you can add the this under your /etc/fstab to make the mount go automatic when you reboot.
... If you are re-compiling your kernel anyway, then you could enable " [*] Debug File System" under "Kernel Hacking"
Good luck, I hope all goes well.
It's likely that the driver you're interested in has exported the Kconfig option to enable the debug messages. In such case, you can enable debug output from just one driver by setting the DEBUG_YOUR_DRIVER=y or YOUR_DRIVER_DEBUG=y in menuconfig and rebuilding the kernel. Search for the driver name and you will see if this is available.
For example, for NXP/Freescale i.MX pinctrl driver, I just changed DEBUG_PINCTRL to y and looked into dmesg to get the debug logs from that driver.
This was the fastest approach for me.
Related
I'm building a custom linux image, using a non-manipulated Linux kernel 2.6.32.65.
the kernel boots just fine until it reaches this:
EXT2-fs warning: mounting unchecked fs, running e2fsck is recommended
VFS: Mounted root (ext2 filesystem) on device 3:1.
Freeing unused kernel memory: 304k freed
init: Unable to mount /dev filesystem: No such device
init: ureadahead main process (983) terminated with status 5
init: console-setup main process (1052) terminated with status 1
I tried the solutions mentioned here although the error is not exactly the same, but no luck. I tried multiple "reference" .config files. I have been googling for a bit but I can't find anything with the same problem.
I'm running this custom image on gem5 simulator, with file system from ubuntu-core and a clean kernel. earlier in the output the kernel shows this:
hda: max request size: 128KiB
hda: 16514064 sectors (8455 MB), CHS=16383/16/63
hda: hda1
So the kernel is able to see partitions just fine. I don't think this is caused by something in the file system. maybe initrd? or the kernel itself? how can I fix it?
1.) The problem is not in devfs, it seems issue is console setup. 2.) This is init issue not a linux kernel issue. 3.) Try to pass /bin/sh instead of init to kernel cmd line
I have the same problem with custom built embedded Linux.
Check you have enabled devfs in kernel .config
# core filesystems
CONFIG_PROC_FS=y
CONFIG_SYSFS=y
## devfs
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
I generate image for a Freescale i.mx6 sabresd using Yocto.
When booting, to activate the hdmi I have to modify u-boot by editing the bootargs. I use the following command :
setenv mmcargs 'setenv bootargs console=ttymxc0,115200 root=/dev/mmcblk2p2 rootwait rw video=mxcfb0:dev=hdmi, 1920x1080M#60, if=RGB24'
I can use saveenv to not enter it at every boot, but I'd like to automatize it to make the deployment easier. So I have made a boot.scr. Here is the boot.txt :
setenv mmcargs 'setenv bootargs console=ttymxc0,115200 root=/dev/mmcblk2p2 rootwait rw video=mxcfb0:dev=hdmi, 1920x1080M#60, if=RGB24'
boot
And I create the boot.scr using mkimage -A arm -T script -O linux -d boot.txt boot.scr. But, when booting, it makes a loop (boot make uboot reload the configuration, where it reads boot so it begin again). Without boot... no boot. I've tried a lot of possibilities : boot 0x120000000, bootm, bootz, with a lot of options, nothing works.
Most likely, your boot fails because you never load a kernel from which you could boot from.
In most cases, if bootcmd loads a boot.scr, then it is up to your boot.scr script to load the kernel (and also a device tree assuming you are using a recent iMX6 kernel). Then your script may boot with this kernel or allow bootcmd to carry on with its boot sequence using the loaded kernel. The load commands would look something like:
loadfdt=fatload mmc ${mmcdev}:${mmcpart} ${fdt_addr} ${fdt_file}
loaduimage=fatload mmc ${mmcdev}:${mmcpart} ${loadaddr} ${uimage}
And the boot command should look something like:
run loaduimage; run loadfdt; bootm ${loadaddr} - ${fdt_addr}
You may already have an mmcboot command defined which already takes care of these.
I would remove the "boot" line from your script and instead load the kernel and device tree and then run bootm (or mmcboot) - basically try adding the above three lines to your script after setting mmcarg - you'll need to ensure all referenced variables are properly set (ie. mmcdev, mmpart, etc).
Using the answer from shibley, I've found a simpler way :
run loadimage
run mmcboot
He was right, mmcboot takes care of the booting of the kernel. But I have to run loadimage (not loaduimage) before.
I need to run a script when my frambe buffer (/dev/fb?) device is loaded. So, for testing, I used a udev rule (called 98-framebuffer.rules) like that:
KERNEL=="fb0", RUN+="/bin/touch /tmp/test"
The file is not being created when the system starts. What have I done wrong?
udev rules are loaded during the boot process. Check dmesg out after rebooting. Your rule is being executed after your mount.
I made a script in /etc/udev/rules.d/local.rules
SUBSYSTEM=="usb", SYSFS{idVendor=="b58e"}, SYSFS{idProduct=="9e84"}, ACTION=="add", RUN+="notify-send USB"
I then reload udev with
sudo udevadm control --reload-rules
I've tried to remove everything but subsystem and run. I've tried the run '=' instead of '+=', I've tired ATTR instead of SYSFS. I tried "sudo service udev restart" and "sudo reload udev". I unplug the device, then plug it in again and it does not run the action. I tried renaming it 70-local.rules and changing permissions to a+x. I've tried changing 'subsystem' to 'bus'. I've tried setting run to be "/path/test.sh" which has the same command.
I'm not an expert and this isn't an answer, but I've found the following steps useful in identifying the appropriate attributes to trigger on:
Locate the device path using udevadm, lsusb, or usb-devices. I normally just use lsusb and let tab completion in my shell guide me. In my case, the path is /dev/bus/usb/003/007.
Use udevadm to identify the device attributes for rule writing. In my case, I used udevadm info -a --attribute-walk --root --name=/dev/bus/usb/003/007.
Write the rule and check that it's triggering. In my case, I'm just changing the device owner to user "stephen" and it's very easy for me to check if it's working by using ls -l /dev/bus/usb/003/007. My rule for this case looks like: SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", OWNER="stephen". I have a similar rule that puzzled me for a little while because the subsystem was expecting ATTRS not ATTR, which is why I recommend walking the attributes. The rule in this latter case became: `SUBSYSTEM=="tty", ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6001", OWNER="stephen".
And, of course, man udev is always helpful. As you said, you should struggle to identify that your rule is triggering properly and may be best off just doing a quick ownership change on the device as I did for a first step. You can run into trouble with bad attributes or symbolic links sometimes and it's
it does not run the action
No, it runs the action. The problem is that it doesn't know where to send the notification, since there's no notification framework running when udev starts. You will need to send a DBus message across the system bus and have a user daemon catch the message and post a notification instead.
Why kgdb always start from kernel/kgdb.c:1749 lines "kgdb:waiting dor connection from remote gdb" just step on the way of kernel of Linux.
I want to start from the beginning.
My environment is:
PC ubuntu10.10
gdb-kernel 2.6.34.1
filesys made by busybox
VirtualMach is qemu
Following the tips from web searches, I have made my linux. I can use it smoothly but when I try to remote-gdb it the kernel always start from:
kernel/kgdb.c:1749 "kgdb:waiting for connection from remote gdb"
which is much too far away from the function start_kernel which I want to meet.
I am using the following:
qemu -kernel /usr/src/work/bzImage -append "root=/dev/sda kgdboc=ttyS0,115200 kgdbwait"
-boot c -hda /usr/src/work/busybox.img -k en-us -serial tcp::4321,server
gdb /usr/src/work/vmlinux (gdb) target remote localhost:4321
Then I add -S so it can start from the beginning. But when I gdb it there is still something wrong.
When I input the command next it doesn't go to the next line and go to other place. For example I set a breakpoint at init.c startkernel() after the next. It is in other file.
If "kgdb:waiting dor connection from remote gdb" isn't early enough for you, you're going to have to try something other than kgdb. Think about this: kgdb is a service provided by the kernel. You can't debug the kernel "from the beginning" because the kernel has to perform enough initialization for it to provide the kgdb service.
Fortunately, there's another option for you. According to this source, if you start qemu with the flags -s -S, qemu will start the system and wait for you to attach a debugger to localhost:1234 before it even loads the kernel. Is that early enough?