I had compile a linux kernel and generating root file system using buildroot for 64-bit powerpc. Target CPU POWER7.
Output of buildroot:
1] rootfs.cpio 2] rootfs.ext2 3] rootfs.tar.gz 4] vmlinux
Start qemu simulation with
qemu-system-ppc64 -m 1024 -kernel output/images/vmlinux -initrd output/images/rootfs.cpio -serial stdio
--> Output:
Output Of 1st command
qemu-system-ppc64 -M pseries -cpu POWER7 -m 1024 -kernel output/images/vmlinux -append 'console=hvc0 root=/dev/sda' -drive file=output/images/rootfs.ext2,if=scsi,index=0,format=raw -serial stdio
--> Output:
Output of 2nd command
What have I done wrong and what can I do to fix it?
How long did you wait for it? I assume a while. In which case it looks like the kernel has crashed somewhere very early in boot, before it could detect the console. That can happen if you have far too little memory, but 1G should be enough. It can also happen if you build the kernel for the wrong machine/CPU type, but you seem to have gotten that right.
There's some instructions here that you could try and are known to work.
Buildroot
https://buildroot.org/
I haven't tested it for powerpc specifically, but I bet it will all just work out of the box just like it worked for every arch I've tried so far (x86, ARM and MIPS) :-)
Just follow the steps explained at: How to download the Torvalds Linux Kernel master, (re)compile it, and boot it with QEMU? and replace arm with ppc.
Then to use your own Linux kernel source, you basically just have to use LINUX_OVERRIDE_SRCDIR as explained at: How to modify the source of Buildroot packages for package development?
Edit: I knew it was just a formality, but I actually tested the commands below after writing this question and the boot worked out of the box as expected:
unset LD_LIBRARY_PATH
make qemu_ppc64_pseries_defconfig
printf '
BR2_CCACHE=y
BR2_PACKAGE_HOST_QEMU=y
BR2_PACKAGE_HOST_QEMU_LINUX_USER_MODE=n
BR2_PACKAGE_HOST_QEMU_SYSTEM_MODE=y
BR2_PACKAGE_HOST_QEMU_VDE2=y
' >>.config
make olddefconfig
time make BR2_JLEVEL="$(nproc)" HOST_QEMU_OPTS='--enable-sdl --with-sdlabi=2.0'
./output/host/usr/bin/qemu-system-ppc64 -M pseries -cpu POWER7 -m 256 -kernel output/images/vmlinux -append 'console=hvc0 root=/dev/sda' -drive file=output/images/rootfs.ext2,if=scsi,index=0,format=raw -serial stdio -display curses
Related
I used these options to indicate which console to use by qemu
-nographic -serial stdio -append "console=ttyS0"
linux boot normally until this step, and when I hit enter the qemu> appear as the next picture illustrate:
It was me who types stop and quit
why did linux stop and qemu appear. I want to work with linux and execute programs from linux bin?
Im trying to debug the boot sequence of a linux kernel with qemu,
the command i'm running is:
qemu -serial stdio -kernel <path to kernel> -hda <path to rootfs> -append "root=/dev/sda terminal = ttyS0"
During boot all the kernel messages are printed to the QEMU window.
Only when the boot has finished i get my prompt back to the terminal i ran QEMU in.
Now i can start using the kernel terminal I'm running and seeing the output in the terminal and not in QEMU window.
How do i get all messages including the boot messages to my terminal and not to QEMU window (because i cant scroll up in that window..) ?
remove -serial stdio parameter
add -nographic parameter
and change the kernel parameter terminal = ttyS0 to console=ttyS0. This should do the trick.
qemu -nographic -kernel ./bzImage -hda ./image.hda -append root=/dev/sda console=ttyS0
You may want to check the script I use for kernel development: https://github.com/arapov/wrap-qemukvm (it's not very "production", but you can find useful qemu cli parameters there)
refer this Redirect Qemu console to a file or the host terminal?
to get the log on both qemu and your terminal .use
"console=ttyAMA0,115200 console=tty highres=off console=ttyS0"
Redirecting Qemu output to terminal might cause some problem (personally i don't like it). You can using options like -noframe (this will create new window but won't any window frame) or -curses to experience qemu output on terminal.
When the run following command on the teriminal
qemu-system-arm -M versatilepb -m 128M -kernel /home/<name>/linux-3.10.10/arch/arm/boot/uImage
A window with black screen will be opened and remain blank with no messages, but I am expecting Kernel messages on this screen. Please suggest a solution.
You have no console. Use '-append "console=tty1"' as shown below. Append gives command line arguments to kernel. Also, after the console issue is fixed, you will probably see a need for a root filesystem; many options exist there, I use a previously-mounted disk image, I have shown a commandline with both below.
qemu-system-arm -M versatilepb -m 128M -kernel /home/<name>/linux-3.10.10/arch/arm/boot/uImage -append "console=tty1 root=/dev/mmcblk0p1 rootfstype=ext3" -drive file=disk.img,if=sd
If you are trying to emulate a Raspberry Pi, use follow the advice in Joe Kul's answer, but instead of tty1, use ttyAMA0 instead.
I'm new to kernel development and I would like to know how to run/debug the linux kernel using QEMU and gdb. I'm actually reading Robert Love's book but unfortunately it doesn't help the reader on how to install proper tools to run or debug the kernel... So what I did was to follow this tutorial http://opensourceforu.efytimes.com/2011/02/kernel-development-debugging-using-eclipse/. I'm using eclipse as an IDE to develop on the kernel but I wanted first to get it work under QEMU/gdb. So what I did so far was:
1) To compile the kernel with:
make defconfig (then setting the CONFIG_DEBUG_INFO=y in the .config)
make -j4
2) Once the compilation is over I run Qemu using:
qemu-system-x86_64 -s -S /dev/zero -kernel /arch/x86/boot/bzImage
which launch the kernel in "stopped" state
3) Thus I have to use gdb, I try the following command:
gdb ./vmlinux
which run it correctly but... Now I don't know what to do... I know that I have to use remote debugging on the port 1234 (default port used by Qemu), using the vmlinux as the symbol table file for debugging.
So my question is: What should I do to run the kernel on Qemu, attach my debugger to it and thus, get them work together to make my life easier with kernel development.
I'd try:
(gdb) target remote localhost:1234
(gdb) continue
Using the '-s' option makes qemu listen on port tcp::1234, which you can connect to as localhost:1234 if you are on the same machine. Qemu's '-S' option makes Qemu stop execution until you give the continue command.
Best thing would probably be to have a look at a decent GDB tutorial to get along with what you are doing. This one looks quite nice.
Step-by-step procedure tested on Ubuntu 16.10 host
To get started from scratch quickly I've made a minimal fully automated QEMU + Buildroot example at: https://github.com/cirosantilli/linux-kernel-module-cheat/blob/c7bbc6029af7f4fab0a23a380d1607df0b2a3701/gdb-step-debugging.md Major steps are covered below.
First get a root filesystem rootfs.cpio.gz. If you need one, consider:
a minimal init-only executable image: https://unix.stackexchange.com/questions/122717/custom-linux-distro-that-runs-just-one-program-nothing-else/238579#238579
a Busybox interactive system: https://unix.stackexchange.com/questions/2692/what-is-the-smallest-possible-linux-implementation/203902#203902
Then on the Linux kernel:
git checkout v4.15
make mrproper
make x86_64_defconfig
cat <<EOF >.config-fragment
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_KERNEL=y
CONFIG_GDB_SCRIPTS=y
EOF
./scripts/kconfig/merge_config.sh .config .config-fragment
make -j"$(nproc)"
qemu-system-x86_64 -kernel arch/x86/boot/bzImage \
-initrd rootfs.cpio.gz -S -s \
-append nokaslr
On another terminal, from inside the Linux kernel tree, supposing you want to start debugging from start_kernel:
gdb \
-ex "add-auto-load-safe-path $(pwd)" \
-ex "file vmlinux" \
-ex 'set arch i386:x86-64:intel' \
-ex 'target remote localhost:1234' \
-ex 'break start_kernel' \
-ex 'continue' \
-ex 'disconnect' \
-ex 'set arch i386:x86-64' \
-ex 'target remote localhost:1234'
and we are done!!
For kernel modules see: How to debug Linux kernel modules with QEMU?
For Ubuntu 14.04, GDB 7.7.1, hbreak was needed, break software breakpoints were ignored. Not the case anymore in 16.10. See also: https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/901944
The messy disconnect and what come after it are to work around the error:
Remote 'g' packet reply is too long: 000000000000000017d11000008ef4810120008000000000fdfb8b07000000000d352828000000004040010000000000903fe081ffffffff883fe081ffffffff00000000000e0000ffffffffffe0ffffffffffff07ffffffffffffffff9fffff17d11000008ef4810000000000800000fffffffff8ffffffffff0000ffffffff2ddbf481ffffffff4600000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007f0300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000801f0000
Related threads:
https://sourceware.org/bugzilla/show_bug.cgi?id=13984 might be a GDB bug
Remote 'g' packet reply is too long
http://wiki.osdev.org/QEMU_and_GDB_in_long_mode osdev.org is as usual an awesome source for these problems
https://lists.nongnu.org/archive/html/qemu-discuss/2014-10/msg00069.html
nokaslr: https://unix.stackexchange.com/questions/397939/turning-off-kaslr-to-debug-linux-kernel-using-qemu-and-gdb/421287#421287
Known limitations:
the Linux kernel does not support (and does not even compile without patches) with -O0: How to de-optimize the Linux kernel to and compile it with -O0?
GDB 7.11 will blow your memory on some types of tab completion, even after the max-completions fix: Tab completion interrupt for large binaries Likely some corner case which was not covered in that patch. So an ulimit -Sv 500000 is a wise action before debugging. Blew up specifically when I tab completed file<tab> for the filename argument of sys_execve as in: https://stackoverflow.com/a/42290593/895245
See also:
https://github.com/torvalds/linux/blob/v4.9/Documentation/dev-tools/gdb-kernel-debugging.rst official Linux kernel "documentation"
Linux kernel live debugging, how it's done and what tools are used?
When you try to start vmlinux exe using gdb, then first thing on gdb is to issue cmds:
(gdb) target remote localhost:1234
(gdb) break start_kernel
(continue)
This will break the kernel at start_kernel.
BjoernID's answer did not really work for me. After the first continuation, no breakpoint is reached and on interrupt, I would see lines such as:
0x0000000000000000 in ?? ()
(gdb) break rapl_pmu_init
Breakpoint 1 at 0xffffffff816631e7
(gdb) c
Continuing.
^CRemote 'g' packet reply is too long: 08793000000000002988d582000000002019[..]
I guess this has something to do with different CPU modes (real mode in BIOS vs. long mode when Linux has booted). Anyway, the solution is to run QEMU first without waiting (i.e. without -S):
qemu-system-x86_64 -enable-kvm -kernel arch/x86/boot/bzImage -cpu SandyBridge -s
In my case, I needed to break at something during boot, so after some deciseconds, I ran the gdb command. If you have more time (e.g. you need to debug a module that is loaded manually), then the timing doesn't really matter.
gdb allows you to specify commands that should be run when started. This makes automation a bit easier. To connect to QEMU (which should now already be started), break on a function and continue execution, use:
gdb -ex 'target remote localhost:1234' -ex 'break rapl_pmu_init' -ex c ./vmlinux
As for me the best solution for debugging the kernel - is to use gdb from Eclipse environment. You should just set appropriate port for gdb (must be the same with one you specified in qemu launch string) in remote debugging section. Here is the manual:
http://www.sw-at.com/blog/2011/02/11/linux-kernel-development-and-debugging-using-eclipse-cdt/
On Linux systems, vmlinux is a statically linked executable file that contains
the Linux kernel in one of the object file formats supported by Linux, which
includes ELF, COFF and a.out. The vmlinux file might be required for kernel
debugging, symbol table generation or other operations, but must be made
bootable before being used as an operating system kernel by adding a multiboot
header, bootsector and setup routines.
An image of this initial root file system must be stored somewhere accessible
by the Linux bootloader to the boot firmware of the computer. This can be the
root file system itself, a boot image on an optical disc, a small partition on
a local disk (a boot paratition, usually using ext4 or FAT file systems), or a
TFTP server (on systems that can boot from Ethernet).
Compile linux kernel
Build the kernel with this series applied, enabling CONFIG_DEBUG_INFO (but leave CONFIG_DEBUG_INFO_REDUCED off)
https://www.kernel.org/doc/html/latest/admin-guide/README.html
https://wiki.archlinux.org/index.php/Kernel/Traditional_compilation
https://lwn.net/Articles/533552/
Install GDB and Qemu
sudo pacman -S gdb qemu
Create initramfs
#!/bin/bash
# Os : Arch Linux
# Kernel : 5.0.3
INIT_DIR=$(pwd)
BBOX_URL="https://busybox.net/downloads/busybox-1.30.1.tar.bz2"
BBOX_FILENAME=$(basename ${BBOX_URL})
BBOX_DIRNAME=$(basename ${BBOX_FILENAME} ".tar.bz2")
RAM_FILENAME="${INIT_DIR}/initramfs.cpio.gz"
function download_busybox {
wget -c ${BBOX_URL} 2>/dev/null
}
function compile_busybox {
tar xvf ${BBOX_FILENAME} && cd "${INIT_DIR}/${BBOX_DIRNAME}/"
echo "[*] Settings > Build options > Build static binary (no shared libs)"
echo "[!] Please enter to continue"
read tmpvar
make menuconfig && make -j2 && make install
}
function config_busybox {
cd "${INIT_DIR}/${BBOX_DIRNAME}/"
rm -rf initramfs/ && cp -rf _install/ initramfs/
rm -f initramfs/linuxrc
mkdir -p initramfs/{dev,proc,sys}
sudo cp -a /dev/{null,console,tty,tty1,tty2,tty3,tty4} initramfs/dev/
cat > "${INIT_DIR}/${BBOX_DIRNAME}/initramfs/init" << EOF
#!/bin/busybox sh
mount -t proc none /proc
mount -t sysfs none /sys
exec /sbin/init
EOF
chmod a+x initramfs/init
cd "${INIT_DIR}/${BBOX_DIRNAME}/initramfs/"
find . -print0 | cpio --null -ov --format=newc | gzip -9 > "${RAM_FILENAME}"
echo "[*] output: ${RAM_FILENAME}"
}
download_busybox
compile_busybox
config_busybox
Boot Linux Kernel With Qemu
#!/bin/bash
KER_FILENAME="/home/debug/Projects/kernelbuild/linux-5.0.3/arch/x86/boot/bzImage"
RAM_FILENAME="/home/debug/Projects/kerneldebug/initramfs.cpio.gz"
qemu-system-x86_64 -s -kernel "${KER_FILENAME}" -initrd "${RAM_FILENAME}" -nographic -append "console=ttyS0"
$ ./qemuboot_vmlinux.sh
SeaBIOS (version 1.12.0-20181126_142135-anatol)
iPXE (http://ipxe.org) 00:03.0 C980 PCI2.10 PnP PMM+07F92120+07EF2120 C980
Booting from ROM...
Probing EDD (edd=off to disable)... o
[ 0.019814] Spectre V2 : Spectre mitigation: LFENCE not serializing, switching to generic retpoline
can't run '/etc/init.d/rcS': No such file or directory
Please press Enter to activate this console.
/ # uname -a
Linux archlinux 5.0.3 #2 SMP PREEMPT Mon Mar 25 10:27:13 CST 2019 x86_64 GNU/Linux
/ #
Debug Linux Kernel With GDB
~/Projects/kernelbuild/linux-5.0.3 ➭ gdb vmlinux
...
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
0xffffffff89a4b852 in ?? ()
(gdb) break start_kernel
Breakpoint 1 at 0xffffffff826ccc08
(gdb)
Display all 190 possibilities? (y or n)
(gdb) info functions
All defined functions:
Non-debugging symbols:
0xffffffff81000000 _stext
0xffffffff81000000 _text
0xffffffff81000000 startup_64
0xffffffff81000030 secondary_startup_64
0xffffffff810000e0 verify_cpu
0xffffffff810001e0 start_cpu0
0xffffffff810001f0 __startup_64
0xffffffff81000410 pvh_start_xen
0xffffffff81001000 hypercall_page
0xffffffff81001000 xen_hypercall_set_trap_table
0xffffffff81001020 xen_hypercall_mmu_update
0xffffffff81001040 xen_hypercall_set_gdt
0xffffffff81001060 xen_hypercall_stack_switch
0xffffffff81001080 xen_hypercall_set_callbacks
0xffffffff810010a0 xen_hypercall_fpu_taskswitch
0xffffffff810010c0 xen_hypercall_sched_op_compat
0xffffffff810010e0 xen_hypercall_platform_op
I need to write a bash script where i have to check whether the Linux kernel is 32 bit or 64 bit.
I am using uname -a command, and it gives me x86_64 result. But I believe I can not use it in the generic way because result may differ if some one using non x86 architecture.
How to check for 32-bit / 64-bit kernel for Linux?
The question is rather: what do you intend to achieve by knowing whether you are on 32 or 64? What are the consequences of being on a hypothetical 128-bit environment? And what part actually is being tested for N-bitness? A CPU may support running in 64-bit mode, but the environment be 32-bit. Furthermore, the environment itself may be a mixed-mode; consider running a 64-bit kernel with a 32-bit userspace (done on a handful of classic RISCs). And then, what if the userspace is not of a homogenous bitness/executable format? That is why getconf LONG_BIT is equally pointless to use, because it depends on how it was compiled.
$ /rt64/usr/bin/getconf LONG_BIT
64
$ /usr/bin/getconf LONG_BIT
32
$ file /usr/bin/getconf /rt64/usr/bin/getconf
/usr/bin/getconf: ELF 32-bit MSB executable, SPARC32PLUS, V8+ Required, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.4, not stripped
/rt64/usr/bin/getconf: ELF 64-bit MSB executable, SPARC V9, relaxed memory ordering, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.4, not stripped
$ uname -m
sparc64
You could query the system for the size of a long int:
getconf LONG_BIT
But I'm not sure if this is completely portable to all different architectures.
I came up with the following. It assumes that init is used (some distros have switched to other loaders, but it should be easy to get a list of reasonably often used ones) and that you use ELF, not a.out or other nowadays exotic executable format. These seem to be sane assumptions for most systems, but probably they may be broken in embedded systems, etc. Still, the general idea should be possible to adapt (get to the init process or equivalent and check its bitness using file). If you are running as root, instead of going through the file's path, you could use file $(sudo readlink -e /proc/1/exe) (PID 1 being init is probably more portable than assuming anything about its path).
if file /sbin/init | fgrep 'ELF 64-bit' &>/dev/null; then
echo "64-bit"
else
echo "not 64-bit"
fi
Search for the lm (long mode) flag in /proc/cpuinfo. If this is true, it means you have a 64 bit processor. A simple grep should give you this information.
Regarding the kernel version, you can always grep on the uname -a information. Better would be to find the source of the uname program so that we can discount the discrepancy due to malicious hostname.
A quite reliable way to determine the supported executable architecture:
A featured function to do this:
# Gets the supported executable architecture
# >: 32-bit | 64-bit
getRunArch() {
local arch
local IFS=' '
read -r _ arch _ <<< $(file --dereference --brief "$(which ls)")
echo -n "${arch}"
}
Testing this:
echo "Supported run-time architecture is: $(getRunArch)"
Supported run-time architecture is: 64-bit
Slightly more reliable than:
getconf LONG_BIT
Depend what you're looking for, I have machine installed in 32 bit on 64 bit proc, all of above would return 32 bit in my case
But if I look at hardware, lshw on Ubuntu (lshw -c cpu), the description of cpu clearly show I have a 64 bit CPU, so I could have install a 64 bit version of Ubuntu.
/proc/cpuinfo is also good source of info about hardware, like the lm flags stand for long mode.
Have a nice day.
Jack.
Grep for '64' in uname output
uname -a | grep 64