I am writing my own bootloader for aarch64 that must boot linux, and in order to execute it properly I need to follow the linux boot protocol.
Here are some memory mappings: located in my linker file
FLASH_START = 0x000100000;
RAM_START = 0x40000000;
TEXT_START = 0x40080000;
Here is the command I am using to lauch my virt, giving 4 cores and 2GB of RAM
qemu-system-aarch64 -nographic -machine virt -cpu cortex-a72 -kernel pflash.bin -initrd initramfs.cpio.gz -serial mon:stdio -m 2G -smp 4
The pflash.bin has the following layout:
dd if=/dev/zero of=pflash.bin bs=1M count=512
dd if=my_bootloader.img of=pflash.bin conv=notrunc bs=1M count=20
dd if=Kernel of=pflash.bin conv=notrunc bs=1M seek=50
Where: Kernel is the linux kernel image file,
and my_bootloader.img is simply the objcopy of the elf file:
aarch64-linux-gnu-objcopy -O binary my_bootloader.elf my_bootloader.img
And the elf file is created in the following manner:
aarch64-linux-gnu-ld -nostdlib -T link.ld my_bootloader.o -o my_bootloader.elf
Here is my_bootloader.S
.section ".text.startup"
.global _start
_start:
ldr x30, =STACK_TOP
mov sp, x30
ldr x0, =RAM_START
ldr x1, =0x43280000
br x1
ret
As you can see All I have done so far is:
Set up the stack
Presumably I have loaded the DTB to the x0 (just like the linux boot protocol demands)
Branch to the location of the linux kernel
So I have not yet loaded the initramfs.cpio.gz which contains the file system but I should already normally get at least some output from the kernal since the DTB was loaded.
My question is, have I loaded it correctly? And I guess that the simple answer is no. But basically I have no clue where qemu puts the dtb in my RAM, and after looking everywhere on the documentation I cannot seem to find this information.
I would much appreciate if someone could tell me where QEMU loads the dtb so I can put it into x0 and the kernel could gladly read it!
Thanks in advance!
Where the dtb (if any) is is board specific. The QEMU documentation for the Arm 'virt' board tells you where the DTB is:
https://www.qemu.org/docs/master/system/arm/virt.html#hardware-configuration-information-for-bare-metal-programming
However, your command line is incorrect. "-kernel pflash.bin" says "this file is a Linux kernel, boot it in the way that the Linux kernel should be booted". What you want is "load this file into the flash, and start up in the way that the CPU would start out of reset on real hardware". For that you want one of the other ways of loading a guest binary (-bios is probably simplest). And you probably don't want to pass QEMU a -initrd option, either, since that is intended for either (a) QEMU's builtin bootloader or (b) QEMU-aware bootloaders that know how to extract a kernel and initrd from the fw-cfg device.
PS: If you tell QEMU to provide more than one guest CPU then your bootloader will need to deal with the secondary CPUs. That either means using PSCI to start them up, or else handling the fact that all the CPUs start executing the same code out of reset (which one depends on how you choose to start QEMU). You're better off sticking to '-smp 1' to start off with, and come back and deal with SMP later.
Using 7th gen i5 processor, nasm, ld.
Written using general purpose registers and system calls, the program runs on Ubuntu. An example is a simple hello world program.
global _start
section .text
_start:
mov eax, 0x4 ; write(int fd, char *buf, int len)
mov ebx, 0x1 ; fd
mov ecx, message ; *buf
mov edx, message_length ; len
int 0x80 ; syscall
mov eax, 0x1 ; exit(int status)
mov ebx, 0x0 ; status
int 0x80 ; syscall
section .data
message db "hello world", 0xA
message_length equ $-message
nasm -f elf64 -o hello_world.o hello_world.s
ld hello_world.o -o hello_world
./hello_world
output: hello_world
However, the program written using the intel system instructions does not work.
global _start
section .text
_start:
CLI
HLT
nasm -f elf64 halt.s -o halt.o
ld halt.o -o halt
./halt
output: Segmentation fault (core dumped)
What prevents this code from compiling this way and running?
How can this code be compiled and run?
You have two questions:
What prevents this code from compiling this way and running?
Nothing prevents it from assembling, because it is valid code.
The reason that it does not run/throws a segmentation fault is, that you are attempting to run instructions that require certain privileges in user mode.
Have a look at HLT which description states
The HLT instruction is a privileged instruction.
Also look at CLI/STI, whose situation is more complicated (which is explained in the below article and the comments below), but is also mostly useful for kernel mode code. Wikipedia says
In all three cases, only privileged applications (usually the OS kernel) may modify IF[Interrupt flag]. [...] CLI and STI are [also] privileged instructions, which trigger a general protection fault if an unprivileged application attempts to execute it, [...]
That should answer your question regarding the difference between these two instructions and why HLT surely generates a GPF (General Protection Fault) executed in User Mode.
How can this code be compiled and run?
The only way to run this code is in privileged mode. So on Windows or Linux you'd have to code a kernel driver or code your own OS to make sensible use of these instructions.
The above only applies to Protected Mode or Long Mode code.
(Real mode code may always modify IF, the Interrupt flag)
I want to get a machine code of the running proccess by his PID for analysing malicious instructions, by using heuristic methods of data analysing.
All I need to know is list of current machine instructions and values of registers (EIP, EAX, EBX...).
I can use gdb for reach this goal gdb output, but is take a several problems:
I don't know how interact with gdb from my application;
malicious code can use some technics of debugger detection like this: http://www.ouah.org/linux-anti-debugging.txt
https://www.youtube.com/watch?v=UTVp4jpJoyc&list=LLw7XNcx80oj63tRYAg7hrsA
for windows;
Getting info from console output makes work of my application slower.
Is are any way to get this information by PID in Linux? Or maybe Windows?
you may have a look to gcore:
$ gcore
usage: gcore [-o filename] pid
so you can dump process core using its pid:
$ gcore 792
warning: Could not load vsyscall page because no executable was specified
0x00007f5f73998410 in ?? ()
Saved corefile core.792
and then open it in gdb:
$ gdb -c core.792
GNU gdb (GDB) Fedora 8.0.1-30.fc26
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
[...]
[New LWP 792]
Missing separate debuginfo for the main executable file
Try: dnf --enablerepo='*debug*' install /usr/lib/debug/.build-id/09/b9d38bb6291b6282de4a2692e45448828d50da
Core was generated by `./a.out'.
#0 0x00007f5f73998410 in ?? ()
(gdb) info registers
rax 0xfffffffffffffe00 -512
rbx 0x0 0
rcx 0x7f5f73998410 140047938061328
rdx 0x1 1
rsi 0x7ffd30683d73 140725415591283
rdi 0x3 3
rbp 0x7ffd30683d90 0x7ffd30683d90
rsp 0x7ffd30683d68 0x7ffd30683d68
r8 0x1d 29
r9 0x0 0
r10 0x3 3
r11 0x246 582
r12 0x4006d0 4196048
r13 0x7ffd30683e70 140725415591536
r14 0x0 0
r15 0x0 0
rip 0x7f5f73998410 0x7f5f73998410
eflags 0x246 [ PF ZF IF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0
or even using the binary image from /proc to get some symbols:
gdb -c core.792 /proc/792/exe
You may know that you can pass scripts to gdb, this can ease not having to interact with it from your binary (man gdb for more details).
if you don't want to use gdb directly you may try using ptrace() directly, but it is for sure more work.
For the anti debugging technics, well... they work and there is no easy way to handle them directly as far as I know, each one may be worked arounded manually, (patching binary, disassembling from unaligned addresses manually by setting then in objdump, etc...)
I'm not an expert of the domain, I hope this will help you a bit.
I'm trying to learn about ARM assembler programming using the GNU assembler. I've setup my PC with QEmu and have a Debian ARM-HF chroot environment.
If I assemble and link my test program:
.text
.global _start
_start:
mov r0, #6
bx lr
with:
as test.s -o test.o
ld test.o -o test
Then load the file into gdb and set a breakpoint on _start:
root#Latitude-E6420:/root# gdb test
GNU gdb (GDB) 7.6.1 (Debian 7.6.1-1)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
For bug reporting instructions, please see:
...
Reading symbols from /root/test...(no debugging symbols found)...done.
(gdb) break _start
Breakpoint 1 at 0x8054
(gdb)
How do I single step the code, display the assembler source code and monitor the registers?
I tried some basic commands and they did not work:
(gdb) break _start
Breakpoint 1 at 0x8054
(gdb) info regi
The program has no registers now.
(gdb) stepi
The program is not being run.
(gdb) disas
No frame selected.
(gdb) r
Starting program: /root/test
qemu: Unsupported syscall: 26
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
qemu: Unsupported syscall: 26
During startup program terminated with signal SIGSEGV, Segmentation fault.
(gdb)
Your problem here is that you're trying to run an ARM gdb under QEMU's user-mode emulation. QEMU doesn't support the ptrace syscall (that's what syscall number 26 is), so this is never going to work.
What you need to do is run your test binary under QEMU with the QEMU options to enable QEMU's own builtin gdb stub which will listen on a TCP port. Then you can run a gdb compiled to run on your host system but with support for ARM targets, and tell that to connect to the TCP port.
(Emulating ptrace within QEMU is technically very tricky, and it would not provide much extra functionality that you can't already achieve via the QEMU builtin gdbstub. It's very unlikely it'll ever be implemented.)
Minimal working QEMU user mode example
I was missing the -fno-pie -no-pie options:
sudo apt-get install gdb-multiarch gcc-arm-linux-gnueabihf qemu-user
printf '
#include <stdio.h>
#include <stdlib.h>
int main() {
puts("hello world");
return EXIT_SUCCESS;
}
' > hello_world.c
arm-linux-gnueabihf-gcc -fno-pie -ggdb3 -no-pie -o hello_world hello_world.c
qemu-arm -L /usr/arm-linux-gnueabihf -g 1234 ./hello_world
On another terminal:
gdb-multiarch -q --nh \
-ex 'set architecture arm' \
-ex 'set sysroot /usr/arm-linux-gnueabihf' \
-ex 'file hello_world' \
-ex 'target remote localhost:1234' \
-ex 'break main' \
-ex continue \
-ex 'layout split'
;
This leaves us at main, in a split code / disassembly view due to layout split. You will also interested in:
layout regs
which shows the registers.
At the end of the day however, GDB Dashboard is more flexible and reliable: gdb split view with code
-fno-pie -no-pie is required because the packaged Ubuntu GCC uses -fpie -pie by default, and those fail due to a QEMU bug: How to GDB step debug a dynamically linked executable in QEMU user mode?
There was no gdbserver --multi-like functionality for the QEMU GDB stub on QEMU 2.11: How to restart QEMU user mode programs from the GDB stub as in gdbserver --multi?
For those learning ARM assembly, I am starting some runnable examples with assertions and using the C standard library for IO at: https://github.com/cirosantilli/arm-assembly-cheat
Tested on Ubuntu 18.04, gdb-multiarch 8.1, gcc-arm-linux-gnueabihf 7.3.0, qemu-user 2.11.
Freestanding QEMU user mode example
This analogous procedure also works on an ARM freestanding (no standard library) example:
printf '
.data
msg:
.ascii "hello world\\n"
len = . - msg
.text
.global _start
_start:
/* write syscall */
mov r0, #1 /* stdout */
ldr r1, =msg /* buffer */
ldr r2, =len /* len */
mov r7, #4 /* Syscall ID. */
swi #0
/* exit syscall */
mov r0, #0 /* Status. */
mov r7, #1 /* Syscall ID. */
swi #0
' > hello_world.S
arm-linux-gnueabihf-gcc -ggdb3 -nostdlib -o hello_world -static hello_world.S
qemu-arm -g 1234 ./hello_world
On another terminal:
gdb-multiarch -q --nh \
-ex 'set architecture arm' \
-ex 'file hello_world' \
-ex 'target remote localhost:1234' \
-ex 'layout split' \
;
We are now left at the first instruction of the program.
QEMU full system examples
Linux kernel: How to debug the Linux kernel with GDB and QEMU?
Bare metal: https://github.com/cirosantilli/newlib-examples/tree/f70f8a33f8b727422bd6f0b2975c4455d0b33efa#gdb
Single step of an assembly instruction is done with stepi. disas will disassemble around the current PC. info regi will display the current register state. There are some examples for various processors on my blog for my ELLCC cross development tool chain project.
You should add the -g option too to the assembling. Otherwise the codeline info is not included.
That crash probably comes from running some garbage after the code lines.
Maybe you should add the exit system call:
mov eax, 1 ; exit
mov ebx, 0 ; returm value
int 0x80 ; system call
When I type uname -a, it gives the following output.
Linux mars 2.6.9-67.0.15.ELsmp #1 SMP Tue Apr 22 13:50:33 EDT 2008 i686 i686 i386 GNU/Linux
How can I know from this that the given OS is 32 or 64 bit?
This is useful when writing configure scripts, for example: what architecture am I building for?
Try uname -m. Which is short of uname --machine and it outputs:
x86_64 ==> 64-bit kernel
i686 ==> 32-bit kernel
Otherwise, not for the Linux kernel, but for the CPU, you type:
cat /proc/cpuinfo
or:
grep flags /proc/cpuinfo
Under "flags" parameter, you will see various values: see "What do the flags in /proc/cpuinfo mean?"
Among them, one is named lm: Long Mode (x86-64: amd64, also known as Intel 64, i.e. 64-bit capable)
lm ==> 64-bit processor
Or using lshw (as mentioned below by Rolf of Saxony), without sudo (just for grepping the cpu width):
lshw -class cpu|grep "^ width"|uniq|awk '{print $2}'
Note: you can have a 64-bit CPU with a 32-bit kernel installed.
(as ysdx mentions in his/her own answer, "Nowadays, a system can be multiarch so it does not make sense anyway. You might want to find the default target of the compiler")
If you were running a 64 bit platform you would see x86_64 or something very similar in the output from uname -a
To get your specific machine hardware name run
uname -m
You can also call
getconf LONG_BIT
which returns either 32 or 64
lscpu will list out these among other information regarding your CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
...
Another useful command for easy determination is as below:
Command:
getconf LONG_BIT
Answer:
32, if OS is 32 bit
64, if OS is 64 bit
The command
$ arch
is equivalent to
$ uname -m
but is twice as fast to type
I was wondering about this specifically for building software in Debian (the installed Debian system can be a 32-bit version with a 32 bit kernel, libraries, etc., or it can be a 64-bit version with stuff compiled for the 64-bit rather than 32-bit compatibility mode).
Debian packages themselves need to know what architecture they are for (of course) when they actually create the package with all of its metadata, including platform architecture, so there is a packaging tool that outputs it for other packaging tools and scripts to use, called dpkg-architecture. It includes both what it's configured to build for, as well as the current host. (Normally these are the same though.) Example output on a 64-bit machine:
DEB_BUILD_ARCH=amd64
DEB_BUILD_ARCH_OS=linux
DEB_BUILD_ARCH_CPU=amd64
DEB_BUILD_GNU_CPU=x86_64
DEB_BUILD_GNU_SYSTEM=linux-gnu
DEB_BUILD_GNU_TYPE=x86_64-linux-gnu
DEB_HOST_ARCH=amd64
DEB_HOST_ARCH_OS=linux
DEB_HOST_ARCH_CPU=amd64
DEB_HOST_GNU_CPU=x86_64
DEB_HOST_GNU_SYSTEM=linux-gnu
DEB_HOST_GNU_TYPE=x86_64-linux-gnu
You can print just one of those variables or do a test against their values with command line options to dpkg-architecture.
I have no idea how dpkg-architecture deduces the architecture, but you could look at its documentation or source code (dpkg-architecture and much of the dpkg system in general are Perl).
#include <stdio.h>
int main(void)
{
printf("%d\n", __WORDSIZE);
return 0;
}
If you have a 64-bit OS, instead of i686, you have x86_64 or ia64 in the output of uname -a. In that you do not have any of these two strings; you have a 32-bit OS (note that this does not mean that your CPU is not 64-bit).
That system is 32bit. iX86 in uname means it is a 32-bit architecture. If it was 64 bit, it would return
Linux mars 2.6.9-67.0.15.ELsmp #1 SMP Tue Apr 22 13:50:33 EDT 2008 x86_64 i686 x86_64 x86_64 GNU/Linux
Nowadays, a system can be multiarch so it does not make sense anyway. You might want to find the default target of the compiler:
$ cc -v 2>&1 | grep ^Target
Target: x86_64-pc-linux-gn
You can try to compile a hello world:
$ echo 'int main() { return 0; }' | cc -x c - -o foo
$ file foo
foo: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=b114e029a08abfb3c98db93d3dcdb7435b5bba0c, not stripped
With respect to the answer "getconf LONG_BIT".
I wrote a simple function to do it in 'C':
/*
* check_os_64bit
*
* Returns integer:
* 1 = it is a 64-bit OS
* 0 = it is NOT a 64-bit OS (probably 32-bit)
* < 0 = failure
* -1 = popen failed
* -2 = fgets failed
*
* **WARNING**
* Be CAREFUL! Just testing for a boolean return may not cut it
* with this (trivial) implementation! (Think of when it fails,
* returning -ve; this could be seen as non-zero & therefore true!)
* Suggestions?
*/
static int check_os_64bit(void)
{
FILE *fp=NULL;
char cb64[3];
fp = popen ("getconf LONG_BIT", "r");
if (!fp)
return -1;
if (!fgets(cb64, 3, fp))
return -2;
if (!strncmp (cb64, "64", 3)) {
return 1;
}
else {
return 0;
}
}
Good idea, the 'getconf'!
You can also check using a environment variable:
echo $HOSTTYPE
Result:
i386 -> 32 bits
x86_64 -> 64 bits
Extracted from: http://www.sysadmit.com/2016/02/linux-como-saber-si-es-32-o-64-bits.html
In Bash, using integer overflow:
if ((1 == 1<<32)); then
echo 32bits
else
echo 64bits
fi
It's much more efficient than invoking another process or opening files.
getconf uses the fewest system calls:
$ strace getconf LONG_BIT | wc -l
253
$ strace arch | wc -l
280
$ strace uname -m | wc -l
281
$ strace grep -q lm /proc/cpuinfo | wc -l
301
If you shift 1 left by 32 and you get 1, your system is 32 bit.
If you shift 1 left by 64 and you get 1, your system is 64 bit.
In other words,
if echo $((1<<32)) gives 1 then your system is 32 bit.
if echo $((1<<64)) gives 1 then your system is 64 bit.
If one is severely limited in available binaries (e.g. in initramfs), my colleagues suggested:
$ ls -l /lib*/ld-linux*.so.2
On my ALT Linux systems, i586 has /lib/ld-linux.so.2 and x86_64 has /lib64/ld-linux-x86-64.so.2.
$ grep "CONFIG_64" /lib/modules/*/build/.config
# CONFIG_64BIT is not set
I can't believe that in all this time, no one has mentioned:
sudo lshw -class cpu
to get details about the speed, quantity, size and capabilities of the CPU hardware.
Simple script to get 64 bit or 32 bit
if $(getconf LONG_BIT | grep '64'); then
echo "64 bit system"
else
echo "32 bit system"
fi
[ -z `uname -m | grep 64` ] && echo "32-bit" || echo "64-bit"
Based on the fact that 64-bit is usually x86_64 and 32-bit is i686 etc.
First you have to download Virtual Box. Then select new and a 32-bit Linux. Then boot the linux using it. If it boots then it is 32 bit if it doesn't then it is a 64 bit.